Deploy Sharded Cluster with Keyfile Authentication

Deploy Sharded Cluster with Keyfile Authentication

Overview

Enforcing access control on a sharded cluster requires configuring:

Security between components of the cluster usingInternal Authentication.
Security between connecting clients and the cluster usingUser Access Controls.

For this tutorial, each member of the sharded cluster must use the sameinternal authentication mechanism and settings. This means enforcing internalauthentication on each mongos and mongod in the cluster.

The following tutorial uses a keyfile toenable internal authentication.

Enforcing internal authentication also enforces user access control. Toconnect to the replica set, clients like the mongo shell need touse a user account. SeeAccess Control.

CloudManager and OpsManager

If you are using Cloud Manager or Ops Manager to manage your deployment,see the respective Cloud Manager manualor the Ops Manager manual to enforce authentication.

Considerations

Tip

When possible, use a logical DNS hostname instead of an ip address,particularly when configuring replica set members or sharded clustermembers. The use of logical DNS hostnames avoids configurationchanges due to ip address changes.

IP Binding

Changed in version 3.6.

Starting with MongoDB 3.6, MongoDB binaries, mongod andmongos, bind to localhost by default.From MongoDB versions 2.6 to 3.4, only the binaries from theofficial MongoDB RPM (Red Hat, CentOS, Fedora Linux, and derivatives)and DEB (Debian, Ubuntu, and derivatives) packages would bind tolocalhost by default. To learn more about this change, seeLocalhost Binding Compatibility Changes.

Keyfile Security

Keyfiles are bare-minimum forms of security and are best suited for testing ordevelopment environments. For production environments we recommend usingx.509 certificates.

Access Control

This tutorial covers creating the minimum number of administrativeusers on the admin database only. For the user authentication,the tutorial uses the default SCRAMauthentication mechanism. Challenge-response security mechanisms arebest suited for testing or development environments. For productionenvironments, we recommend using x.509certificates or LDAP Proxy Authentication(available for MongoDB Enterprise only) or Kerberos Authentication(available for MongoDB Enterprise only).

For details on creating users for specific authentication mechanism,refer to the specific authentication mechanism pages.

See Configure Role-Based Access Control for bestpractices for user creation and management.

Users

In general, to create users for a sharded clusters, connect to themongos and add the sharded cluster users.

However, some maintenance operations require direct connections tospecific shards in a sharded cluster. To perform these operations, youmust connect directly to the shard and authenticate as a shard-localadministrative user.

Shard-local users exist only in the specific shard and should only beused for shard-specific maintenance and configuration. You cannotconnect to the mongos with shard-local users.

This tutorial requires creating sharded cluster users, but includesoptional steps for adding shard-local users.

See the Users security documentation for moreinformation.

Operating System

This tutorial uses the mongod and mongosprograms. Windows users should use the mongod.exe andmongos.exe programs instead.

Deploy Sharded Cluster with Keyfile Access Control

The following procedures involve creating a new sharded cluster that consistsof a mongos, the config servers, and two shards.

Tip

Create the Keyfile

With keyfile authentication, eachmongod or mongos instances in the sharded cluster uses the contents of the keyfile as theshared password for authenticating other members in the deployment. Onlymongod or mongos instances with the correct keyfile can join the sharded cluster.

Note

Starting in MongoDB 4.2, keyfiles for internal membershipauthentication use YAML format to allow formultiple keys in a keyfile. The YAML format accepts content of:

a single key string (same as in earlier versions),
multiple key strings (each string must be enclosed in quotes), or
sequence of key strings.

The YAML format is compatible with the existing single-keykeyfiles that use the text file format.

A key’s length must be between 6 and 1024 characters and may onlycontain characters in the base64 set. All members of thesharded cluster must share at least one common key.

Note

On UNIX systems, the keyfile must not have group or worldpermissions. On Windows systems, keyfile permissions are not checked.

You can generate a keyfile using any method you choose. For example,the following operation uses openssl to generate a complexpseudo-random 1024 character string to use as a shared password. It thenuses chmod to change file permissions to provide readpermissions for the file owner only:

openssl rand -base64 756 > <path-to-keyfile>
chmod 400 <path-to-keyfile>

See Keyfiles for additional details and requirementsfor using keyfiles.

Distribute the Keyfile

Copy the keyfile to each server hosting the sharded cluster members.Ensure that the user running the mongod or mongos instances is the owner of thefile and can access the keyfile.

Avoid storing the keyfile on storage mediums that can be easilydisconnected from the hardware hosting the mongod or mongos instances, such as aUSB drive or a network attached storage device.

Create the Config Server Replica Set

The following steps deploys a config server replica set.

For a production deployment, deploys a config server replica set with atleast three members. For testing purposes, you can create asingle-member replica set.

Start eachmongod in the config server replica set.Include the keyFile setting. The keyFile settingenforces both Internal/Membership Authentication andRole-Based Access Control.

You can specify the mongod settings either via aconfiguration file or the command line.

Configuration File

If using a configuration file, set security.keyFile to thekeyfile’s path, sharding.clusterRole to configsvr,and replication.replSetName to the desired name of theconfig server replica set.

security:
  keyFile: <path-to-keyfile>
sharding:
  clusterRole: configsvr
replication:
  replSetName: <setname>

Include additional options as requiredfor your configuration. For instance, if you wish remote clients toconnect to your deployment or your deployment members are run ondifferent hosts, specify the net.bindIp setting. For moreinformation, see Localhost Binding Compatibility Changes.

Start the mongod specifying the —config option and thepath to the configuration file.

mongod --config <path-to-config-file>

Command Line

If using the command line parameters, start the mongod withthe —keyFile, —configsvr, and —replSet parameters.

mongod --keyFile <path-to-keyfile> --configsvr --replSet <setname> --dbpath <path>

Include additional options as required for your configuration. Forinstance, if you wish remote clients to connect to your deploymentor your deployment members are run on different hosts, specify the—bind_ip. For more information, seeLocalhost Binding Compatibility Changes.

Connect to a member of the replica set over the localhost interface.

Connect a mongo shell to one of themongod instances over the localhostinterface. You must run the mongoshell on the same physical machine as the mongod instance.

The localhost interface is only availablesince no users have been created for the deployment. Thelocalhost interface closes after thecreation of the first user.

The rs.initiate() method initiates the replica set and cantake an optional replica set configuration document. In the replica setconfiguration document, include:

The _id. The _idmust match the —replSetparameter passed to the mongod.
The members field. The members field is anarray and requires a document per each member of the replica set.
The configsvr field. The configsvr field mustbe set to true for the config server replica set.

See Replica Set Configuration for more information onreplica set configuration documents.

Initiate the replica set using the rs.initiate() methodand a configuration document:

rs.initiate(
  {
    _id: "<replSetName>",
    configsvr: true,
    members: [
      { _id : 0, host : "cfg1.example.net:27019" },
      { _id : 1, host : "cfg2.example.net:27019" },
      { _id : 2, host : "cfg3.example.net:27019" }
    ]
  }
)

Once the config server replica set (CSRS) is initiated and up, proceedto creating the shard replica sets.

Create the Shard Replica Sets

For a production deployment, use a replica set with at least threemembers. For testing purposes, you can create a single-member replicaset.

These steps include optional procedures for adding shard-local users.Executing them now ensures that there are users available for eachshard to perform shard-level maintenance.

Start each member of the replica set with access control enabled.

Running a mongod with the keyFile parameter enforces bothInternal/Membership Authentication andRole-Based Access Control.

Start eachmongod in the replica set using eithera configuration file or the command line.

Configuration File

If using a configuration file, set the security.keyFile optionto the keyfile’s path, the replication.replSetName to thedesired name of the replica set, and the sharding.clusterRoleoption to shardsvr.

security:
  keyFile: <path-to-keyfile>
sharding:
  clusterRole: shardsvr
replication:
  replSetName: <replSetName>
storage:
   dbPath: <path>

Start the mongod specifying the —config optionand the path to the configuration file.

mongod --config <path-to-config-file>

Command Line

If using the command line option, when starting the component, specifythe —keyFile, replSet, and —shardsvr parameters, as in thefollowing example:

mongod --keyFile <path-to-keyfile> --shardsvr --replSet <replSetName>  --dbpath <path>

Include additional options as required for your configuration. Forinstance, if you wish remote clients to connect to your deploymentor your deployment members are run on different hosts, specify the—bind_ip. For more information, seeLocalhost Binding Compatibility Changes.

Connect to a member of the replica set over the localhost interface.

Connect a mongo shell to one of themongod instances over the localhostinterface. You must run the mongoshell on the same physical machine as the mongod instance.

The localhost interface is only availablesince no users have been created for the deployment. Thelocalhost interface closes after thecreation of the first user.

Initiate the replica set.

From the mongo shell, run the rs.initiate() method.

rs.initiate() can take an optional replica setconfiguration document. In thereplica set configuration document, include:

The _id field set to the replica set name specified ineither the replication.replSetName or the —replSetoption.
The members array with a document per each member of thereplica set.

The following example initates a three member replica set.

rs.initiate(
  {
    _id : <replicaSetName>,
    members: [
      { _id : 0, host : "s1-mongo1.example.net:27018" },
      { _id : 1, host : "s1-mongo2.example.net:27018" },
      { _id : 2, host : "s1-mongo3.example.net:27018" }
    ]
  }
)

rs.initiate() triggers an election andelects one of the members to be the primary.

Connect to the primary before continuing. Use rs.status() tolocate the primary member.

Create the shard-local user administrator (optional).

Important

After you create the first user, the localhost exception is no longer available.

The first user must have privileges to create other users, suchas a user with the userAdminAnyDatabase. This ensuresthat you can create additional users after the Localhost Exceptioncloses.

If at least one user does not have privileges to create users,once the localhost exception closes you may be unable to createor modify users with new privileges, and therefore unable toaccess necessary operations.

Add a user using the db.createUser() method. The user shouldhave at minimum the userAdminAnyDatabase role on theadmin database.

You must be connected to the primary to create users.

The following example creates the user fred with theuserAdminAnyDatabase role on the admin database.

Important

Passwords should be random, long, and complex to ensure system securityand to prevent or delay malicious access.

Tip

Starting in version 4.2 of the mongo shell, you canuse the passwordPrompt() method in conjunction withvarious user authentication/management methods/commands to promptfor the password instead of specifying the password directly in themethod/command call. However, you can still specify the passworddirectly as you would with earlier versions of themongo shell.

admin = db.getSiblingDB("admin")
admin.createUser(
  {
    user: "fred",
    pwd: passwordPrompt(), // or cleartext password
    roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
  }
)

Enter the password when prompted.See Database User Roles for a full list of built-in roles andrelated to database administration operations.

Authenticate as the shard-local user administrator (optional).

Authenticate to the admin database.

In the mongo shell, use db.auth() toauthenticate. For example, the following authenticate as the useradministrator fred:

Tip

db.getSiblingDB("admin").auth("fred", passwordPrompt()) // or cleartext password

Alternatively, connect a new mongo shell to the primaryreplica set member using the -u <username>, -p <password>, andthe —authenticationDatabase parameters.

mongo -u "fred" -p  --authenticationDatabase "admin"

If you do not specify the password to the -pcommand-line option, the mongo shell prompts for thepassword.

Create the shard-local cluster administrator (optional).

The shard-local cluster administrator user has theclusterAdmin role, which provides privileges that allowaccess to replication operations.

For a full list of roles related to replica set operations seeCluster Administration Roles.

Create a cluster administrator user and assign theclusterAdmin role in the admin database:

Tip

db.getSiblingDB("admin").createUser(
  {
    "user" : "ravi",
    "pwd" : passwordPrompt(),     // or cleartext password
    roles: [ { "role" : "clusterAdmin", "db" : "admin" } ]
  }
)

Enter the password when prompted.

See Cluster Administration Roles for a full list of built-in roles related toreplica set and sharded cluster operations.

Connect a mongos to the Sharded Cluster

Connect a mongos to the cluster

Start a mongos specifyingthe keyfile using either a configuration file or a command line parameter.

Configuration File

If using a configuration file, set thesecurity.keyFile to the keyfile’s path and thesharding.configDB to the replica set name and at leastone member of the replica set in <replSetName>/<host:port>format.

security:
  keyFile: <path-to-keyfile>
sharding:
  configDB: <configReplSetName>/cfg1.example.net:27019,cfg2.example.net:27019,...

Start the mongos specifying the —config option and thepath to the configuration file.

mongos --config <path-to-config>

Command Line

If using command line parameters start the mongos and specifythe —keyFile and —configdb parameters.

mongos --keyFile <path-to-keyfile> --configdb <configReplSetName>/cfg1.example.net:27019,cfg2.example.net:27019,...

Include additional options as required for your configuration. Forinstance, if you wish remote clients to connect to your deploymentor your deployment members are run on different hosts, specify the—bind_ip. For more information, seeLocalhost Binding Compatibility Changes.

Connect to a mongos over the localhost interface.

Connect a mongo shell to one of themongos instances over the localhostinterface. You must run the mongoshell on the same physical machine as the mongos instance.

The localhost interface is only availablesince no users have been created for the deployment. Thelocalhost interface closes after thecreation of the first user.

Create the user administrator.

Important

After you create the first user, the localhost exception is no longer available.

The first user must have privileges to create other users, suchas a user with the userAdminAnyDatabase. This ensuresthat you can create additional users after the Localhost Exceptioncloses.

If at least one user does not have privileges to create users,once the localhost exception closes you cannot createor modify users, and therefore may be unable toperform necessary operations.

Add a user using the db.createUser() method. The user shouldhave at minimum the userAdminAnyDatabase role on theadmin database.

Important

Passwords should be random, long, and complex to ensure system securityand to prevent or delay malicious access.

The following example creates the user fred on theadmin database:

Tip

admin = db.getSiblingDB("admin")
admin.createUser(
  {
    user: "fred",
    pwd:  passwordPrompt(),     // or cleartext password
    roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
  }
)

See Database User Roles for a full list of built-in roles andrelated to database administration operations.

Authenticate as the user administrator.

Use db.auth() to authenticate as the user administratorto create additional users:

Tip

db.getSiblingDB("admin").auth("fred", passwordPrompt()) // or cleartext password

Enter the password when prompted.

Alternatively, connect a new mongo shell to the targetreplica set member using the -u <username>, -p <password>, andthe —authenticationDatabase "admin" parameters. You must usethe Localhost Exception to connect to the mongos.

mongo -u "fred" -p  --authenticationDatabase "admin"

If you do not specify the password to the -pcommand-line option, the mongo shell prompts for thepassword.

Create Administrative User for Cluster Management

The cluster administrator user has the clusterAdmin role,which grants access to replication and sharding operations.

Create a clusterAdmin user in the admin database.

The following example creates the user ravi on the admindatabase.

Important

Passwords should be random, long, and complex to ensure systemsecurity and to prevent or delay malicious access.

Tip

db.getSiblingDB("admin").createUser(
  {
    "user" : "ravi",
    "pwd" : passwordPrompt(),     // or cleartext password
    roles: [ { "role" : "clusterAdmin", "db" : "admin" } ]
  }
)

See Cluster Administration Roles for a full list of built-in roles related toreplica set and sharded cluster operations.

Create additional users (Optional).

Create users to allow clients to connect and access thesharded cluster. See Database User Roles for available built-inroles, such as read and readWrite.You may also want additional administrative users.For more information on users, see Users.

To create additional users, you must authenticate as a user withuserAdminAnyDatabase or userAdmin roles.

Add Shards to the Cluster

To proceed, you must be connected to the mongos andauthenticated as the cluster administrator user for the sharded cluster.

Note

This is the cluster administrator for the sharded cluster and _not_the shard-local cluster administrator.

To add each shard to the cluster, use the sh.addShard()method. If the shard is a replica set, specify the name of the replicaset and specify a member of the set. In production deployments, _all_shards should be replica sets.

The following operation adds a single shard replica set to the cluster:

sh.addShard( "<replSetName>/s1-mongo1.example.net:27017")

The following operation is an example of adding a standalone mongodshard to the cluster:

sh.addShard( "s1-mongo1.example.net:27017")

Repeat these steps until the cluster includes all shards. At thispoint, the sharded cluster enforces access control for the cluster aswell as for internal communications between each sharded clustercomponent.

Enable Sharding for a Database

To proceed, you must be connected to the mongos andauthenticated as the cluster administrator user for the sharded cluster.

Note

This is the cluster administrator for the sharded cluster and _not_the shard-local cluster administrator.

Enabling sharding on a database makes it possible to shard collectionswithin the database. Use the sh.enableSharding() method toenable sharding on the target database.

sh.enableSharding("<database>")

Shard a Collection

To proceed, you must be connected to the mongos andauthenticated as the cluster administrator user for the sharded cluster.

Note

This is the cluster administrator for the sharded cluster and _not_the shard-local cluster administrator.

To shard a collection, use the sh.shardCollection() method.You must specify the full namespace of the collection and a document containingthe shard key.

Your selection of shard key affects the efficiency of sharding, as well asyour ability to take advantage of certain sharding features such aszones. See the selection considerations listedin the Choosing a Shard Key.

If the collection already contains data, you must create an index on theshard key using the db.collection.createIndex() method beforeusing shardCollection().

If the collection is empty, MongoDB creates the index as part ofsh.shardCollection().

The following is an example of the sh.shardCollection() method:

sh.shardCollection("<database>.<collection>", { <key> : <direction> } )

Next Steps

Create users to allow clients to connect to and interact withthe sharded cluster.

See Database User Roles for basic built-in roles to use in creatingread-only and read-write users.

x.509 Internal Authentication

For details on using x.509 for internal authentication, seeUse x.509 Certificate for Membership Authentication.

To upgrade from keyfile internal authentication to x.509 internalauthentication, seeUpgrade from Keyfile Authentication to x.509 Authentication.