Compatibility Changes in MongoDB 4.2

Removal of MMAPv1 Storage Engine

MongoDB 4.2 removes support for the deprecated MMAPv1 storage engine.

If your 4.0 deployment uses MMAPv1, you must change the deployment toWiredTiger Storage Engine before upgrading to MongoDB 4.2. For details,see:

• Change Standalone to WiredTiger
• Change Replica Set to WiredTiger
• Change Sharded Cluster to WiredTiger

MMAPv1 Specific Configuration Options

MongoDB removes the following MMAPv1 specific configurationoptions:

Note

Starting in version 4.2, MongoDB processes will not start withthese options. Remove any MMAPv1 specific configurationoptions if using a WiredTiger deployment.

MMAPv1 Specific Parameters

MongoDB removes the following MMAPv1 parameters:

• newCollectionsUsePowerOf2Sizes
• replIndexPrefetch

MMAPv1 Specific Command

MongoDB removes the MMAPv1 specific touch command.

MMAPv1 Specific Options for Commands and Methods

MongoDB removes the MMAPv1 specific options:

• noPadding and usePowerOf2Sizes for collMod
• verbose for collStats
• flags for create
• paddingFactor, paddingBytes, preservePadding fordb.createCollection().

Removed or Deprecated Commands and Methods

Remove Support for the group Command

Starting in version 4.2, MongoDB removes the group command(deprecated since version 3.4) and its mongo shellhelper db.collection.group().

Use db.collection.aggregate() with the $groupstage instead.

Remove Support for the eval Command

Starting in version 4.2, MongoDB removes the eval command. evalcommand has been deprecated since version 3.0.

The associated MongoDB 4.2 mongo shell methodsdb.eval() and db.collection.copyTo() can only berun when connected to MongoDB 4.0 or earlier.

Remove Support for the copydb and clone Commands

Starting in version 4.2, MongoDB removes the deprecated copydbcommand and clone command.

The corresponding mongo shell helpersdb.copyDatabase() and db.cloneDatabase() can only be run whenconnected to MongoDB 4.0 or earlier.

As alternatives, users can use mongodump andmongorestore (with the mongorestore options—nsFrom and —nsTo) or write a script using the drivers.

For example, to copy the test database from a local instancerunning on the default port 27017 to the examples database on thesame instance, you can:

• Use mongodump to dump the test database toan archive mongodump-test-db:

mongodump --archive="mongodump-test-db" --db=test

• Use mongorestore with —nsFrom and —nsTo to restore (with database name change) from thearchive:

mongorestore --archive="mongodump-test-db" --nsFrom='test.*' --nsTo='examples.*'

Tip

Include additional options as necessary, such as to specifythe uri or host, username, password and authenticationdatabase.

Alternatively, instead of using an archive file, you canmongodump the test database to the standardoutput stream and pipe into mongorestore:

mongodump --archive --db=test | mongorestore --archive  --nsFrom='test.*' --nsTo='examples.*'

Remove Support for the parallelCollectionScan Command

Starting in version 4.2, MongoDB removes the parallelCollectionScancommand.

Remove maxScan

MongoDB removes the deprecated option maxScan for thefind command and the mongo shell helpercursor.maxScan(). Use either the maxTimeMS option for thefind command or the helper cursor.maxTimeMS() instead.

Remove Support for the geoNear Command

Starting in version 4.2, MongoDB removes the geoNear command. Usethe $geoNear aggregation stage instead.

The options for $geoNear are similar to the removedgeoNear command with the following exceptions:

• The removed geoNear command includes in its output a field nameddis that included the distance information.

For the $geoNear stage, specify the distance field namein distanceField.

• The removed geoNear command accepts a boolean value for theincludeLocs option to include the loc field.

For the $geoNear stage, specify the location field namein includeLocs.

• The removed geoNear command includes the avgDistance andmaxDistance of the returned results.

You can use the aggregation pipeline to return the avgDistanceand maxDistance as well. Specifically, after the$geoNear stage, include a $group stage tocalculate the avgDistance and maxDistance:

db.places.aggregate([
   { $geoNear: { near: <...>, distanceField: "dis", includeLocs: "loc", spherical: true, ... } },
   { $group: { _id: null, objectsLoaded: { $sum: 1 }, maxDistance:
         { $max: "$dis" }, avgDistance: { $avg: "$dis" } } }
])

See also

Remove limit and num Options from $geoNear

Remove Support for the repairDatabase Command

Starting in version 4.2, MongoDB removes the repairDatabase commandand its mongo shell helper db.repairDatabase() aswell as the repairDatabase privilege.

As alternatives:

• To compact data for a mongod, use thecompact command. For details about the operation, seecompact command.
• To rebuild indexes on a standalone, use the reIndexcommand or its helper db.collection.reIndex(). For detailsabout the operation, see reIndex command anddb.collection.reIndex() reference pages.
• To recover data for a standalone, use mongod —repair. SeeRecover a Standalone after an Unexpected Shutdown fordetails.

Deprecate Support for cloneCollection

MongoDB deprecates the cloneCollection command and itsmongo shell helper db.cloneCollection()

As alternatives,

• Users can use mongoexport andmongoimport or write a script using the drivers.

Deprecated Plan Cache Commands/Methods

MongoDB deprecates the following:

• PlanCache.getPlansByQuery()method/planCacheListPlans command.

To get the cached query plans for a shape, use the$planCacheStats aggregation stage instead. SeeFind Cache Entry Details for a Query Shape.

• PlanCache.listQueryShapes()method/planCacheListQueryShapes command.

To list the cached query shapes, use the$planCacheStats aggregation stage instead. SeeList Query Shapes.

Aggregation

$out Stage Restrictions

$out and Views

The view definition pipeline cannot includethe $out stage. If you already have an existing viewthat includes the $out stage, you can no longer createnew views from this existing view.

For existing views that include the $out stage, youshould either drop andrecreate the views without the$out stage or use replace the viewdefinition with a new pipeline that does not contain the$out stage.

$out and $lookup

The $lookup stage cannot include the $outstage in its nested pipeline field for the joined collection.

$out and linearizable Read Concern Level

The $out stage cannot be used in conjunction with readconcern "linearizable".

$out and Explain

You cannot run the db.collection.explain() method (or theexplain command) in executionStats mode orallPlansExecution mode if the aggregation pipeline contains the$out stage.

If the aggregation pipeline contains the $out stage, toview executionStats or allPlansExecution information, runexplain without the $out stage in order to returnexplain results for the preceding stages.

Alternatively, you can run explain in queryPlanner mode for anaggregation pipeline that contains the $out stage.

$out and majority Read Concern Level

Starting in MongoDB 4.2, you can specify read concern level "majority" for anaggregation that includes an $out stage.

In MongoDB 4.0 and earlier, you cannot include the $outstage to use "majority" read concern for the aggregation.

Remove limit and num Options from $geoNear

Starting in version 4.2, MongoDB removes the limit and numoptions for the $geoNear stage as well as the defaultlimit of 100 documents. To limit the results of$geoNear, use the $geoNear stage with the$limit stage.

For example, the following aggregation where the $geoNearstage contains the num option is no longer valid in 4.2.

db.places.aggregate([
   {
     $geoNear: {
        near: { type: "Point", coordinates: [ -73.99279 , 40.719296 ] },
        distanceField: "distance",
        num: 5,                 // Not supported in 4.2
        spherical: true
     }
   }
])

Instead, you can rewrite the aggregation to the following pipeline:

db.places.aggregate([
   {
     $geoNear: {
        near: { type: "Point", coordinates: [ -73.99279 , 40.719296 ] },
        distanceField: "distance",
        spherical: true
     }
   },
   { $limit: 5 }
])

See also

Aggregation Improvements

Transactions

• Starting in MongoDB 4.2, you cannot specify killCursors asthe first operation in a transaction.
• Starting in MongoDB 4.2, you cannot write to capped collections in transactions. Reads from capped collections are still supportedin transactions.
• Starting in MongoDB 4.2, MongoDB removes the 16MB total sizelimit for a transaction. In version 4.2, MongoDB creates as manyoplog entries as necessary to the encapsulate all write operations in atransaction. In previous versions, MongoDB creates a single entry forall write operations in a transaction, thereby imposing a 16MBtotal size limit for a transaction.

Change Streams

Availability

Starting in MongoDB 4.2, change streams areavailable regardless of the "majority" read concernsupport; that is, read concern majority support can be eitherenabled (default) or disabledto use change streams.

In MongoDB 4.0 and earlier, change streams areavailable only if "majority" read concern support isenabled (default).

See also

4.2 Changes to Change Streams

Default Collation

Starting in MongoDB 4.2, change streams use simple binarycomparisons unless an explicit collation is provided. In earlierversions, change streams opened on a single collection(db.collection.watch()) would inherit that collection’sdefault collation.

Resume Token Modification

Starting in MongoDB 4.2, change streams will throw an exception ifthe change stream aggregation pipeline modifies an event’s _id field.

MongoDB Tools

FIPS Mode

Starting in version 4.2, MongoDB removes the —sslFIPSModeoption for the following programs:

• mongodump
• mongoexport
• mongofiles
• mongoimport
• mongorestore
• mongostat
• mongotop

The programs will use FIPS compliant connections tomongod/mongos if themongod/mongos instances areconfigured to use FIPS mode.

Extended JSON v2

Starting in version 4.2:

For details on MongoDB extended JSON v2, seeMongoDB Extended JSON (v2).

—query Options

Starting in version 4.2, the query option for mongodump and mongoexportmust be in Extended JSON v2 format (relaxed or canonical/strictmode), including enclosing thefield names and the operators in quotes, as in the following:

mongoexport -d=test -c=records -q='{ "a": { "$gte": 3 }, "date": { "$lt": { "$date": "2016-01-01T00:00:00.000Z" } } }' --out=exportdir/myRecords.json

In earlier versions, the query options uses the Extended JSON v1format and the field names andthe operators do not need to be in quotes:

mongoexport -d=test -c=records -q='{ a: { $gte: 3 }, date: { $lt: { "$date": "2016-01-01T00:00:00.000Z" } } }' --out=exportdir/myRecords.json

General Changes

Indexes

Stronger Restrictions on reIndex

MongoDB implements a stronger restriction on runningreIndex command and db.collection.reIndex()shell helper against a collection in a sharded cluster by disallowingreIndex and db.collection.reIndex() on amongos.

Restriction on db.collection.dropIndex() Method

You cannot specify db.collection.dropIndex("*") to drop all non-_id indexes. Usedb.collection.dropIndexes() instead.

Duplicate Index Creation Attempt Error Message

MongoDB changes the returned response if you create an index with onename, and then try to create the index again with another name.

Starting in version 4.2, the createIndexes command and themongo shell helpersdb.collection.createIndex() anddb.collection.createIndexes() report an error if you createan index with one name, and then try to create the same index again butwith another name.

{
   "ok" : 0,
   "errmsg" : "Index with name: x_1 already exists with a different name",
   "code" : 85,
   "codeName" : "IndexOptionsConflict"
}

In previous versions, MongoDB did not create the index again, butwould return a response object with ok value of 1 and a notethat implied that the index was not recreated. For example:

{
   "numIndexesBefore" : 2,
   "numIndexesAfter" : 2,
   "note" : "all indexes already exist",
   "ok" : 1
}

Hashed Index on PowerPC

For hashed indexes, MongoDB 4.2 ensuresthat the hashed value for the floating point value 2⁶³ onPowerPC is consistent with other platforms. In previous versions, thehashed value for the floating point value 2⁶³ on PowerPC isinconsistent with other platforms.

Although hashed indexes on a field that maycontain floating point values greater than 2⁵³ is anunsupported configuration, clients may still insert documents where theindexed field has the value 2⁶³.

To list all hashed indexes for your deployment, seePowerPC and 263.

If the current MongoDB 4.0 sharded cluster on PowerPC contains hashedvalues for 2⁶³ as part of the shard key, additionalconsiderations must be taken before upgrading the sharded cluster to4.2. See Upgrade a Sharded Cluster to 4.2.

min()/max()

Starting in MongoDB 4.2, when specifyingmin()/max() for adb.collection.find() operation, you must explicitly specifythe index for min()/max() with thecursor.hint() method unless thefind() query is an equality condition on the_id field { _id: <value> }.

Similarly, when specifying min/max in the findcommand, you must also explicitly specify the hint for themin/max index.

In previous versions, you could runmin()/max() (or the correspondingmin/max fields in the command) with or without explicitlyhinting the index regardless of the query condition. If run without thehint in 4.0 and earlier, MongoDB selects the index using the fields inthe indexBounds; however, if multiple indexes exist on same fieldswith different sort orders, the selection of the index may be ambiguous.

CurrentOp

• When reporting on "getmore" operations, the$currentOp aggregation stage, along withcurrentOp command and the db.currentOp()helper, now returns the originatingCommand field as a nestedfield in the new cursor field. In previous versions, theoriginatingCommand was a top-level field for the associated"getmore" document. See also 4.2 currentOp Changes.

Server Status

• The serverStatus and db.serverStatus methodreturn opcounters and opcountersReplmetrics as 64-bit integers (i.e. NumberLong) instead of 32-bitintegers (i.e. NumberInt).

Logging

• When logging to syslog, the format of themessage text includes the component.For example:

...  ACCESS   [repl writer worker 5] Unsupported modification to roles collection ...

Previously, the syslog message text didnot include the component. Forexample:

... [repl writer worker 1] Unsupported modification to roles collection ...

• Starting in MongoDB 4.2, the getLog command truncatesany event that contains more than 1024 characters. In earlierversions, getLog truncates after 512 characters.

• Starting in version 4.2, MongoDB logs the debug verbosity level. For example, if verbosity levelis 2, MongoDB logs D2.

In previous versions, MongoDB log messages only specified D forDebug level.

Wire Protocol

• MongoDB no longer supports the deprecated internal OP_COMMAND and thecorresponding OP_COMMANDREPLY wire protocol.

Replica Set State Changes

Primary Step Down

Starting in MongoDB 4.2, replSetStepDown (andreplSetReconfig that results in a step down) no longercloses all client connections. However, writes that were in progressare killed.

In MongoDB 4.0 and earlier, replSetStepDown closes allclient connections during the step down.

ROLLBACK State

Starting in version 4.2, MongoDB kills all in-progress useroperations when a member enters the ROLLBACK state.

killCursors Changes

Transactions

Starting in MongoDB 4.2, you cannot specify killCursors asthe first operation in a transaction.

Privileges

Starting in MongoDB 4.2, users can always kill their own cursors,regardless of whether the users have the privilege tokillCursors. As such, the killCursorsprivilege has no effect in MongoDB 4.2+.

In MongoDB 3.6.3 through MongoDB 4.0.x, users required thekillCursors privilege in order to kill their own cursorswhen access control is enabled.

Removes AsyncRequestsSenderUseBaton Parameter

In MongoDB 4.2+ deployment, MongoDB removes theAsyncRequestsSenderUseBaton parameter and always enables theperformance enhancement controlled by the parameter.

Stricter Validation of count Syntax

Starting in version 4.2, MongoDB implements a stricter validation ofthe option names for the count command. The command nowerrors if you specify an unknown option name.

In previous versions, MongoDB ignores invalid option names.

Causal Consistency Sessions

Starting in MongoDB 4.2, the following commands no longer support afterClusterTime:

• dbHash command
• mapReduce command
• validate command

As such, these operations cannot be associated with causallyconsistent sessions.

Removes fastmodinsert Metric

MongoDB 4.2 removes the deprecated fastmodinsert metric fromvarious outputs, including the explain executionStats, the profileroutput, etc.

Map-Reduce

Starting in version 4.2, MongoDB deprecates:

• The map-reduce option to create a new sharded collection as wellas the use of the sharded option formap-reduce. To output to a sharded collection, create the shardedcollection first. MongoDB 4.2 also deprecates the replacement ofan existing sharded collection.
• The explicit specification of nonAtomic: false option.

Balancer State and Autosplit

Starting in MongoDB 4.2:

• The balancerStart command and themongo shell helper methodssh.startBalancer() andsh.setBalancerState(true) alsoenable auto-splitting for the sharded cluster.

To disable auto-splitting when the balancer is enabled, you canuse sh.disableAutoSplit().

• The balancerStop command and the mongoshell helper methods sh.stopBalancer() andsh.setBalancerState(false) alsodisable auto-splitting for the sharded cluster.

To enable auto-splitting when the balancer is disabled, you canuse sh.enableAutoSplit().

The mongo methodssh.enableBalancing(namespace) andsh.disableBalancing(namespace) have no affect on theauto-splitting.

Lock Diagnostics Reporting

Starting in version 4.2, MongoDB reports onReplicationStateTransition lock information.

In addition, MongoDB 4.2 separates ParallelBatchWriterMode lockinformation from Global lock information. Earlier MongoDB versionsreport ParallelBatchWriterMode lock information as partof Global locks.

For operations that report on lock information, see:

• serverStatus command and db.serverStatus() method.
• $currentOp aggregation pipeline stage, currentOp command, and db.currentOp() method.

findAndModify Query/Sort/Projection Argument Validation

Starting in MongoDB 4.2 (and 4.0.12+ and 3.6.14+), thefindAndModify command and its associatedmongo shell methods error if the specified query,sort, or projection argument is not a document.

In earlier versions, the operation treated non-document query orsort argument as an empty document {}.

See:

• findAndModify
• db.collection.findOneAndDelete()
• db.collection.findOneAndReplace()
• db.collection.findOneAndUpdate()
• db.collection.findAndModify()

dropDatabase and movePrimary

Starting in MongoDB 4.2,

• If you drop a database and create a new database with the same name,either:
  • Restart all mongos instances andmongod shard members; or
  • Use the flushRouterConfig command on allmongos instances and mongod shardmembers before reading or writing to that database.
• If you use the movePrimary command to move _unsharded_collections, either:
  • Restart all mongos instances andmongod shard members; or
  • Use the flushRouterConfig command on allmongos instances and mongod shardmembers before reading or writing to that database.

This ensures that mongos and shard instances refreshtheir metadata cache. Otherwise, the you may miss data on reads, andmay not write data to the correct shard. To recover, you must manuallyintervene.

In earlier versions, you only need to restart or runflushRouterConfig on the mongos instances.

For more information, see dropDatabase andmovePrimary.

4.2 Drivers Enable Retryable Writes by Default

The official MongoDB 3.6 and 4.0-compatible drivers required including theretryWrites=true option in the connectionstring to enable retryable writes for that connection.

The official MongoDB 4.2-compatible drivers enable Retryable Writes bydefault. Applications upgrading to the 4.2-compatible drivers that requireretryable writes may omit the retryWrites=trueoption. Applications upgrading to the 4.2-compatible drivers that requiredisabling retryable writes must includeretryWrites=false in the connection string.

Important

The local database does not support retryable writes.Applications which write to the local database will encounterwrite errors upon upgrading to a 4.2-series driver _unless_retryable writes are explicitly disabled.

libldap and libldap_r

For MongoDB 4.2 (and 4.0.9) Enterprise binaries linked againstlibldap (such as when running on RHEL), access to thelibldap is synchronized, incurring some performance/latencycosts.

For MongoDB 4.2 (and 4.0.9) Enterprise binaries linked againstlibldap_r, there is no change in behavior from earlier MongoDBversions.

To avoid the automatic synchronization with libldap, you maywish to link to libldap_r. Contact support forassistance.

Connection Pooling and LDAP Server

Starting in version 4.2, MongoDB changes theldapUseConnectionPool default values to:

• true on Windows.
• true on Linux where MongoDB Enterprise binaries are linked againstlibldap_r.

That is, on those systems, MongoDB, by default, uses connection poolingto connect to the LDAP server for authentication/authorization.

In earlier versions (versions 4.0.9+), MongoDB uses false as thedefault value for ldapUseConnectionPool. That is, MongoDB,by default, does not use connection pooling to connect to the LDAPserver for authentication/authorization.

See ldapUseConnectionPool for details.

Arbiter Downgrade Requires Clearing Data Directory

MongoDB 4.2 arbiter data files are incompatible with MongoDB 4.0.Downgrading from MongoDB 4.2 to 4.0 requires deleting arbiter datafiles as an intermediary step. Running a MongoDB 4.0 arbiter againstMongoDB 4.2 data files may result in unexpected behavior.

The downgrade instructions for replica sets and sharded clustersinclude specific steps for downgrading arbiters from 4.2 to 4.0:

• Downgrade 4.2 Replica Set to 4.0.
• Downgrade 4.2 Sharded Cluster to 4.0.

MongoDB Tools

FIPS Mode

Starting in version 4.2, MongoDB removes the —sslFIPSModeoption for the following programs:

• mongodump
• mongoexport
• mongofiles
• mongoimport
• mongorestore
• mongostat
• mongotop

The programs will use FIPS compliant connections tomongod/mongos if themongod/mongos instances areconfigured to use FIPS mode.

4.2 Feature Compatibility

Some features in 4.2 require not just the 4.2 binaries but thefeatureCompatibilityVersion (fCV) set to 4.2. Thesefeatures include:

• Distributed transactions.
• Removal of Index Key Limit for MongoDB versions with fCV setto 4.2+. In concert with the removal of this limit, thefailIndexKeyTooLong parameter has no effect for MongoDBversions with fCV set to 4.2+ and only applies for MongoDB 2.6through MongoDB versions with fCV set to "4.0" or earlier.
• Removal of Index Name Length for MongoDB versions with fCVset to 4.2+.
• New internal format for unique indexes.The new format applies to both existing unique indexes as well asnewly created/rebuilt unique indexes.
• Starting in MongoDB 4.2, users can no longer use the query filter$type: 0 as a synonym for$exists:false. To query for null or missing fields, seeQuery for Null or Missing Fields.
• MongoDB 4.2 adds wildcard indexes tosupport workloads where users query against custom fields or a largevariety of fields in a collection.