Release Notes for MongoDB 4.2

MongoDB 4.2 Released Aug 13, 2019

Minor Releases

4.2.1 - Oct 18, 2019

Issues fixed:

• SERVER-37768: Platform Support: Add Community & Enterprise Debian 10 x64
• SERVER-37772: Platform Support: Add Community & Enterprise RHEL 8 x64
• SERVER-41506: Track metrics associated with a node calling an election
• SERVER-41499: Track number of elections called for each reason in serverStatus
• SERVER-42518: Wildcard index plans miss results when the query path has multiple subsequent array indexes
• SERVER-42856: Transactions with write can be sent to the wrong shard
• All JIRA issues closed in 4.2.1%20AND%20resolution%3D%27Fixed%27%20and%20fixversion%3D%274.2.1%27)
• 4.2.1 Changelog

Distributed Transactions

In version 4.2, MongoDB introduces distributed transactions.Distributed transactions:

• Adds support for multi-document transactions on sharded clusters.
  • All members of the 4.2 sharded clusters must havefeatureCompatibilityVersion of 4.2.
  • Clients must use MongoDB drivers updated for MongoDB 4.2
• Incorporates the existing support for transactions on replica sets.
  • All members of the 4.2 replica set must havefeatureCompatibilityVersion of 4.2.
  • Clients must use MongoDB drivers updated for MongoDB 4.2
• Removes the 16MB total size limit for a transaction. In version4.2, MongoDB creates as many oplog entries (maximum size 16MBeach) as necessary to the encapsulate all write operations in atransaction. In MongoDB 4.0, MongoDB creates a single entry forall write operations in a transaction, thereby imposing a 16MBtotal size limit for a transaction.
• Extends transaction support to deployments whose secondary membersuse the in-memory storage engine. That is,transactions are available for deployments that use the WiredTigerstorage engine for the primary and either the WiredTiger or thein-memory storage engine for the secondarymembers. In MongoDB 4.0, transactions are available for deploymentsthat use the WiredTiger storage engine only.

For more information, see Transactions.

See also

4.2 Transaction Compatibility Changes

Removed MMAPv1 Storage Engine

MongoDB 4.2 removes the deprecated MMAPv1 storage engine.

If your 4.0 deployment uses MMAPv1, you must change the deployment toWiredTiger Storage Engine before upgrading to MongoDB 4.2. For details,see:

• Change Standalone to WiredTiger
• Change Replica Set to WiredTiger
• Change Sharded Cluster to WiredTiger

MMAPv1 Specific Configuration Options

MongoDB removes the following MMAPv1 specific configurationoptions:

Note

Starting in version 4.2, MongoDB processes will not start withthese options. Remove any MMAPv1 specific configurationoptions if using a WiredTiger deployment.

MMAPv1 Specific Parameters

MongoDB removes the following MMAPv1 parameters:

• newCollectionsUsePowerOf2Sizes
• replIndexPrefetch

MMAPv1 Specific Command

MongoDB removes the MMAPv1 specific touch command.

MMAPv1 Specific Options for Binaries, Commands and Methods

MongoDB removes the MMAPv1 specific options:

• noPadding and usePowerOf2Sizes for collMod
• verbose for collStats
• flags for create
• paddingFactor, paddingBytes, preservePadding for thedb.createCollection() method and the compactcommand.
• repair for mongodump

Removed Commands and Methods

Remove maxScan Option

MongoDB removes the deprecated option maxScan for thefind command and the mongo shell helpercursor.maxScan(). Use either the maxTimeMS option for thefind command or the helper cursor.maxTimeMS() instead.

MongoDB Drivers

The following drivers are feature compatible [1] with MongoDB 4.2:

Sharded Clusters

Mutable Shard Key Values

Starting in MongoDB 4.2, you can update a document’s shard key valueunless the shard key field is the immutable _id field. For detailson updating the shard key, see Change a Document’s Shard Key Value.

Before MongoDB 4.2, a document’s shard key field value is immutable.

Backups

mongodump and mongorestorecannot be part of a backup strategy for 4.2+ sharded clustersthat have sharded transactions in progress as these tools cannotguarantee a atomicity guarantees of data across the shards.

For 4.2+ sharded clusters with in-progress sharded transactions, forcoordinated backup and restore processes that maintain the atomicityguarantees of transactions across shards, see:

• MongoDB Atlas,
• MongoDB Cloud Manager, or
• MongoDB Ops Manager.

Balancer State and Autosplit

Starting in MongoDB 4.2:

• The balancerStart command and themongo shell helper methodssh.startBalancer() andsh.setBalancerState(true) alsoenable auto-splitting for the sharded cluster.

To disable auto-splitting when the balancer is enabled, you canuse sh.disableAutoSplit().

• The balancerStop command and the mongoshell helper methods sh.stopBalancer() andsh.setBalancerState(false) alsodisable auto-splitting for the sharded cluster.

To enable auto-splitting when the balancer is disabled, you canuse sh.enableAutoSplit().

The mongo methodssh.enableBalancing(namespace) andsh.disableBalancing(namespace) have no affect on theauto-splitting.

mongos Connection Pool

Starting in MongoDB 4.2, MongoDB adds the parameterShardingTaskExecutorPoolReplicaSetMatching thatdetermines the minimum size (can vary during runtime) of themongos instance’s connection pools to each member ofthe sharded cluster.

By default, for each replica set in the sharded cluster (i.e. shardreplica set and config servers), mongos maintainsconnection pools to each replica set secondary that are at leastequal to the size of its connection pool to the primary.

To modify, see ShardingTaskExecutorPoolReplicaSetMatching.

Sharded Collections and Replace Documents

Starting in MongoDB 4.2,

• Replace document operations, such asdb.collection.replaceOne() or db.collection.update(query, replacement, … ),attempt to target a single shard, first by using the query filter. If theoperation cannot target a single shard by the query filter, it thenattempts to target by the replacement document.

In earlier versions, the operation attempts to target using thereplacement document.

• For a replace document operation that includes upsert: trueand is on a sharded collection, the filter must include anequality match on the full shard key.

Security Improvements

Resolved Common Vulnerabilities and Exposures

MongoDB 4.2 includes fixes that resolve the following CommonVulnerabilities and Exposures (CVEs):

• CVE-2019-2389 (See SERVER-40563)
• CVE-2019-2386 (See SERVER-38984)

New TLS Options

MongoDB 4.2 adds TLS options for the mongod, the mongos, and themongo shell to replace the correspondingSSL options (deprecated in 4.2). The new TLS options provideidentical functionality as the deprecated SSL options as MongoDBhas always supported TLS 1.0 and later.

• For the command-line TLS options, refer to the mongod, mongos, andmongo shell pages.
• For the corresponding mongod and mongos configuration fileoptions, refer to the configuration file page.
• For the connection string tls options, refer to theconnection string page.

Tip

Most of the new TLS option names are similar to the SSLoption name; e.g. —tlsMode instead of—sslMode. The exceptions are:

• net.tls.certificateKeyFile vs. net.ssl.PEMKeyFile
• net.tls.certificateKeyFilePassword vs. net.ssl.PEMKeyPassword
• —tlsCertificateKeyFile vs. —sslPEMKeyFile
• —tlsCertificateKeyFilePassword vs. —sslPEMKeyPassword

See also

New tlsClusterCAFile Option

Deprecated SSL Options

MongoDB 4.2 deprecates the SSL options for the mongod, the mongos, and themongo shell as well as the correspondingnet.ssl Options configuration file options.

Use the new TLS options instead.

New tls Parameters

New tlsClusterCAFile Option

MongoDB 4.2 adds the—tlsClusterCAFile option/net.tls.clusterCAFile formongod and mongos, which specifies a.pem file for validating the TLS certificate from a clientestablishing a connection. This lets you use separate Certificate Authoritiesto verify the client to server and server to client portions of the TLShandshake.

See also

New TLS Options

Forward Secrecy

Starting in version 4.2 on Linux:

• If the platform’s OpenSSL supports automatic curve selection forElliptic Curve Diffie-Hellman, MongoDB enables support forEphemeral Elliptic Curve Diffie-Hellman (ECDHE).
• If the platform’s OpenSSL does not support automatic curve selectionfor Elliptic Curve Diffie-Hellman, MongoDB attempts to enable ECDHEsupport using prime256v1 as the named curve.
• If support for ECDHE is enabled, MongoDB attempts to enable supportfor Ephemeral Diffie-Hellman (DHE) if Ephemeral Diffie-Hellman (DHE) is not explicitly enabled.

In earlier versions of MongoDB (3.6.14+ and 4.0.3+), MongoDB enablessupport for Ephemeral Elliptic Curve Diffie-Hellman (ECDHE) if, duringcompile time, the Linux platform’s OpenSSL supports automatic curveselection of ECDH parameters.

On Windows and macOS, MongoDB’s support for ECDHE and DH remainunchanged from earlier versions; that is, support is implicit throughthe use of the platform’s respective native TLS/SSL OS libraries.

For more information, see Forward Secrecy.

passwordPrompt()

Starting in version 4.2 of the mongo shell, you canuse the passwordPrompt() method in conjunction withvarious user authentication/management methods/commands to promptfor the password instead of specifying the password directly in themethod/command call. However, you can still specify the passworddirectly as you would with earlier versions of themongo shell.

For example:

db.createUser( {
   user:"user123",
   pwd: passwordPrompt(),   // Instead of specifying the password in cleartext
   roles:[ "readWrite" ]
} )

Keyfile Format Change to YAML

Starting in MongoDB 4.2, keyfiles for internal membershipauthentication use YAML format to allow formultiple keys in a keyfile. The YAML format accepts content of:

• a single key string (same as in earlier versions),
• multiple key strings (each string must be enclosed in quotes), or
• sequence of key strings.

The YAML format is compatible with the existing single-keykeyfiles that use the text file format.

The new format allows for rolling upgrade of the keys without downtime.See Rotate Keys for Replica Sets andRotate Keys for Sharded Clusters.

libldap and libldap_r

For MongoDB 4.2 (and 4.0.9) Enterprise binaries linked againstlibldap (such as when running on RHEL), access to thelibldap is synchronized, incurring some performance/latencycosts.

For MongoDB 4.2 (and 4.0.9) Enterprise binaries linked againstlibldap_r, there is no change in behavior from earlier MongoDBversions.

To avoid the automatic synchronization with libldap, you maywish to link to libldap_r. Contact support forassistance.

Encrypted Storage Engine

For encrypted storage engineconfigured with AES256-GCM cipher:

• • Restoring from Hot Backup
  • Starting in 4.2, if you restore from files taken via “hot”backup (i.e. the mongod is running), MongoDBcan detect “dirty” keys on startup and automatically rolloverthe database key to avoid IV (Initialization Vector) reuse.
• • Restoring from Cold Backup
  • However, if you restore from files taken via “cold” backup(i.e. the mongod is not running), MongoDBcannot detect “dirty” keys on startup, and reuse of IV voidsconfidentiality and integrity guarantees.

Starting in 4.2, to avoid the reuse of the keys afterrestoring from a cold filesystem snapshot, MongoDB adds a newcommand-line option —eseDatabaseKeyRollover. When started with the—eseDatabaseKeyRollover option, the mongodinstance rolls over the database keys configured withAES256-GCM cipher and exits.

For more information, see encrypted storage engine and —eseDatabaseKeyRollover.

General Security Enhancements

• Add serverStatus to the backup built-inrole.

• To connect a client over TLS/SSL connection, MongoDB 4.2 supportsmatching by IP addresses as well as DNS for Subject Alternative Name(SAN) matching.

For example, a mongod instance’s x.509 certificatehas the following SAN:

X509v3 Subject Alternative Name:
    DNS:hostname.example.com, DNS:localhost, IP Address:127.0.0.1

Then, to connect a mongo shell to the instance, youcan specify the host of 127.0.0.1 or the DNS names:

mongo "mongodb:\\127.0.0.1:27017\test" --tls --tlsCAFile /etc/ssl/ca.pem ...

mongo "mongodb:\\hostname.example.com:27017\test" --tls --tlsCAFile /etc/ssl/ca.pem ...

mongo "mongodb:\\localhost:27017\test" --tls --tlsCAFile /etc/ssl/ca.pem ...

In previous versions, MongoDB only supported DNS entries for SANmatching.

mongo "mongodb:\\hostname.example.com:27017\test" --tls --tlsCAFile /etc/ssl/ca.pem ...

mongo "mongodb:\\localhost:27017\test" --tls  --tlsCAFile /etc/ssl/ca.pem ...

LDAP Query Template {PROVIDED_USER} Token

Starting in version 4.2, MongoDB Enterprise adds a new token{PROVIDED_USER} that can be used insecurity.ldap.authz.queryTemplate. When used in thetemplate, MongoDB substitutes the supplied username, i.e. before eitherauthentication or LDAP transformation.

Aggregation Improvements

On-Demand Materialized View ($merge Stage)

MongoDB 4.2 adds the $merge aggregation stage.

With the new stage you can:

• Can output to a collection in the same or different database.
• Can incorporate results (merge documents, replace documents, keepexisting documents, fail the operation, process documents with ancustom update pipeline) into an existing collection.
• Can output to an existing sharded collection.

The new stage allows users to create on-demand materialized views, where the content of the output collectioncan be incrementally updated each time the pipeline is run.

Aggregation Trigonometry Expressions

MongoDB 4.2 adds newtrigonometry expressions for use in aggregation pipelines.

Trigonometry expressions perform trigonometric operations on numbers.Values that represent angles are always input or output in radians. Use$degreesToRadians and $radiansToDegrees toconvert between degree and radian measurements.

Aggregation Arithmetic Expressions

MongoDB 4.2 adds the $round aggregation expression.Use $round to round numerical values to a specificdigit or decimal place.

MongoDB 4.2 adds expanded functionality and new syntax to$trunc. Use $trunc with the new syntaxto truncate numerical values to a specific digit or decimal place.

Aggregation Regular Expressions (regex) Operators

MongoDB 4.2 adds the following regular expression (regex) patternmatching operators for use in the aggregation pipeline:

Prior to MongoDB 4.2, aggregation pipeline can only use the queryoperator $regex in the $match stage.

New Stages

MongoDB 4.2 adds the following new aggregation pipeline stages:

New Variables

MongoDB 4.2 adds the following new aggregation pipeline variables:

Availability

Starting in MongoDB 4.2, you can use the aggregation pipeline forupdates in:

For the updates, the pipeline can consist of the following stages:

• $addFields and its alias $set
• $project and its alias $unset
• $replaceRoot and its alias $replaceWith.

Using the aggregation pipeline allows for a more expressive updatestatement, such as expressing conditional updates based on currentfield values or updating one field using the value of another field(s).

See the individual reference pages for details and examples.

See also

Aggregation

Change Stream

startAfter Option for Change Streams

MongoDB 4.2 adds startAfter as an option for Change Streams,which starts a new change stream after the event indicated by a resumetoken. With this option, you can start a change stream from aninvalidate event, thereby guaranteeingno missed notifications after the previous stream was invalidated.

Change Streams Resume Tokens

MongoDB 4.2 uses the version 1 (i.e. v1) change streamsresume tokens, introduced inversion 4.0.7.

Starting in MongoDB 4.2, change streams will throw an exception ifthe change stream aggregation pipeline modifies an event’s _id field.

Availability

Starting in MongoDB 4.2, change streams areavailable regardless of the "majority" read concernsupport; that is, read concern majority support can be eitherenabled (default) or disabledto use change streams.

In MongoDB 4.0 and earlier, change streams areavailable only if "majority" read concern support isenabled (default).

Change Stream Pipeline

Starting in MongoDB 4.2, you can use additional stages in thechange stream aggregation pipelineto modify the change stream output (i.e. the event documents):

• $replaceWith
• $set
• $unset

Starting in MongoDB 4.2, change streams will throw an exception ifthe change stream aggregation pipeline modifies an event’s _id field.

See also

Change Streams Compatibility Changes

Update Enhancements

Update and Aggregation

Starting in MongoDB 4.2, you can use the aggregation pipeline forupdates in:

For the updates, the pipeline can consist of the following stages:

• $addFields and its alias $set
• $project and its alias $unset
• $replaceRoot and its alias $replaceWith.

Using the aggregation pipeline allows for a more expressive updatestatement, such as expressing conditional updates based on currentfield values or updating one field using the value of another field(s).

See the individual reference pages for details and examples.

Update and Hint

Starting in MongoDB 4.2, the update command and theassociated mongo shell methoddb.collection.update() can accept a hint argument to specifythe index to use. See:

• update
• db.collection.update()

Sharded Collections and Replace Documents

Starting in MongoDB 4.2,

• Replace document operations, such asdb.collection.replaceOne() or db.collection.update(query, replacement, … ),attempt to target a single shard, first by using the query filter. If theoperation cannot target a single shard by the query filter, it thenattempts to target by the replacement document.

In earlier versions, the operation attempts to target using thereplacement document.

• For a replace document operation that includes upsert: trueand is on a sharded collection, the filter must include anequality match on the full shard key.

Wildcard Indexes

MongoDB 4.2 introduces wildcard indexes forsupporting queries against fields whose names are unknown or arbitrary.

Consider an application that captures user-defined data under theuserMetadata field and supports querying against that data:

{ "userMetadata" : { "likes" : [ "dogs", "cats" ] } }
{ "userMetadata" : { "dislikes" : "pickles" } }
{ "userMetadata" : { "age" : 45 } }
{ "userMetadata" : "inactive" }

Administrators want to create indexes to support queries on anysubfield of userMetadata.

A wildcard index on userMetadatacan support single-field queries on userMetadata,userMetadata.likes, userMetadata.dislikes, anduserMetadata.age:

db.userData.createIndex( { "userMetadata.$**" : 1 } )

The index can support the following queries:

db.userData.find({ "userMetadata.likes" : "dogs" })
db.userData.find({ "userMetadata.dislikes" : "pickles" })
db.userData.find({ "userMetadata.age" : { $gt : 30 } })
db.userData.find({ "userMetadata" : "inactive" })

A non-wildcard index on userMetadata can only support queries onvalues of userMetadata.

Important

Wildcard indexes are not designed to replace workload-based indexplanning. For more information on creating indexes to supportqueries, see Create Indexes to Support Your Queries. Forcomplete documentation on wildcard index limitations, seeWildcard Index Restrictions.

The mongodfeatureCompatibilityVersion must be 4.2 tocreate wildcard indexes. For instructions on setting the fCV, seeSet Feature Compatibility Version on MongoDB 4.2 Deployments.

You can create a wildcard index using the createIndexesdatabase command or its shell helpersdb.collection.createIndex() anddb.collection.createIndexes(). For examples of creatinga wildcard index, see Create a Wildcard Index.

See Wildcard Indexes for complete documentation.

Platform Support

• MongoDB 4.2 adds support for:
  • Ubuntu 18.04 on ARM64
• MongoDB 4.2 removes support for:
  • Debian 8
  • Ubuntu 14.04
  • Ubuntu 16.04 ARM64 for MongoDB Community Edition
  • Ubuntu 16.04 POWER/PPC64LE (Also removed in version 3.6.13 and 3.4.21)
  • macOS 10.11

See Supported Platforms.

MongoDB Tools

FIPS Mode

Starting in version 4.2, MongoDB removes the —sslFIPSModeoption for the following programs:

• mongodump
• mongoexport
• mongofiles
• mongoimport
• mongorestore
• mongostat
• mongotop

The programs will use FIPS compliant connections tomongod/mongos if themongod/mongos instances areconfigured to use FIPS mode.

—uri Option

Starting in version 4.2,

• For the following command-line tools, if the write concern isspecified in both the —uri connection string and the—writeConcern option, the —writeConcern option overridesthe one in the connection string:
  • mongofiles
  • mongoimport
  • mongorestore
• For the following command-line tools, if the read preference isspecified in both the —uri connection string and the—readPreference option, the —readPreference optionoverrides the one in the connection string:
  • mongodump
  • mongoexport
  • mongofiles

Extended JSON v2

Starting in version 4.2:

For details on MongoDB extended JSON v2, seeMongoDB Extended JSON (v2).

See also

—query Options

mongofiles

The mongofiles command get_id and delete_id can acceptboth ObjectId or non-ObjectId values for the _id.

Bulk Operations for mongoimport and mongorestore

mongoimport

Starting in version 4.2:

• mongoimport uses maximum batch size of 100,000 toperform bulk insert/upsert operations.
• mongoimport by default,continues when it encounters duplicate key and documentvalidation errors. To ensure that the program stops on these errors, specify —stopOnError.
• Specifying —maintainInsertionOrder for mongoimport:
  • Maintains document insertion order using ordered bulk writeoperations; i.e. both the batch order and document order within the batches are maintained. In earlier versions, only the batch order is maintained; document order within batches are not maintained.
  • Enables —stopOnError and setsnumInsertionWorkers to1.

mongorestore

Starting in version 4.2:

• mongorestore by default,continues when it encounters duplicate key and documentvalidation errors. To ensure that the program stops on these errors, specify —stopOnError.
• Specifying —maintainInsertionOrder for mongorestore:
  • Maintains document insertion order using ordered bulk writeoperations; i.e. both the batch order and document order within the batches are maintained. In earlier versions, only the batch order is maintained; document order within batches are not maintained.
  • Enables —stopOnError andsets —numInsertionWorkersPerCollection to 1.

Lock Optimization for Specific DDL Operations

Starting with MongoDB 4.2, the following operations take anexclusive collection lock instead of an exclusive database lock:

Prior to MongoDB 4.2, these operations took an exclusive lock on thedatabase, blocking all operations on the database and its collectionsuntil the operation completed.

In earlier versions, get_id and delete_id can only acceptObjectId values for the _id.

Monitoring

Starting in version 4.2, the Storage Node Watchdog is available in both the MongoDB Communityedition and the MongoDB Enterprise edition.

In earlier versions, the feature is available in the MongoDB Enterpriseedition only.

Flow Control

MongoDB 4.2 introduces a flow control mechanism to control the rate atwhich the primary applies its writes in order to keep themajority committedlag under a specified maximum value.

Flow control is enabled by default.

Note

For flow control to engage, the replica set/sharded cluster musthave: featureCompatibilityVersion (FCV) of4.2 and read concern majority enabled. That is, enabled flowcontrol has no effect if FCV is not 4.2 or if read concernmajority is disabled.

For more information, see Replication Lag and Flow Control.

Logging and Diagnostics

Logging

• Added INITSYNC component to log messages.

• Added ELECTION component to log messages.

• For debug messages, include the verbosity level (i.e. D[1-5]).For example, if verbosity level is 2, MongoDB logs D2. Inprevious versions, MongoDB log messages only specified D forDebug level.

• When logging to syslog, the format of themessage text includes the component.For example:

...  ACCESS   [repl writer worker 5] Unsupported modification to roles collection ...

Previously, the syslog message text didnot include the component. Forexample:

... [repl writer worker 1] Unsupported modification to roles collection ...

• MongoDB 4.2 adds a usedDisk indicator to the profiler logmessages anddiagnostic log messages for theaggregate operation. The usedDisk indicates whetherany stages of an aggregate operation wrote data totemporary files due to memory restrictions. For more information onaggregation memory restrictions, see Memory Restrictions.

• Starting in version 4.2 (also available starting in 4.0.6),secondary members of a replica set now log oplog entries that takelonger than the slow operation threshold to apply. These messages arelogged for the secondaries under theREPL component with the text applied op: <oplog entry> took<num>ms.

2018-11-16T12:31:35.886-0500 I REPL   [repl writer worker 13] applied op: command { ... }, took 112ms

The slow oplog application logging on secondaries are:

• Not affected by theslowOpSampleRate; i.e. all slowoplog entries are logged by the secondary.
• Not affected by thelogLevel/systemLog.verbosity level (or thesystemLog.component.replication.verbosity level); i.e.for oplog entries, the secondary logs only the slow oplog entries.Increasing the verbosity level does not log all oplog entries.

• Not captured by the profiler and not affected by theprofiling level.For more information on setting the slow operation threshold, see

• mongod —slowms

• slowOpThresholdMs
• The profile command ordb.setProfilingLevel() shell helper method.
  • Starting in MongoDB 4.2, the getLog command truncatesany event that contains more than 1024 characters. In earlierversions, getLog truncates after 512 characters.

• Starting in MongoDB 4.2 (and in 4.0.9), for slow operations, the profilerentries anddiagnostic log messages includestorage information.

• Starting in MongoDB 4.2, the profiler entries and the diagnosticlog messages (i.e. mongod/mongos logmessages) for read/write operations include:

  • queryHash to help identify slow queries with the samequery shape.
  • planCacheKey to provide more insight into the query plancache for slow queries.See Query Plan Improvements.

currentOp

MongoDB 4.2 adds a new option idleCursors to the$currentOp aggregation stage in order to return informationon idle cursors.

In addition, MongoDB 4.2 adds the following new fields to the documentsreturned from the $currentOp aggregation stage,currentOp command, and db.currentOp() helper:

See also 4.2 current op compatibility changes

serverStatus Metrics

Starting in MongoDB 4.2, the serverStatus command and themongo shell method db.serverStatus() includethe following output changes:

Replica Set Status Metrics

Starting in version MongoDB 4.2, replSetGetStatus and itsmongo shell helper rs.status() return:

• The IP address, replSetGetStatus.members[n].ip for thereplica set members.

• ISODate-formatted date string fields that correspond to the variousreplSetGetStatus.optimes.

New ISODate-Formatted Date String FieldCorresponding Optime FieldlastCommittedWallTimelastCommittedOpTimereadConcernMajorityWallTimereadConcernMajorityOpTimelastAppliedWallTimeappliedOpTimelastDurableWallTimedurableOpTime

MongoDB 4.2 deprecates the fieldlastStableCheckpointTimestamp.

Lock Diagnostics Reporting

Starting in version 4.2, MongoDB reports onReplicationStateTransition lock information.

In addition, MongoDB 4.2 separates ParallelBatchWriterMode lockinformation from Global lock information. Earlier MongoDB versionsreport ParallelBatchWriterMode lock information as partof Global locks.

For operations that report on lock information, see:

• serverStatus command and db.serverStatus() method.
• $currentOp aggregation pipeline stage, currentOp command, and db.currentOp() method.

collStats Improvements

Starting in MongoDB 4.2, the $collStats aggregation, thecollStats command, and the mongo shellhelper db.collection.stats return information on indexes thatare currently being built.

For details, see:

• collStats.nindexes
• collStats.indexDetails
• collStats.indexBuilds
• collStats.totalIndexSize
• collStats.indexSizes

Starting in MongoDB 4.2, the $collStats aggregation, thecollStats command, and the mongo shellhelper db.collection.stats return the scaleFactor used toscale the various size data.

dbStats Improvements

Starting in MongoDB 4.2, the dbStats command, and themongo shell helper db.stats() returnthe scaleFactor used to scale the various size data.

General Improvements

Externally Sourced Values for Configuration Files

MongoDB supports usingexpansion directives in configurationfiles to load externally sourced values. Expansion directives canload values for specificconfiguration file optionsor load theentire configuration file.

The following expansion directives are available:

For complete documentation, see Externally Sourced Configuration File Values.

outputConfig Option

MongoDB 4.2 adds the —outputConfigoption for mongod and mongos. The optionoutputs to stdout the mongod/mongosinstance’s configuration, in YAML format.

If the configuration uses any Externally Sourced Configuration File Values, theoption returns the resolved value for those options.

Warning

This may include any configured passwords or secrets previouslyobfuscated through the external source.

For usage examples, see:

• Output the Configuration File with Resolved Expansion Directive Values
• Convert Command-Line Options to YAML

Remove Index Key Size Limit

Starting in MongoDB 4.2, for featureCompatibilityVersion set to "4.2" or greater, MongoDB removes theIndex Key Limit. For fCV set to "4.0", the limit stillapplies.

See also

4.2 Indexes Compatibility Changes

Remove Index Name Length Limit

Starting in version 4.2, for featureCompatibilityVersion set to "4.2" or greater, MongoDB removes theIndex Name Length limit of 127 byte maximum. In previousversions or MongoDB versions withfeatureCompatibilityVersion (fCV) set to"4.0", index names must fall within thelimit.

See also

4.2 Indexes Compatibility Changes,4.2 Feature Compatibility

Improvements to dropIndexes

Drop multiple indexes

Starting in MongoDB 4.2, you can specify multiple indexes to thedropIndexes command and its mongo shellhelper db.collection.dropIndexes(). To specify multipleindexes to drop, pass an array of index names todropIndexes/db.collection.dropIndexes().

Kill related queries only

Starting in MongoDB 4.2, the dropIndexes or its shell helpers dropIndex() and dropIndexes() operation only killsqueries that are using the index being dropped. This may includequeries considering the index as part ofquery planning.

Prior to MongoDB 4.2, dropping an index on acollection would kill all open queries on the collection.

zstd Availability

Starting in version 4.2, MongoDB supports zstd for:

• block compression. Seestorage.wiredTiger.collectionConfig.blockCompressor.
• journal compression. Seestorage.wiredTiger.engineConfig.journalCompressor.
• network compression. See net.compression.compressors.
  • For clients using the MongoDB drivers, they must use driversupdated for MongoDB 4.2.
  • The network compressors for mongod andmongos default to both snappy,zstd,zlibcompressors, in that order. In version 4.0, mongodand mongos enable network compression by defaultwith snappy as the compressor.

bulkWrite() Error Handling inside Transactions

Starting in MongoDB 4.2, if a db.collection.bulkWrite()operation encounters an error inside a transaction, the method throws a BulkWriteException (same as outside a transaction).

In 4.0, if the bulkWrite operation encounters an error inside atransaction, the error thrown is not wrapped as aBulkWriteException.

Inside a transaction, the first error in a bulk write causes theentire bulk write to fail and aborts the transaction, even if thebulk write is unordered.

Query Plan Improvements

Plan Cache States

Starting in MongoDB 4.2, the cache entry is associated with a state:

• Missing
• Inactive
• Active

Associating states with entries helps reduce the likelihood thatsub-optimal cache entries remain in the cache. For more information,see Query Plans.

queryHash and planCacheKey

• • queryHash
  • To help identify slow queries with the same query shape,starting in MongoDB 4.2, each query shape is associated witha queryHash. The queryHash is ahexadecimal string that represents a hash of the query shape andis dependent only on the query shape.

Note

As with any hash function, two different query shapes may resultin the same hash value. However, the occurrence of hashcollisions between different query shapes is unlikely.

• • planCacheKey
  • To provide more insight into the query plan cache, MongoDB 4.2 introduces the planCacheKey.

planCacheKey is a hash of the key for the plan cache entryassociated with the query.

Note

Unlike the queryHash, the planCacheKey is a function ofboth the query shape and the currently available indexes for theshape. That is, if indexes that can support the query shape areadded/dropped, the planCacheKey value may change whereas thequeryHash value would not change.

See also

planCacheKey

• The queryHash and planCacheKey are available in:

  • profiler entryfields queryHash andplanCacheKey the logged query operations.
  • diagnostic log messages (i.e. mongod/mongos logmessages) for the logged query operations.
  • explain() output fields:queryHash andplanCacheKey

The fields are also available in operations that return informationabout the query plan cache:

• $planCacheStats aggregation stage (New in MongoDB 4.2)
• PlanCache.listQueryShapes()method/planCacheListQueryShapes command (Deprecated in MongoDB 4.2)
• PlanCache.getPlansByQuery()method/planCacheListPlans command (Deprecated in MongoDB 4.2)

See also

Deprecated Plan Cache Commands/Methods

$regex and $not

Starting in MongoDB 4.2 (and 4.0.7), $not operator can perform logicalNOT operation on $regex operator expressions as well ason regular expression objects (i.e. /pattern/).

In 4.0 and earlier versions, you could use $not operator withregular expression objects (i.e. /pattern/) but not with$regex operator expressions.

Kill Own Cursors

Starting in MongoDB 4.2, users can always kill their own cursors,regardless of whether the users have the privilege tokillCursors. As such, the killCursorsprivilege has no effect starting in MongoDB 4.2.

In MongoDB 4.0, users required the killCursors privilegein order to kill their own cursors.

New Parameters

MongoDB 4.2 adds the parameter replBatchLimitBytes toconfigure the maximum oplog application batch size. The parameter isalso available starting in MongoDB 4.0.10.

Retryable Writes on Certain Single-Document Upserts

MongoDB 4.2 will retry certain single-document upserts(update with upsert: true and multi: false) that encounter aduplicate key exception. See Duplicate Key Errors on Upsert forconditions.

Prior to MongoDB 4.2, MongoDB would not retry upsert operationsthat encountered a duplicate key error.

db.dropDatabase() and Write Concern

Starting in MongODB 4.2, the mongo shell methoddb.dropDatabase() can take an optional write concern document.

dropConnections

The dropConnections command drops themongod/mongos instance’s outgoingconnections to the specified hosts. The dropConnectionsmust be run against the admin database.

Client Disconnection

For the following operations, if the issuing client disconnects beforethe operation completes, MongoDB marks the following operations for termination(e.g. killOp on the operation):

Startup Warnings

In-Memory Storage Engines

Starting in version 4.2 (and 4.0.13 and 3.6.14 ), if a replica setmember uses the in-memory storage engine(voting or non-voting) but the replica set haswriteConcernMajorityJournalDefault set to true, thereplica set member logs a startup warning.

mongo Shell

Starting in MongoDB 4.2 (and 4.0.13), the mongo shell displays awarning message when connected to non-genuine MongoDB instances asthese instances may behave differently from the official MongoDBinstances; e.g. missing or incomplete features, different featurebehaviors, etc.

Map-Reduce

Starting in version 4.2, MongoDB deprecates:

• The map-reduce option to create a new sharded collection as wellas the use of the sharded option formap-reduce. To output to a sharded collection, create the shardedcollection first. MongoDB 4.2 also deprecates the replacement ofan existing sharded collection.
• The explicit specification of nonAtomic: false option.

Rollback Time Limit

Starting in MongoDB 4.2, the rollback time limit is calculated betweenthe first operation after the common point and the last point in theoplog for the member to roll back.

In MongoDB 4.0, the rollback time limit is calculated between thecommon point and the last point in the oplog for the member to rollback.

For more information, see Rollback Elapsed Time Limitations.

isInteractive()

MongoDB 4.2 adds a new mongo shell methodisInteractive() that returns a boolean indicating whether themongo shell is running in interactive or script mode.

Change to explain Output

Starting in MongoDB 4.2, the explain output can include a newoptimizedPipeline field. For details,refer to optimizedPipeline.

Change to isMaster Output

Starting in MongoDB 4.2, the output for isMaster, and thedb.isMaster() helper method, returns theconnectionId for themongod/mongos instance’s connection tothe client.

Optimized Index Builds

MongoDB index builds against a populated collection require an exclusiveread-write lock against the collection. Operations that require a reador write lock on the collection must wait until themongod releases the lock. MongoDB uses an optimized buildprocess that only holds the exclusive lock at the beginning and end ofthe index build. The rest of the build process yields to interleavingread and write operations.

For feature compatibility version (fcv) 4.2,MongoDB 4.2 index builds fully replace the index build processessupported in previous MongoDB versions. MongoDB ignores thebackground index build option if specified tocreateIndexes or its shell helperscreateIndex() andcreateIndexes().

Requires featureCompatibilityVersion 4.2

For MongoDB clusters upgraded from 4.0 to 4.2, you must set thefeature compatibility version (fcv) to 4.2to enable the optimized build process. For more information onsetting the fCV, see setFeatureCompatibilityVersion.

MongoDB 4.2 clusters running with fCV 4.0 only support 4.0 indexbuilds.

For complete documentation on the index build process, seeIndex Builds on Populated Collections.

Changes Affecting Compatibility

Some changes can affect compatibility and may require user actions. Fora detailed list of compatibility changes, seeCompatibility Changes in MongoDB 4.2.

Upgrade Procedures

Feature Compatibility Version

To upgrade, the 4.0 instances must havefeatureCompatibilityVersion set to 4.0. To check the version:

db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )

For specific details on verifying and setting thefeatureCompatibilityVersion as well as information on otherprerequisites/considerations for upgrades, refer to the individualupgrade instructions:

• Upgrade a Standalone to 4.2
• Upgrade a Replica Set to 4.2
• Upgrade a Sharded Cluster to 4.2

If you need guidance on upgrading to 4.2, MongoDB offers major versionupgrade servicesto help ensure a smooth transition without interruption to your MongoDB application.

Download

To download MongoDB 4.2, go to the MongoDB Download Center

See also

• All Third Party License Notices.

Known Issues

Report an Issue

To report an issue, seehttps://github.com/mongodb/mongo/wiki/Submit-Bug-Reports forinstructions on how to file a JIRA ticket for the MongoDB server or oneof the related projects.