MongoDB Performance

As you develop and operate applications with MongoDB, you may need toanalyze the performance of the application and its database.When you encounter degraded performance, it is often a function of databaseaccess strategies, hardware availability, and the number of open databaseconnections.

Some users may experience performance limitations as a result of inadequateor inappropriate indexing strategies, or as a consequence of poor schemadesign patterns. Locking Performance discusses how these canimpact MongoDB’s internal locking.

Performance issues may indicate that the database is operating atcapacity and that it is time to add additional capacity to thedatabase. In particular, the application’s working set should fit inthe available physical memory.

In some cases performance issues may be temporary and related toabnormal traffic load. As discussed in Number of Connections, scalingcan help relax excessive traffic.

Database Profiling can help you to understand what operations are causingdegradation.

Locking Performance

MongoDB uses a locking system to ensure data set consistency. Ifcertain operations are long-running or a queue forms, performancewill degrade as requests and operations wait for the lock.

Lock-related slowdowns can be intermittent. To see if the lock has beenaffecting your performance, refer to the lockssection and the globalLock section of theserverStatus output.

Dividing locks.timeAcquiringMicros bylocks.acquireWaitCountcan give an approximate average wait time for a particular lock mode.

locks.deadlockCount providethe number of times the lock acquisitions encountered deadlocks.

If globalLock.currentQueue.total is consistently high,then there is a chance that a large number of requests are waiting fora lock. This indicates a possible concurrency issue that may be affectingperformance.

If globalLock.totalTime ishigh relative to uptime, the database hasexisted in a lock state for a significant amount of time.

Long queries can result from ineffective use of indexes;non-optimal schema design; poor query structure; system architecture issues; orinsufficient RAM resulting in disk reads.

Number of Connections

In some cases, the number of connections between the applications and thedatabase can overwhelm the ability of the server to handle requests. Thefollowing fields in the serverStatus document can provide insight:

If there are numerous concurrent application requests, the database may havetrouble keeping up with demand. If this is the case, then you will need toincrease the capacity of your deployment.

For read-heavy applications, increase the size of your replica set anddistribute read operations to secondary members.

For write-heavy applications, deploy sharding and add one or moreshards to a sharded cluster to distribute load amongmongod instances.

Spikes in the number of connections can also be the result ofapplication or driver errors. All of the officially supported MongoDBdrivers implement connection pooling, which allows clients to use andreuse connections more efficiently. Extremely high numbers ofconnections, particularly without corresponding workload is oftenindicative of a driver or other configuration error.

Unless constrained by system-wide limits, MongoDB has no limit onincoming connections. On Unix-based systems, you can modify system limitsusing the ulimit command, or by editing your system’s/etc/sysctl file. See UNIX ulimit Settings for moreinformation.

Database Profiling

The Database Profiler collects detailedinformation about operations run against a mongod instance. Theprofiler’s output can help to identify inefficient queries andoperations.

You can enable and configure profiling for individual databases or forall databases on a mongod instance.Profiler settings affect only a single mongod instance andwill not propagate across a replica set or shardedcluster.

See Database Profiler for information onenabling and configuring the profiler.

The following profiling levels are available:

LevelDescription
0The profiler is off and does not collect any data.This is the default profiler level.
1The profiler collects data for operations that take longerthan the value of slowms.
2The profiler collects data for all operations.

Important

Profiling can impact performance and shares settings with the systemlog. Carefully consider any performance and security implicationsbefore configuring and enabling the profiler on a productiondeployment.

See Profiler Overhead for more information onpotential performance degradation.

Note

When logLevel is set to 0, MongoDB records _slow_operations to the diagnostic log at a rate determined byslowOpSampleRate. Starting in MongoDB4.2, the secondaries of replica sets log all oplog entry messagesthat take longer than the slow operation threshold to apply regardless of the sample rate.

At higher logLevel settings, all operations appear inthe diagnostic log regardless of their latency with the followingexception: the logging of slow oplog entry messages by thesecondaries. The secondaries log only the slow oplogentries; increasing the logLevel does not log alloplog entries.

Starting in MongoDB 4.2, the profiler entries and the diagnosticlog messages (i.e. mongod/mongos logmessages) for read/write operations include:

  • queryHash to help identify slow queries with the samequery shape.
  • planCacheKey to provide more insight into the query plancache for slow queries.

Full Time Diagnostic Data Capture

To facilitate analysis of the MongoDB server behavior by MongoDB Inc.engineers, mongod and mongos processes include aFull Time Diagnostic Data Collection (FTDC) mechanism. FTDC data filesare compressed, are not human-readable, and inherit the same file accesspermissions as the MongoDB data files. Only users with access to FTDCdata files can transmit the FTDC data. MongoDB Inc. engineers cannotaccess FTDC data independent of system owners or operators. MongoDBprocesses run with FTDC on by default. For more information on MongoDBSupport options, visitGetting Started With MongoDB Support.

FTDC Privacy

FTDC data files are compressed and not human-readable. MongoDB Inc.engineers cannot access FTDC data without explicit permissionand assistance from system owners or operators.

FTDC data never contains any of the following information:

  • Samples of queries, query predicates, or query results
  • Data sampled from any end-user collection or index
  • System or MongoDB user credentials or security certificates

FTDC data contains certain host machine information such ashostnames, operating system information, and the options or settingsused to start the mongod ormongos. This information may beconsidered protected or confidential by some organizations orregulatory bodies, but is not typically considered to be PersonallyIdentifiable Information (PII). For clusters where these fields wereconfigured with protected, confidential, or PII data, please notifyMongoDB Inc. engineers before sending the FTDC data so appropriatemeasures can be taken.

FTDC periodically collects statistics produced by the followingcommands:

Depending on the host operating system, the diagnostic data may includeone or more of the following statistics:

  • CPU utilization
  • Memory utilization
  • Disk utilization related to performance. FTDC does not includedata related to storage capacity.
  • Network performance statistics. FTDC only captures metadata anddoes not capture or inspect any network packets.

FTDC collects statistics produced by the following commands onfile rotation or startup:

mongod processes store FTDC data files in adiagnostic.data directory under the instancesstorage.dbPath. All diagnostic data files are storedunder this directory. For example, given a dbPathof /data/db, the diagnostic data directory would be/data/db/diagnostic.data.

mongos processes store FTDC data files in adiagnostic directory relative to the systemLog.path logpath setting. MongoDB truncates the logpath’s file extension andconcatenates diagnostic.data to the remaining name. For example,given a path setting of /var/log/mongos.log,the diagnostic data directory would be/var/log/mongos.diagnostic.data.

FTDC runs with the following defaults:

  • Data capture every 1 second
  • 200MB maximum diagnostic.data folder size.

These defaults are designed to provide useful data to MongoDB Inc.engineers with minimal impact on performance or storage size. Thesevalues only require modifications if requested by MongoDB Inc. engineersfor specific diagnostic purposes.

You can view the FTDC source code on theMongoDB Github Repository.The ftdcsystem_stats*.ccp files specifically define anysystem-specific diagnostic data captured.

To disable FTDC, start up the mongod ormongos with thediagnosticDataCollectionEnabled: false option specified to thesetParameter setting in your configuration file:

  1. setParameter:
  2.   diagnosticDataCollectionEnabled: false

Disabling FTDC may increase the time or resources required whenanalyzing or debugging issues with support from MongoDB Inc. engineers.