我的书签
添加书签
移除书签WiredTiger Storage Engine
Starting in MongoDB 3.2, the WiredTiger storage engine is the defaultstorage engine. For existing deployments, if you do not specify the—storageEngine or the storage.engine setting, theversion 3.2+ mongod instance can automatically determinethe storage engine used to create the data files in the —dbpath orstorage.dbPath. See Default Storage Engine Change.
Document Level Concurrency
WiredTiger uses document-level concurrency control for writeoperations. As a result, multiple clients can modify differentdocuments of a collection at the same time.
For most read and write operations, WiredTiger uses optimisticconcurrency control. WiredTiger uses only intent locks at the global,database and collection levels. When the storage engine detectsconflicts between two operations, one will incur a write conflictcausing MongoDB to transparently retry that operation.
Some global operations, typically short lived operations involvingmultiple databases, still require a global “instance-wide” lock.Some other operations, such as dropping a collection, still requirean exclusive database lock.
Snapshots and Checkpoints
WiredTiger uses MultiVersion Concurrency Control (MVCC). At the startof an operation, WiredTiger provides a point-in-time snapshot of thedata to the operation. A snapshot presents a consistent view of thein-memory data.
When writing to disk, WiredTiger writes all the data in a snapshot todisk in a consistent way across all data files. The now-durabledata act as a checkpoint in the data files. The checkpoint ensuresthat the data files are consistent up to and including the lastcheckpoint; i.e. checkpoints can act as recovery points.
Starting in version 3.6, MongoDB configures WiredTiger to createcheckpoints (i.e. write the snapshot data to disk) at intervalsof 60 seconds. In earlier versions, MongoDB sets checkpoints tooccur in WiredTiger on user data at an interval of 60 seconds orwhen 2 GB of journal data has been written, whichever occursfirst.
During the write of a new checkpoint, the previous checkpoint is stillvalid. As such, even if MongoDB terminates or encounters an error whilewriting a new checkpoint, upon restart, MongoDB can recover from thelast valid checkpoint.
The new checkpoint becomes accessible and permanent when WiredTiger’smetadata table is atomically updated to reference the new checkpoint.Once the new checkpoint is accessible, WiredTiger frees pages from theold checkpoints.
Using WiredTiger, even without journaling, MongoDB can recover from the lastcheckpoint; however, to recover changes made after the last checkpoint,run with journaling.
Note
Starting in MongoDB 4.0, you cannot specify —nojournal option or storage.journal.enabled:false for replica set members that use theWiredTiger storage engine.
Journal
WiredTiger uses a write-ahead log (i.e. journal) in combination withcheckpoints to ensure datadurability.
The WiredTiger journal persists all data modifications betweencheckpoints. If MongoDB exits between checkpoints, it uses the journalto replay all data modified since the last checkpoint. For informationon the frequency with which MongoDB writes the journal data to disk,see Journaling Process.
WiredTiger journal is compressed using the snappy compressionlibrary. To specify a different compression algorithm or nocompression, use thestorage.wiredTiger.engineConfig.journalCompressor setting.For details on changing the journal compressor, seeChange WiredTiger Journal Compressor.
Note
If a log record less than or equal to 128 bytes (the mininumlog record size for WiredTiger),WiredTiger does not compress that record.
You can disable journaling for standalone instances by settingstorage.journal.enabled to false, which can reduce theoverhead of maintaining the journal. For standalone instances,not using the journal means that, when MongoDB exits unexpectedly, youwill lose all data modifications prior to the last checkpoint.
Note
Starting in MongoDB 4.0, you cannot specify —nojournal option or storage.journal.enabled:false for replica set members that use theWiredTiger storage engine.
See also
Compression
With WiredTiger, MongoDB supports compression for all collections andindexes. Compression minimizes storage use at the expense of additionalCPU.
By default, WiredTiger uses block compression with the snappycompression library for all collections and prefix compressionfor all indexes.
For collections, the following block compression libraries are also available:
To specify an alternate compression algorithm or no compression, usethe storage.wiredTiger.collectionConfig.blockCompressorsetting.
For indexes, to disable prefix compression, use thestorage.wiredTiger.indexConfig.prefixCompression setting.
Compression settings are also configurable on a per-collection andper-index basis during collection and index creation. SeeSpecify Storage Engine Options anddb.collection.createIndex() storageEngine option.
For most workloads, the default compression settings balance storageefficiency and processing requirements.
The WiredTiger journal is also compressed by default. For informationon journal compression, see Journal.
Memory Use
With WiredTiger, MongoDB utilizes both the WiredTiger internal cacheand the filesystem cache.
Starting in MongoDB 3.4, the default WiredTiger internal cache size isthe larger of either:
- 50% of (RAM - 1 GB), or
- 256 MB.
For example, on a system with a total of 4GB of RAM the WiredTigercache will use 1.5GB of RAM (0.5 (4 GB - 1 GB) = 1.5 GB).Conversely, a system with a total of 1.25 GB of RAM will allocate 256MB to the WiredTiger cache because that is more than half of thetotal RAM minus one gigabyte (0.5 (1.25 GB - 1 GB) = 128 MB < 256 MB).
Note
In some instances, such as when running in a container, the databasecan have memory constraints that are lower than the total systemmemory. In such instances, this memory limit, rather than the totalsystem memory, is used as the maximum RAM available.
To see the memory limit, see hostInfo.system.memLimitMB.
By default, WiredTiger uses Snappy block compression for all collectionsand prefix compression for all indexes. Compression defaults are configurableat a global level and can also be set on a per-collection and per-indexbasis during collection and index creation.
Different representations are used for data in the WiredTiger internal cacheversus the on-disk format:
- Data in the filesystem cache is the same as the on-disk format, includingbenefits of any compression for data files. The filesystem cache is usedby the operating system to reduce disk I/O.
- Indexes loaded in the WiredTiger internal cache have a different datarepresentation to the on-disk format, but can still take advantage ofindex prefix compression to reduce RAM usage. Index prefix compressiondeduplicates common prefixes from indexed fields.
- Collection data in the WiredTiger internal cache is uncompressedand uses a different representation from the on-disk format. Blockcompression can provide significant on-disk storage savings, butdata must be uncompressed to be manipulated by the server.
Via the filesystem cache, MongoDB automatically uses all free memorythat is not used by the WiredTiger cache or by other processes.
To adjust the size of the WiredTiger internal cache, seestorage.wiredTiger.engineConfig.cacheSizeGB and—wiredTigerCacheSizeGB. Avoid increasing the WiredTigerinternal cache size above its default value.
