Journaling

To provide durability in the event of a failure, MongoDB uses writeahead logging to on-disk journal files.

Journaling and the WiredTiger Storage Engine

Important

The log mentioned in this section refers to the WiredTigerwrite-ahead log (i.e. the journal) and not the MongoDB log file.

WiredTiger uses checkpoints to provide a consistent view of dataon disk and allow MongoDB to recover from the last checkpoint. However,if MongoDB exits unexpectedly in between checkpoints, journaling isrequired to recover information that occurred after the last checkpoint.

Note

Starting in MongoDB 4.0, you cannot specify —nojournal option or storage.journal.enabled:false for replica set members that use theWiredTiger storage engine.

With journaling, the recovery process:

  • Looks in the data files to find the identifier of the lastcheckpoint.
  • Searches in the journal files for the record thatmatches the identifier of the last checkpoint.
  • Apply the operations in the journal files since the last checkpoint.

Journaling Process

Changed in version 3.2.

With journaling, WiredTiger creates one journal record for each clientinitiated write operation. The journal record includes any internalwrite operations caused by the initial write. For example, an update toa document in a collection may result in modifications to the indexes;WiredTiger creates a single journal record that includes both theupdate operation and its associated index modifications.

MongoDB configures WiredTiger to use in-memory buffering for storingthe journal records. Threads coordinate to allocate and copyinto their portion of the buffer. All journal records up to 128kB are buffered.

WiredTiger syncs the buffered journal records to disk upon any ofthe following conditions:

  • For replica set members (primary and secondary members),

    • If there are operations waiting for oplog entries. Operationsthat can wait for oplog entries include:
    • Additionally for secondary members, after every batchapplication of the oplog entries.
  • If a write operation includes or implies a write concern ofj: true.

Note

Write concern "majority" implies j: true ifthe writeConcernMajorityJournalDefault is true.

  • At every 100 milliseconds (See storage.journal.commitIntervalMs).

  • When WiredTiger creates a new journal file. Because MongoDB uses ajournal file size limit of 100 MB, WiredTiger creates a newjournal file approximately every 100 MB of data.

Important

In between write operations, while the journal recordsremain in the WiredTiger buffers, updates can be lost following ahard shutdown of mongod.

See also

The serverStatus command returns information on theWiredTiger journal statistics in the wiredTiger.log field.

Journal Files

For the journal files, MongoDB creates a subdirectory named journalunder the dbPath directory. WiredTiger journalfiles have names with the following format WiredTigerLog.<sequence>where <sequence> is a zero-padded number starting from0000000001.

Journal Records

Journal files contain a record per each client initiated write operation

  • The journal record includes any internal write operations caused bythe initial write. For example, an update to a document in acollection may result in modifications to the indexes; WiredTigercreates a single journal record that includes both the updateoperation and its associated index modifications.
  • Each record has a unique identifier.
  • The minimum journal record size for WiredTiger is 128 bytes.

Compression

By default, MongoDB configures WiredTiger to use snappy compression forits journaling data. To specify a different compression algorithm or nocompression, use thestorage.wiredTiger.engineConfig.journalCompressor setting.For details, see Change WiredTiger Journal Compressor.s

Note

If a log record less than or equal to 128 bytes (the mininumlog record size for WiredTiger),WiredTiger does not compress that record.

Journal File Size Limit

WiredTiger journal files for MongoDB have a maximum size limit ofapproximately 100 MB.

  • Once the file exceeds that limit, WiredTiger creates a new journalfile.
  • WiredTiger automatically removes old journal files to maintain onlythe files needed to recover from last checkpoint.

Pre-Allocation

WiredTiger pre-allocates journal files.

Journaling and the In-Memory Storage Engine

Starting in MongoDB Enterprise version 3.2.6, the In-MemoryStorage Engine is part of general availability (GA).Because its data is kept in memory, there is no separate journal. Writeoperations with a write concern of j: true areimmediately acknowledged.

If any voting member of a replica set uses the in-memorystorage engine, you must setwriteConcernMajorityJournalDefault to false.

Note

Starting in version 4.2 (and 4.0.13 and 3.6.14 ), if a replica setmember uses the in-memory storage engine(voting or non-voting) but the replica set haswriteConcernMajorityJournalDefault set to true, thereplica set member logs a startup warning.

With writeConcernMajorityJournalDefault set to false,MongoDB does not wait for w: "majority"writes to be written to the on-disk journal before acknowledging thewrites. As such, majority write operations couldpossibly roll back in the event of a transient loss (e.g. crash andrestart) of a majority of nodes in a given replica set.

See also

In-Memory Storage Engine: Durability