Back Up and Restore with Filesystem Snapshots

This document describes a procedure for creating backups of MongoDBsystems using system-level tools, such as LVM or storageappliance, as well as the corresponding restoration strategies.

These filesystem snapshots, or “block-level” backup methods, use systemlevel tools to create copies of the device that holds MongoDB’s datafiles. These methods complete quickly and work reliably, but requireadditional system configuration outside of MongoDB.

See also

MongoDB Backup Methods andBack Up and Restore with MongoDB Tools.

Snapshots Overview

Snapshots work by creating pointers between the live data and aspecial snapshot volume. These pointers are theoretically equivalentto “hard links.” As the working data diverges from the snapshot,the snapshot process uses a copy-on-write strategy. As a result, the snapshotonly stores modified data.

After making the snapshot, you mount the snapshot image on yourfile system and copy data from the snapshot. The resulting backupcontains a full copy of all data.

Considerations

WiredTiger Storage Engine

MongoDB 3.2 added support for volume-level back up of MongoDB instancesusing the WiredTiger storage engine whenthe MongoDB instance’s data files and journal files reside on separatevolumes. However, to create a coherent backup, the database must be lockedand all writes to the database must be suspended during the backup process.

Prior to MongoDB 3.2, creating volume-level backupsof MongoDB instances using WiredTiger required that the data files and journalreside on the same volume.

Encrypted Storage Engine (MongoDB Enterprise Only)

For encrypted storage engines thatuse AES256-GCM encryption mode, AES256-GCM requires that everyprocess use a unique counter block value with the key.

For encrypted storage engineconfigured with AES256-GCM cipher:

    • Restoring from Hot Backup
    • Starting in 4.2, if you restore from files taken via “hot”backup (i.e. the mongod is running), MongoDBcan detect “dirty” keys on startup and automatically rolloverthe database key to avoid IV (Initialization Vector) reuse.
    • Restoring from Cold Backup
    • However, if you restore from files taken via “cold” backup(i.e. the mongod is not running), MongoDBcannot detect “dirty” keys on startup, and reuse of IV voidsconfidentiality and integrity guarantees.

Starting in 4.2, to avoid the reuse of the keys afterrestoring from a cold filesystem snapshot, MongoDB adds a newcommand-line option —eseDatabaseKeyRollover. When started with the—eseDatabaseKeyRollover option, the mongodinstance rolls over the database keys configured withAES256-GCM cipher and exits.

Tip

  • In general, if using filesystem based backups for MongoDBEnterprise 4.2+, use the “hot” backup feature, if possible.
  • For MongoDB Enterprise versions 4.0 and earlier, if you useAES256-GCM encryption mode, do not make copies ofyour data files or restore from filesystem snapshots (“hot” or“cold”).

Valid Database at the Time of Snapshot

The database must be valid when the snapshot takes place. This meansthat all writes accepted by the database need to be fully written todisk: either to the journal or to data files.

If there are writes that are not on disk when the backup occurs, the backupwill not reflect these changes.

For the WiredTiger storage engine, the datafiles reflect a consistent state as of the last checkpoint. Checkpoints occur with every 2 GB ofdata or every minute.

Entire Disk Image

Snapshots create an image of an entire disk image. Unless you need toback up your entire system, consider isolating your MongoDB data files,journal (if applicable), and configuration on one logical disk thatdoesn’t contain any other data.

Alternately, store all MongoDB data files on a dedicated deviceso that you can make backups without duplicating extraneous data.

Site Failure Precaution

Ensure that you copy data from snapshots onto other systems. Thisensures that data is safe from site failures.

No Incremental Backups

This tutorial does not include procedures for incremental backups.Although different snapshot methods provide different features, theLVM method outlined below does not provide any capacity for capturingincremental backups.

Snapshots With Journaling

If your mongod instance has journaling enabled, then you canuse any kind of file system or volume/block level snapshot tool tocreate backups.

If you manage your own infrastructure on a Linux-based system, configureyour system with LVM to provide your disk packages and providesnapshot capability. You can also use LVM-based setups within acloud/virtualized environment.

Note

Running LVM provides additional flexibility and enables thepossibility of using snapshots to back up MongoDB.

Snapshots with Amazon EBS in a RAID 10 Configuration

If your deployment depends on Amazon’s Elastic Block Storage (EBS) withRAID configured within your instance, it is impossible to get aconsistent state across all disks using the platform’s snapshot tool. Asan alternative, you can do one of the following:

  • Flush all writes to disk and create a write lock to ensureconsistent state during the backup process.

If you choose this option see Back up Instances with Journal Files on Separate Volume or without Journaling.

  • Configure LVM to run and hold your MongoDB data files on top of theRAID within your system.

If you choose this option, perform the LVM backup operation describedin Create a Snapshot.

Back Up and Restore Using LVM on Linux

This section provides an overview of a simple backup processusing LVM on a Linux system. While the tools, commands, and paths maybe (slightly) different on your system the following steps provide ahigh level overview of the backup operation.

Note

Only use the following procedure as a guideline for a backup systemand infrastructure. Production backup systems must consider a numberof application specific requirements and factors unique to specificenvironments.

Create a Snapshot

Changed in version 3.2: Starting in MongoDB 3.2, for the purpose of volume-level backup ofMongoDB instances using WiredTiger, the data files and the journalare no longer required to reside on a single volume.

To create a snapshot with LVM, issue a command as root in thefollowing format:

  1. lvcreate --size 100M --snapshot --name mdb-snap01 /dev/vg0/mongodb

This command creates an LVM snapshot (with the —snapshot option)named mdb-snap01 of the mongodb volume in the vg0volume group.

This example creates a snapshot named mdb-snap01 located at/dev/vg0/mdb-snap01. The location and paths to your systems volumegroups and devices may vary slightly depending on your operatingsystem’s LVM configuration.

The snapshot has a cap of at 100 megabytes, because of the parameter—size 100M. This size does not reflect the total amount of thedata on the disk, but rather the quantity of differences between thecurrent state of /dev/vg0/mongodb and the creation of the snapshot(i.e. /dev/vg0/mdb-snap01.)

Warning

Ensure that you create snapshots with enough space to account fordata growth, particularly for the period of time that it takes to copydata out of the system or to a temporary image.

If your snapshot runs out of space, the snapshot imagebecomes unusable. Discard this logical volume and create another.

The snapshot will exist when the command returns. You can restoredirectly from the snapshot at any time or by creating a new logicalvolume and restoring from this snapshot to the alternate image.

While snapshots are great for creating high quality backupsquickly, they are not ideal as a format for storing backupdata. Snapshots typically depend and reside on the same storageinfrastructure as the original disk images. Therefore, it’s crucialthat you archive these snapshots and store them elsewhere.

Archive a Snapshot

After creating a snapshot, mount the snapshot and copy the data toseparate storage. Your system might try to compress the backup images asyou move them offline. Alternatively, take a block level copy of thesnapshot image, such as with the following procedure:

  1. umount /dev/vg0/mdb-snap01
  2. dd if=/dev/vg0/mdb-snap01 | gzip > mdb-snap01.gz

The above command sequence does the following:

  • Ensures that the /dev/vg0/mdb-snap01 device is not mounted. Nevertake a block level copy of a filesystem or filesystem snapshot that ismounted.

  • Performs a block level copy of the entire snapshot image using the ddcommand and compresses the result in a gzipped file in thecurrent working directory.

Warning

This command will create a large gz file in your currentworking directory. Make sure that you run this command in a filesystem that has enough free space.

Restore a Snapshot

To restore a snapshot created with LVM, issue the followingsequence of commands:

  1. lvcreate --size 1G --name mdb-new vg0
  2. gzip -d -c mdb-snap01.gz | dd of=/dev/vg0/mdb-new
  3. mount /dev/vg0/mdb-new /srv/mongodb

The above sequence does the following:

  • Creates a new logical volume named mdb-new, in the /dev/vg0volume group. The path to the new device will be /dev/vg0/mdb-new.

Warning

This volume will have a maximum size of 1 gigabyte. The originalfile system must have had a total size of 1 gigabyte or smaller, orelse the restoration will fail.

Change 1G to your desired volume size.

  • Uncompresses and unarchives the mdb-snap01.gz into themdb-new disk image.

  • Mounts the mdb-new disk image to the /srv/mongodb directory.Modify the mount point to correspond to your MongoDB data filelocation, or other location as needed.

Note

The restored snapshot will have a stale mongod.lock file. Ifyou do not remove this file from the snapshot, and MongoDB mayassume that the stale lock file indicates an unclean shutdown. Ifyou’re running with storage.journal.enabled enabled, andyou do not use db.fsyncLock(), you do not need toremove the mongod.lock file. If you usedb.fsyncLock() you will need to remove the lock.

Restore Directly from a Snapshot

To restore a backup without writing to a compressed gz file, usethe following sequence of commands:

  1. umount /dev/vg0/mdb-snap01
  2. lvcreate --size 1G --name mdb-new vg0
  3. dd if=/dev/vg0/mdb-snap01 of=/dev/vg0/mdb-new
  4. mount /dev/vg0/mdb-new /srv/mongodb

Note

New in version 3.6:

All MongoDB collections haveUUIDs by default. WhenMongoDB restores collections, the restored collections retain theiroriginal UUIDs. When restoring a collection where no UUID waspresent, MongoDB generates a UUID for the restored collection.

For more information on collection UUIDs, seeCollections.

Remote Backup Storage

You can implement off-system backups using the combined process and SSH.

This sequence is identical to procedures explained above, except that itarchives and compresses the backup on a remote system using SSH.

Consider the following procedure:

  1. umount /dev/vg0/mdb-snap01
  2. dd if=/dev/vg0/mdb-snap01 | ssh username@example.com gzip > /opt/backup/mdb-snap01.gz
  3. lvcreate --size 1G --name mdb-new vg0
  4. ssh username@example.com gzip -d -c /opt/backup/mdb-snap01.gz | dd of=/dev/vg0/mdb-new
  5. mount /dev/vg0/mdb-new /srv/mongodb

Back up Instances with Journal Files on Separate Volume or without Journaling

Changed in version 3.2: Starting in MongoDB 3.2, for the purpose of volume-level backup ofMongoDB instances using WiredTiger, the data files and the journalare no longer required to reside on a single volume. However, the databasemust be locked and all writes to the database must be suspended during thebackup process to ensure the consistency of the backup.

If your mongod instance is either running without journalingor has the journal files on a separate volume, you must flush allwrites to disk and lock the database to prevent writes during thebackup process. If you have a replica set configuration, thenfor your backup use a secondary which is not receiving reads(i.e. hidden member).

Flush writes to disk and lock the database to prevent further writes.

To flush writes to disk and to “lock” the database, issue thedb.fsyncLock() method in the mongo shell:

  1. db.fsyncLock();

Perform the backup operation described in Create a Snapshot.

After the snapshot completes, unlock the database.

To unlock the database after the snapshot has completed, use thefollowing command in the mongo shell:

  1. db.fsyncUnlock();