Managing replication directories and files

Replication database directory considerations

If your application is going to locate databases in any directory other than the environment home directory, you need to consider the directory structure for all sites. There are several recommendations to make in this area.

The use of absolute pathnames is strongly discouraged when replication is in use. Absolute pathnames will not work if there is more than one site on a single machine. Replication with absolute pathnames is unlikely to work across different machines unless great care is taken to make sure the entire path is exactly the same on every machine.

If the master uses a data directory, as specified via DB_ENV->add_data_dir() or DB_ENV->set_create_dir(), it is recommended that you create the same directory structure on all client sites. When the same directory structure appears on a master and the client, replication creates the client databases in the same directory as the master regardless of the local client directory settings. If a master directory is missing on a client, replication decides where to create the client databases by using the client’s local directory settings and the Berkeley DB file naming rules as described in File naming.

Managing replication internal files

Whether you use the Base API or the Replication Manager, replication creates a set of internal files that are normally stored on-disk in your environment home directory. These files contain metadata which is necessary for replication operations, and so you should never delete these files.

You can cause these files to not be stored on disk, but instead to be held entirely in-memory, by specifying the DB_REP_CONF_INMEM flag to the DB_ENV->rep_set_config() method. Doing this can improve your application’s data throughput by avoiding the disk I/O associated with these metadata files. However, in the event that your application is shut down, the contents of these files are lost. This results in some loss of functionality, including an increased chance that elections will fail, or that the wrong site will win an election. See the DB_REP_CONF_INMEM flag description for more information.

Note that turning on DB_REP_CONF_INMEM means that Replication Manager cannot store group membership changes persistently. This is because Replication Manager stores group membership information in an internal database, which is held in memory when DB_REP_CONF_INMEM is turned on. For this reason, if your Replication Manager application requires replication metadata to be stored in memory, then you must manually identify all the sites in your replication group using the DB_LEGACY site configuration attribute. Be aware that this configuration needs to be made permanent. (Normally, DB_LEGACY is used only on a temporary basis for the purpose of upgrading old Replication Manager applications.)

Do the following:

1. Shut down all the sites in your replication group.

2. For every site in your replication group:

   1. Configure a DB_SITE handle for the local site. Use DB_SITE->set_config() to indicate that this is a legacy site by setting the DB_LEGACY parameter.

   2. Configure a DB_SITE handle for every other site in the replication group. Set the DB_LEGACY parameter for each of these handles.

      Please pay careful attention to this step. To repeat: a DB_SITE handle MUST be configured for EVERY site in the replication group.

3. Restart all the sites in the replication group.

Alternatively, you can store persistent environment metadata files, including those required by replication, in a location other than your environment home directory. This is necessary if your environment home directory is on a device that is unstable, because the persistent metadata files cannot be lost or deleted. You do this using the DB_ENV->set_metadata_dir() method.

Note that you must configure the handling of your environment metadata consistently across your entire replication group. That is, if you place your replication metadata in-memory on one site, then it must be placed in-memory on all the sites in the group. Similarly, if you place your replication metadata files in a non-standard directory location on one site, then they must be placed in the exact same directory location on all the sites in your group.