Checkpoints

The second component of the infrastructure is performing checkpoints of the log files. Performing checkpoints is necessary for two reasons.

First, you may be able to remove Berkeley DB log files from your database environment after a checkpoint. Change records are written into the log files when databases are modified, but the actual changes to the database are not necessarily written to disk. When a checkpoint is performed, changes to the database are written into the backing database file. Once the database pages are written, log files can be archived and removed from the database environment because they will never be needed for anything other than catastrophic failure. (Log files which are involved in active transactions may not be removed, and there must always be at least one log file in the database environment.)

The second reason to perform checkpoints is because checkpoint frequency is inversely proportional to the amount of time it takes to run database recovery after a system or application failure. This is because recovery after failure has to redo or undo changes only since the last checkpoint, as changes before the checkpoint have all been flushed to the databases.

Berkeley DB provides the db_checkpoint utility, which can be used to perform checkpoints. Alternatively, applications can write their own checkpoint thread using the underlying DB_ENV->txn_checkpoint() function. The following code fragment checkpoints the database environment every 60 seconds:

  1. int
  2. main(int argc, char *argv)
  3. {
  4.     extern int optind;
  5.     DB *db_cats, *db_color, *db_fruit;
  6.     DB_ENV *dbenv;
  7.     pthread_t ptid;
  8.     int ch;
  10.     while ((ch = getopt(argc, argv, "")) != EOF)
  11.         switch (ch) {
  12.         case '?':
  13.         default:
  14.             usage();
  15.         }
  16.     argc -= optind;
  17.     argv += optind;
  19.     env_dir_create();
  20.     env_open(&dbenv);
  22.     /* Start a checkpoint thread. */
  23.     if ((errno = pthread_create(
  24.         &ptid, NULL, checkpoint_thread, (void *)dbenv)) != 0) {
  25.         fprintf(stderr,
  26.             "txnapp: failed spawning checkpoint thread: %s\n",
  27.             strerror(errno));
  28.         exit (1);
  29.     }
  31.     /* Open database: Key is fruit class; Data is specific type. */
  32.     db_open(dbenv, &db_fruit, "fruit", 0);
  34.     /* Open database: Key is a color; Data is an integer. */
  35.     db_open(dbenv, &db_color, "color", 0);
  37.     /*
  38.      * Open database:
  39.      *    Key is a name; Data is: company name, cat breeds.
  40.      */
  41.     db_open(dbenv, &db_cats, "cats", 1);
  43.     add_fruit(dbenv, db_fruit, "apple", "yellow delicious");
  45.     add_color(dbenv, db_color, "blue", 0);
  46.     add_color(dbenv, db_color, "blue", 3);
  48.     add_cat(dbenv, db_cats,
  49.         "Amy Adams",
  50.         "Oracle",
  51.         "abyssinian",
  52.         "bengal",
  53.         "chartreaux",
  54.         NULL);
  56.     return (0);
  57. }
  59. void *
  60. checkpoint_thread(void *arg)
  61. {
  62.     DB_ENV *dbenv;
  63.     int ret;
  65.     dbenv = arg;
  66.     dbenv->errx(dbenv, "Checkpoint thread: %lu", (u_long)pthread_self());
  68.     /* Checkpoint once a minute. */
  69.     for (;; sleep(60))
  70.         if ((ret = dbenv->txn_checkpoint(dbenv, 0, 0, 0)) != 0) {
  71.             dbenv->err(dbenv, ret, "checkpoint thread");
  72.             exit (1);
  73.         }
  75.     /* NOTREACHED */
  76. }

Because checkpoints can be quite expensive, choosing how often to perform a checkpoint is a common tuning parameter for Berkeley DB applications.