Sizing a database environment

The Berkeley DB environment allocates memory to hold shared structures, either in shared regions or in process data space (if the DB_PRIVATE flag is specified). There are three distinct memory regions:

• The memory pool (also known as the database page cache),

• the area containing mutexes, and

• the main region which holds all other shared structures.

The shared structures in the main region are used by the lock, transaction, logging, thread and replicatoin subsystems.

Determining the amount of space allocated for each of these shared structures is dependent upon the structure in question. The sizing of the memory pool is discussed in Configuring the memory pool. The amount of memory needed for mutexes is calculated from the number of mutexes needed by various subsystems and can be adjusted using the DB_ENV->mutex_set_increment() method.

For applications using shared memory (that is, they do not specify DB_PRIVATE), a maximum memory size for the main region must be specified or left to default. The maximum memory size is specified using the DB_ENV->set_memory_max() method.

The amount of memory needed by an application is dependent on the resources that the application uses. For a very rough estimate, add all of the following together:

1. The environment has an overhead of about 80 kilobytes without statistics enabled or 250 kilobytes with statistics enabled.

2. Identify the amount of space you require for your locks:

   1. Estimate the number of threads of control that will simultaneously access the environment.

   2. Estimate the number of concurrency locks that, on average, will be required by each thread. For information on sizing concurrency locks, see Configuring locking: sizing the system.

   3. Multiply these two numbers, then multiply by 1/2 to arrive at the number of kilobytes required to service your locks.

3. Estimate the number of open database handles you will use at any given time. For each database handle, there is an overhead of about 1/2 kilobyte.

4. Add 1 kilobyte for each active transaction.

Note that these are very rough guidelines. It is best to overestimate the needs of your applications, because if the memory allocation is exhausted the application must be shutdown to increase the allocation.

The estimate for maximum memory need not be exact. In most situations there is little penalty for over estimating. For systems using memory mapped files for the shared environment, this only allocates the address space in the process to hold the maximum memory. The backing file will only be extended as needed. For systems running with DB_PRIVATE specified, the maximum memory serves only as a limit and memory is allocated from the process data space as needed. No maximum need be set for private environments.

For locking and thread information, groups of objects are allocated when needed so that there is less contention in the allocator during performance critical operations. Once allocated to a particular use, this memory will only be used for that structure. To avoid runtime contention, or to ensure a minimum number of a particular type of object, the DB_ENV->set_memory_init() method can be used. This method can set the initial numbers of particular types of structures to allocate at environment creation time.