Compaction can be triggered manually by calling the DB::CompactRange
or DB::CompactFiles
method. This is meant to be used by advanced users to implement custom compaction strategies, including but not limited to the following use cases -
- Optimize for read heavy workloads by compacting to lowest level after ingesting a large amount of data
- Force the data to go through the compaction filter in order to consolidate it
- Migrate to a new compaction configuration. For example, if changing number of levels,
CompactRange
can be called to compact to bottommost level and then move the files to a target level
The example code below shows how to use the APIs.
Options dbOptions;
DB* db;
Status s = DB::Open(dbOptions, "/tmp/rocksdb", &db);
// Write some data
...
Slice begin("key1");
Slice end("key100");
CompactRangeOptions options;
s = db->CompactRange(options, &begin, &end);
Or
CompactionOptions options;
std::vector<std::string> input_file_names;
int output_level;
...
Status s = db->CompactFiles(options, input_file_names, output_level);
CompactRange
The begin
and end
arguments define the key range to be compacted. The behavior varies depending on the compaction style being used by the db. In case of universal and FIFO compaction styles, the begin
and end
arguments are ignored and all files are compacted. Also, files in each level are compacted and left in the same level. For leveled compaction style, all files containing keys in the given range are compacted to the last level containing files. If either begin
or end
are NULL, it is taken to mean the key before all keys in the db or the key after all keys respectively.
If more than one thread calls manual compaction, only one will actually schedule it while the other threads will simply wait for the scheduled manual compaction to complete. If CompactRangeOptions::exclusive_manual_compaction
is set to true, the call will disable scheduling of automatic compaction jobs and wait for existing automatic compaction jobs to finish.
DB::CompactRange
waits while compaction is performed on the background threads and thus is a blocking call.
The CompactRangeOptions
supports the following options -
CompactRangeOptions::exclusive_manual_compaction
When set to true, no other compaction will run when this manual compaction is running. Default value istrue
CompactRangeOptions::change_level
,CompactRangeOptions::target_level
Together, these options control the level where the compacted files will be placed. Iftarget_level
is -1, the compacted files will be moved to the minimum level whose computed max_bytes is still large enough to hold the files. Intermediate levels must be empty. For example, if the files were initially compacted to L5 and L2 is the minimum level large enough to hold the files, they will be placed in L2 if L3 and L4 are empty or in L4 if L3 is non-empty. Iftarget_level
is positive, the compacted files will be placed in that level provided intermediate levels are empty. If any any of the intermediate levels are not empty, the compacted files will be left where they are.CompactRangeOptions::target_path_id
Compaction outputs will be placed in options.db_paths[target_path_id] directory.CompactRangeOptions::bottommost_level_compaction
When set toBottommostLevelCompaction::kSkip
, or when set toBottommostLevelCompaction::kIfHaveCompactionFilter
and a compaction filter is defined for the column family, the bottommost level files are not compacted.
CompactFiles
This API compacts all the input files into a set of output files in the output_level
. The number of output files is determined by the size of the data and the setting of CompactionOptions::output_file_size_limit
. This API is not supported in ROCKSDB_LITE.