Tiered cache

This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated GitHub issue.

A tiered cache is a multi-level cache in which each tier has its own characteristics and performance levels. By combining different tiers, you can achieve a balance between cache performance and size.

Types of tiered caches

OpenSearch provides an implementation of a _tiered spillover cache_. This implementation spills any items removed from the upper tiers to the lower tiers of cache. The upper tier, such as the on-heap tier, is smaller in size but offers better latency. The lower tier, such as the disk cache, is larger in size but slower in terms of latency. OpenSearch offers both on-heap and disk tiers.

Enabling a tiered cache

To enable a tiered cache, configure the following setting in opensearch.yml:

  1. opensearch.experimental.feature.pluggable.caching.enabled: true

copy

For more information about ways to enable experimental features, see Experimental feature flags.

Installing required plugins

To use tiered caching, install a tiered cache plugin. As of OpenSearch 2.13, the only available cache plugin is the cache-ehcache plugin. This plugin provides a disk cache implementation that can be used as a disk tier within a tiered cache. For more information about installing non-bundled plugins, see Additional plugins.

A tiered cache will fail to initialize if the cache-ehcache plugin is not installed or if disk cache properties are not set.

Tiered cache settings

In OpenSearch 2.14, a request cache can be used in a tiered cache. To begin, configure the following settings in the opensearch.yml file.

Cache store name

To use the OpenSearch-provided tiered spillover cache implementation, set the cache store name to tiered_spillover, as shown in the following example:

  1. indices.requests.cache.store.name: tiered_spillover

copy

Setting on-heap and disk store tiers

Set the on-heap and disk store tiers to opensearch_onheap and ehcache_disk, as shown in the following example:

  1. indices.requests.cache.tiered_spillover.onheap.store.name: opensearch_onheap
  2. indices.requests.cache.tiered_spillover.disk.store.name: ehcache_disk

The opensearch_onheap setting uses the built-in on-heap cache available in OpenSearch.

The ehcache_disk setting is the disk cache implementation from Ehcache and requires installing the cache-ehcache plugin.

copy

Configuring on-heap and disk stores

The following table lists the cache store settings for the opensearch_onheap store.

SettingData typeDefaultDescription
indices.requests.cache.opensearch_onheap.sizePercentage1% of the heap sizeThe size of the on-heap cache. Optional.
indices.requests.cache.opensearch_onheap.expireTime unitMAX_VALUE (disabled)Specifies a time-to-live (TTL) for the cached results. Optional.

The following table lists the disk cache store settings for the ehcache_disk store.

SettingData typeDefaultDescription
indices.requests.cache.ehcache_disk.max_size_in_bytesLong1073741824 (1 GB)Defines the size of the disk cache. Optional.
indices.requests.cache.ehcache_disk.storage.pathString{data.paths}/nodes/{node.id}/request_cacheDefines the storage path for the disk cache. Optional.
indices.requests.cache.ehcache_disk.expire_after_accessTime unitMAX_VALUE (disabled)Specifies a TTL for the cached results. Optional.
indices.requests.cache.ehcache_disk.aliasStringehcacheDiskCache#INDICES_REQUEST_CACHESpecifies an alias for the disk cache. Optional.
indices.requests.cache.ehcache_disk.segmentsInteger16Defines the number of segments into which the disk cache is separated. Used for concurrency. Optional.
indices.requests.cache.ehcache_disk.concurrencyInteger1Defines the number of distinct write queues created for the disk store, where a group of segments shares a write queue. Optional.

Additional settings for the tiered_spillover store

The following table lists additional settings for the tiered_spillover store setting.

SettingData typeDefaultDescription
indices.requests.cache.tiered_spillover.disk.store.policies.took_time.thresholdTime unit10msA policy used to determine whether to cache a query into a disk cache based on its took time. This is a dynamic setting. Optional.
indices.requests.cache.tiered_spillover.disk.store.enabledBooleanTrueEnables or disables the disk cache dynamically within a tiered spillover cache. Note: After disabling a disk cache, entries are not removed automatically and requires the cache to be manually cleared. Optional.
indices.requests.cache.tiered_spillover.onheap.store.sizePercentage1% of the heap sizeDefines the size of the on-heap cache within tiered cache. Optional.
indices.requests.cache.tiered_spillover.disk.store.sizeLong1073741824 (1 GB)Defines the size of the disk cache within tiered cache. Optional.
indices.requests.cache.tiered_spillover.segmentsInteger2 ^ (ceil(log2(CPU_CORES * 1.5)))This determines the number of segments in the tiered cache, with each segment secured by a re-entrant read/write lock. These locks enable multiple concurrent readers without contention, while the segmentation allows multiple writers to operate simultaneously, resulting in higher write throughput. Optional.

Delete stale entries settings

The following table lists the settings related to the deletion of stale entries from the cache.

SettingData typeDefaultDescription
indices.requests.cache.cleanup.staleness_thresholdString0%Defines the percentage of stale keys in the cache post. After identification, all stale cache entries are deleted. Optional.
indices.requests.cache.cleanup.intervalTime unit1mDefines the frequency at which the request cache’s stale entries are deleted. Optional.

Getting statistics for the tiered_spillover store

To assess the impact of using the tiered spillover cache, use the Node Stats API, as shown in the following example:

  1. GET /_nodes/stats/caches/request_cache?level=tier