Settings

Shard indexing backpressure adds several settings to the standard OpenSearch cluster settings. They are dynamic, so you can change the default behavior of this feature without restarting your cluster.

High-level controls

The high-level controls allow you to turn the shard indexing backpressure feature on or off.

Setting	Default	Description
`shard_indexing_pressure.enabled`	False	Change to `true` to enable shard indexing backpressure.
`shard_indexing_pressure.enforced`	False	Run shard indexing backpressure in shadow mode or enforced mode. In shadow mode (value set as `false`), shard indexing backpressure tracks all granular-level metrics, but it doesn’t actually reject any indexing requests. In enforced mode (value set as `true`), shard indexing backpressure rejects any requests to the cluster that might cause a dip in its performance.

Node-level limits

Node-level limits allow you to control memory usage on a node.

Setting	Default	Description
`shard_indexing_pressure.primary_parameter.node.soft_limit`	70%	Define the percentage of the node-level memory threshold that acts as a soft indicator for strain on a node.

Shard-level limits

Shard-level limits allow you to control memory usage on a shard.

Setting	Default	Description
`shard_indexing_pressure.primary_parameter.shard.min_limit`	0.001d	Specify the minimum assigned quota for a new shard in any role (coordinator, primary, or replica). Shard indexing backpressure increases or decreases this allocated quota based on the inflow of traffic for the shard.
`shard_indexing_pressure.operating_factor.lower`	75%	Specify the lower occupancy limit of the allocated quota of memory for the shard. If the total memory usage of a shard is below this limit, shard indexing backpressure decreases the current allocated memory for that shard.
`shard_indexing_pressure.operating_factor.optimal`	85%	Specify the optimal occupancy of the allocated quota of memory for the shard. If the total memory usage of a shard is at this level, shard indexing backpressure doesn’t change the current allocated memory for that shard.
`shard_indexing_pressure.operating_factor.upper`	95%	Specify the upper occupancy limit of the allocated quota of memory for the shard. If the total memory usage of a shard is above this limit, shard indexing backpressure increases the current allocated memory for that shard.

Performance degradation factors

The performance degradation factors allow you to control the dynamic performance thresholds for a shard.

Setting	Default	Description
`shard_indexing_pressure.secondary_parameter.throughput.request_size_window`	2,000	The number of requests in the sampling window size on a shard. Shard indexing backpressure compares the overall performance of requests with the requests in the sample window to detect any performance degradation.
`shard_indexing_pressure.secondary_parameter.throughput.degradation_factor`	5x	The degradation factor per unit byte for a request. This parameter determines the threshold for any latency spikes. The default value is 5x, which implies that if the latency shoots up 5 times in the historic view, shard indexing backpressure marks it as a performance degradation.
`shard_indexing_pressure.secondary_parameter.successful_request.elapsed_timeout`	300000 ms	The amount of time a request is pending in a cluster. This parameter helps identify any stuck-request scenarios.
`shard_indexing_pressure.secondary_parameter.successful_request.max_outstanding_requests`	100	The maximum number of pending requests in a cluster.