Redis proxy

Statistics

Every configured Redis proxy filter has statistics rooted at redis.<stat_prefix>. with the following statistics:

Name

Type

Description

downstream_cx_active

Gauge

Total active connections

downstream_cx_protocol_error

Counter

Total protocol errors

downstream_cx_rx_bytes_buffered

Gauge

Total received bytes currently buffered

downstream_cx_rx_bytes_total

Counter

Total bytes received

downstream_cx_total

Counter

Total connections

downstream_cx_tx_bytes_buffered

Gauge

Total sent bytes currently buffered

downstream_cx_tx_bytes_total

Counter

Total bytes sent

downstream_cx_drain_close

Counter

Number of connections closed due to draining

downstream_rq_active

Gauge

Total active requests

downstream_rq_total

Counter

Total requests

Splitter statistics

The Redis filter will gather statistics for the command splitter in the redis.<stat_prefix>.splitter. with the following statistics:

Name

Type

Description

invalid_request

Counter

Number of requests with an incorrect number of arguments

unsupported_command

Counter

Number of commands issued which are not recognized by the command splitter

Per command statistics

The Redis filter will gather statistics for commands in the redis.<stat_prefix>.command.<command>. namespace. By default latency stats are in milliseconds and can be changed to microseconds by setting the configuration parameter latency_in_micros to true.

Name

Type

Description

total

Counter

Number of commands

success

Counter

Number of commands that were successful

error

Counter

Number of commands that returned a partial or complete error response

latency

Histogram

Command execution time in milliseconds (including delay faults)

error_fault

Counter

Number of commands that had an error fault injected

delay_fault

Counter

Number of commands that had a delay fault injected

Runtime

The Redis proxy filter supports the following runtime settings:

redis.drain_close_enabled

% of connections that will be drain closed if the server is draining and would otherwise attempt a drain close. Defaults to 100.

Fault Injection

The Redis filter can perform fault injection. Currently, Delay and Error faults are supported. Delay faults delay a request, and Error faults respond with an error. Moreover, errors can be delayed.

Note that the Redis filter does not check for correctness in your configuration - it is the user’s responsibility to make sure both the default and runtime percentages are correct! This is because percentages can be changed during runtime, and validating correctness at request time is expensive. If multiple faults are specified, the fault injection percentage should not exceed 100% for a given fault and Redis command combination. For example, if two faults are specified; one applying to GET at 60 %, and one applying to all commands at 50%, that is a bad configuration as GET now has 110% chance of applying a fault. This means that every request will have a fault.

If a delay is injected, the delay is additive - if the request took 400ms and a delay of 100ms is injected, then the total request latency is 500ms. Also, due to implementation of the redis protocol, a delayed request will delay everything that comes in after it, due to the proxy’s need to respect the order of commands it receives.

Note that faults must have a fault_enabled field, and are not enabled by default (if no default value or runtime key are set).

Example configuration:

  1. faults:
  2. - fault_type: ERROR
  3. fault_enabled:
  4. default_value:
  5. numerator: 10
  6. denominator: HUNDRED
  7. runtime_key: "bogus_key"
  8. commands:
  9. - GET
  10. - fault_type: DELAY
  11. fault_enabled:
  12. default_value:
  13. numerator: 10
  14. denominator: HUNDRED
  15. runtime_key: "bogus_key"
  16. delay: 2s

This creates two faults- an error, applying only to GET commands at 10%, and a delay, applying to all commands at 10%. This means that 20% of GET commands will have a fault applied, as discussed earlier.