Configure Spinnaker’s Usage of Redis

Several Spinnaker services use Redis. Consider tuning them to meet your usage requirements.

Spinnaker relies on its Redis cache for a number of reasons: caching infrastructure, storing live executions, returning pipeline definitions faster, etc… Most of these can be configured to your needs, whether it is to make Spinnaker more responsive, or reduce the load on a downstream dependency.

Configure infrastructure caching

By default, Spinnaker’s Clouddriver service spawns “caching agents” every 30 seconds. These agents are scheduled and coordinated across all instances of Clouddriver, allowing you to reduce the load on any single instance of Clouddriver by creating more replicas of the service.

To adjust how this caching happens, Clouddriver exposes a few properties that can be set in ~/.hal/$DEPLOYMENT/profiles/clouddriver-local.yml:

  1. # How many seconds (default 30s) between runs of agent. Lowering this number
  2. # means the resources in the Spinnaker UI will be updated more frequently,
  3. # at the cost higher API/quota usage of your cloud provider.
  4. redis.poll.intervalSeconds:
  7. # How many seconds (default 300s, 5 minutes) Clouddriver will wait to reschedule
  8. # an agent that never completes (never throws an error or returns cache data).
  9. # If your agents are taking a long time to complete their cache cycles
  10. # successfully and Clouddriver is prematurely rescheduling them, you can try to
  11. # raise this number.
  12. redis.poll.timeoutSeconds:

$DEPLOYMENT is typically default. See the documentation for more details.

Configure pipeline cleanup

If you see that your Redis memory usage is growing unbounded, it’s probably because the orchestration engine stores all pipeline executions forever by default. This can be configured by setting the following properties in ~/.hal/$DEPLOYMENT/profiles/orca-local.yml:

  1. pollers:
  2.   oldPipelineCleanup:
  3.     enabled: true                  # This enables old pipeline execution cleanup (default: false)
  4.     intervalMs: 3600000            # How many milliseconds between pipeline cleanup runs (default: 1hr or 3600000)
  5.     thresholdDays: 30              # How old a pipeline execution must be to be deleted (default: 30)
  6.     minimumPipelineExecutions: 5   # How many executions to keep around (default: 5)
  8. tasks:
  9.   daysOfExecutionHistory: 180      # How many days to keep old task executions around.

$DEPLOYMENT is typically default. See the documentation for more details.

Last modified May 4, 2021: rest of migration (700781a)