Loki Storage Retention
Retention in Loki is achieved through the Table Manager.In order to enable the retention support, the Table Manager needs to beconfigured to enable deletions and a retention period. Please refer to thetable_manager_config
section of the Loki configuration reference for all available options.Alternatively, the table-manager.retention-period
andtable-manager.retention-deletes-enabled
command line flags can be used. Theprovided retention period needs to be a duration represented as a string thatcan be parsed using Go’s time.Duration.
WARNING: The retention period must be a multiple of the index and chunks table
period
, configured in theperiod_config
block. See the Table Manager documentation formore information.
When using S3 or GCS, the bucket storing the chunks needs to have the expirypolicy set correctly. For more details checkS3’s documentationorGCS’s documentation.
Currently, the retention policy can only be set globally. A per-tenant retentionpolicy with an API to delete ingested logs is still under development.
Since a design goal of Loki is to make storing logs cheap, a volume-baseddeletion API is deprioritized. Until this feature is released, if you suddenlymust delete ingested logs, you can delete old chunks in your object store. Note,however, that this only deletes the log content and keeps the label indexintact; you will still be able to see related labels but will be unable toretrieve the deleted log content.
For further details on the Table Manager internals, refer to theTable Manager documentation.
Example Configuration
Example configuration with GCS with a 30 day retention:
schema_config:
configs:
- from: 2018-04-15
store: bigtable
object_store: gcs
schema: v9
index:
prefix: loki_index_
period: 168h
storage_config:
bigtable:
instance: BIGTABLE_INSTANCE
project: BIGTABLE_PROJECT
gcs:
bucket_name: GCS_BUCKET_NAME
table_manager:
retention_deletes_enabled: true
retention_period: 720h