Defaults

defaults is used to set default values ​​in the Pipelines configuration. Takes effect when no value is set in the Pipeline, or is used to override the default parameter.

Example

  1. defaults:
  2.   sources:
  3.     - type: file
  4.       watcher:
  5.         cleanFiles:
  6.           maxHistory: 10
  7.   sink:
  8.     type: dev
  9.     printEvents: true

sources

Consistent with source in Pipeline. When Pipelines is configured with a source of the same type, value of the field that is not filled in will be overwritten.

exapmle:

  1.     sources:
  2.       - type: file
  3.         watcher:
  4.           cleanFiles:
  5.             maxHistory: 10

If the Pipeline is configured with a file source, you can set the global file cleaning retention period to 10 days, instead of setting it in each Pipeline’s file source.

sink

Consistent with the sink in Pipeline, if the cluster only needs to set a global sink output source, it only needs to be configured here once to avoid filling in each Pipeline.

interceptors

The interceptors configured in defaults will be added to the interceptors defined in the pipeline, but the interceptors in the pipeline will override the interceptors of the same type in defaults. If you do not want to cover the same type of interceptor, but add the same type of interceptor, and process them in turn, you need to fill in the name field for identification.

The normalize interceptor has been defined in defaults as follows:

  1. defaults:
  2.   interceptors:
  3.   - type: normalize
  4.     processor:
  5.     - addMeta: ~

If the normalize interceptor is defined in the pipelines as follows:

  1. pipelines:
  2.   interceptors:
  3.   - type: normalize
  4.     processor:
  5.     - drop:
  6.       ...

At this time, the normalize interceptor in defaults will be overwritten and will not take effect.

If we want to execute the normalize interceptor in defaults first, and then execute the normalize interceptor in the pipeline, we can change the defaults to:

  1. defaults:
  2.   interceptors:
  3.   - type: normalize
  4.     name: global # for identification
  5.     order: 500   # execution order. the larger, the latter to be executed. default 900.
  6.     processor:
  7.     - addMeta: ~

Loggie will add 3 built-in interceptors metric、maxbytes、retry by default.
If you need to add other default interceptors, the above built-in interceptors will be overwritten, so it is strongly recommended to add built-in interceptors at this time, unless you confirm that you do not need the built-in interceptors.

queue

Channel queue by default.