AWS Metadata

The AWS Filter Enriches logs with AWS Metadata. Currently the plugin adds the EC2 instance ID and availability zone to log records. To use this plugin, you must be running in EC2 and have the instance metadata service enabled.

Configuration Parameters

The plugin supports the following configuration parameters:

Key Description Default
imds_version Specify which version of the instance metadata service to use. Valid values are ‘v1’ or ‘v2’. v2
az The availability zone; for example, “us-east-1a”. true
ec2_instance_id The EC2 instance ID. true
ec2_instance_type The EC2 instance type. false
private_ip The EC2 instance private ip. false
ami_id The EC2 instance image id. false
account_id The account ID for current EC2 instance. false
hostname The hostname for current EC2 instance. false
vpc_id The VPC ID for current EC2 instance. false

Note: If you run Fluent Bit in a container, you may have to use instance metadata v1. The plugin behaves the same regardless of which version is used.

Command Line

  1. $ bin/fluent-bit -c /PATH_TO_CONF_FILE/fluent-bit.conf
  3. [2020/01/17 07:57:17] [ info] [engine] started (pid=32744)
  4. [0] dummy: [1579247838.000171227, {"message"=>"dummy", "az"=>"us-west-2c", "ec2_instance_id"=>"i-0c862eca9038f5aae", "ec2_instance_type"=>"t2.medium", "private_ip"=>"172.31.6.59", "vpc_id"=>"vpc-7ea11c06", "ami_id"=>"ami-0841edc20334f9287", "account_id"=>"YOUR_ACCOUNT_ID", "hostname"=>"ip-172-31-6-59.us-west-2.compute.internal"}]
  5. [0] dummy: [1601274509.970235760, {"message"=>"dummy", "az"=>"us-west-2c", "ec2_instance_id"=>"i-0c862eca9038f5aae", "ec2_instance_type"=>"t2.medium", "private_ip"=>"172.31.6.59", "vpc_id"=>"vpc-7ea11c06", "ami_id"=>"ami-0841edc20334f9287", "account_id"=>"YOUR_ACCOUNT_ID", "hostname"=>"ip-172-31-6-59.us-west-2.compute.internal"}]

Configuration File

  1. [INPUT]
  2.     Name dummy
  3.     Tag dummy
  5. [FILTER]
  6.     Name aws
  7.     Match *
  8.     imds_version v1
  9.     az true
  10.     ec2_instance_id true
  11.     ec2_instance_type true
  12.     private_ip true
  13.     ami_id true
  14.     account_id true
  15.     hostname true
  16.     vpc_id true
  18. [OUTPUT]
  19.     Name stdout
  20.     Match *