Configuring indices created by Metricbeat 7 or internal collection

Configuring indices created by Metricbeat 7 or internal collection

When monitoring using Metricbeat 7 or internal collection, data is stored in a set of indices called either:

  • .monitoring-{product}-7-mb-{date}, when using Metricbeat 7.
  • .monitoring-{product}-7-{date}, when using internal collection.

The settings and mappings for these indices are determined by legacy index templates named .monitoring-{product}. You can retrieve these templates in Kibana by navigating to Stack Management > Index Management > Index Templates, or by using the Elasticsearch _template API:

  1. resp = client.indices.get_template(
  2.     name=".monitoring-*",
  3. )
  4. print(resp)
  1. response = client.indices.get_template(
  2.   name: '.monitoring-*'
  3. )
  4. puts response
  1. const response = await client.indices.getTemplate({
  2.   name: ".monitoring-*",
  3. });
  4. console.log(response);
  1. GET /_template/.monitoring-*

To change the settings of the indices, add a custom index template. You can do that in Kibana, or using the Elasticsearch API:

  • Set index_patterns to match the .monitoring-{product}-7-* indices.
  • Set the template order to 1. This ensures your template is applied after the default template, which has an order of 0.
  • Specify the number_of_shards and/or number_of_replicas in the settings section.
  1. resp = client.indices.put_template(
  2.     name="custom_monitoring",
  3.     index_patterns=[
  4.         ".monitoring-beats-7-*",
  5.         ".monitoring-es-7-*",
  6.         ".monitoring-kibana-7-*",
  7.         ".monitoring-logstash-7-*"
  8.     ],
  9.     order=1,
  10.     settings={
  11.         "number_of_shards": 5,
  12.         "number_of_replicas": 2
  13.     },
  14. )
  15. print(resp)
  1. response = client.indices.put_template(
  2.   name: 'custom_monitoring',
  3.   body: {
  4.     index_patterns: [
  5.       '.monitoring-beats-7-*',
  6.       '.monitoring-es-7-*',
  7.       '.monitoring-kibana-7-*',
  8.       '.monitoring-logstash-7-*'
  9.     ],
  10.     order: 1,
  11.     settings: {
  12.       number_of_shards: 5,
  13.       number_of_replicas: 2
  14.     }
  15.   }
  16. )
  17. puts response
  1. const response = await client.indices.putTemplate({
  2.   name: "custom_monitoring",
  3.   index_patterns: [
  4.     ".monitoring-beats-7-*",
  5.     ".monitoring-es-7-*",
  6.     ".monitoring-kibana-7-*",
  7.     ".monitoring-logstash-7-*",
  8.   ],
  9.   order: 1,
  10.   settings: {
  11.     number_of_shards: 5,
  12.     number_of_replicas: 2,
  13.   },
  14. });
  15. console.log(response);
  1. PUT /_template/custom_monitoring
  2. {
  3.   "index_patterns": [".monitoring-beats-7-*", ".monitoring-es-7-*", ".monitoring-kibana-7-*", ".monitoring-logstash-7-*"],
  4.   "order": 1,
  5.   "settings": {
  6.     "number_of_shards": 5,
  7.     "number_of_replicas": 2
  8.   }
  9. }

After changing the index template, the updated settings are only applied to new indices.