10. Virtual machine monitoring

Overview

Support of monitoring VMware environments is available in Zabbix starting with version 2.2.0.

Zabbix can use low-level discovery rules to automatically discover VMware hypervisors and virtual machines and create hosts to monitor them, based on pre-defined host prototypes.

The default dataset in Zabbix offers several ready-to-use templates for monitoring VMware vCenter or ESX hypervisor.

The minimum required VMware vCenter or vSphere version is 5.1.

Details

The virtual machine monitoring is done in two steps. First, virtual machine data is gathered by vmware collector Zabbix processes. Those processes obtain necessary information from VMware web services over the SOAP protocol, pre-process it and store into Zabbix server shared memory. Then, this data is retrieved by pollers using Zabbix simple check VMware keys.

Starting with Zabbix version 2.4.4 the collected data is divided into 2 types: VMware configuration data and VMware performance counter data. Both types are collected independently by vmware collectors. Because of this it is recommended to enable more collectors than the monitored VMware services. Otherwise retrieval of VMware performance counter statistics might be delayed by the retrieval of VMware configuration data (which takes a while for large installations).

Currently only datastore, network interface and disk device statistics and custom performance counter items are based on the VMware performance counter information.

Configuration

For virtual machine monitoring to work, Zabbix should be compiled with the —with-libxml2 and —with-libcurl compilation options.

The following configuration file options can be used to tune the Virtual machine monitoring:

  • StartVMwareCollectors - the number of pre-forked vmware collector instances.
    This value depends on the number of VMware services you are going to monitor. For the most cases this should be:
    servicenum < StartVMwareCollectors < (servicenum \ 2)
    where     servicenum is the number of VMware services. E. g. if you have 1 VMware service to monitor set StartVMwareCollectors to 2, if you have 3 VMware services, set it to 5. Note that in most cases this value should not be less than 2 and should not be 2 times greater than the number of VMware services that you monitor. Also keep in mind that this value also depends on your VMware environment size and VMwareFrequency and VMwarePerfFrequency* configuration parameters (see below).
  • VMwareCacheSize
  • VMwareFrequency
  • VMwarePerfFrequency
  • VMwareTimeout

For more details, see the configuration file pages for Zabbix server and proxy.

To support datastore capacity metrics Zabbix requires VMware configuration vpxd.stats.maxQueryMetrics parameter to be at least 64. See also the VMware knowledge base article.

Discovery

Zabbix can use a low-level discovery rule to automatically discover VMware hypervisors and virtual machines.

10. Virtual machine monitoring - 图1

All mandatory input fields are marked with a red asterisk.

The discovery rule key in the above screenshot is vmware.hv.discovery[{$URL}].

Host prototypes

Host prototypes can be created with the low-level discovery rule. When virtual machines are discovered, these prototypes become real hosts. Prototypes, before becoming discovered, cannot have their own items and triggers, other than those from the linked templates. Discovered hosts will belong to an existing host.

10. Virtual machine monitoring - 图2

In order for hosts created from a prototype to have unique host names, the Host name field must contain at least one low-level discovery macro.

Since Zabbix 5.2, discovered hosts may be configured with custom interfaces or inherit the IP of a host the discovery rule belongs to (default). To add one or more custom interface, switch the Interface selector from Inherit to Custom mode, then press add_link.png and select the required interface type from the drop-down menu that appears. All supported types of interfaces can be defined for a host prototype: Zabbix agent, SNMP, JMX, IPMI. Interface fields support low-level discovery macros and user macros. If several custom interfaces are specified - use the Default column to specify the primary interface.

Notes:

  • If Custom is selected, but no interfaces have been specified the hosts will be created without interfaces.
  • If Inherit is selected for a host prototype that belongs to a template, discovered hosts will inherit the interface of a host to which the template is linked to.

A host will not be created, if the host interface contains incorrect data

10. Virtual machine monitoring - 图4

LLD macros can also be used for the visible name, host group prototype fields, tag values, or values of host prototype user macros.

Other options that can be specified for a host prototype are:

  • Linkage to existing host groups
  • Template linkage
  • Encryption

If Create enabled is checked, the host will be added in an enabled state. If unchecked, the host will be added, but in a disabled state.

If Discover is checked (default), the host will be created. If unchecked, the host will not be created, unless this setting is overridden in the discovery rule. This functionality provides additional flexibility when creating discovery rules.

Discovered hosts are prefixed with the name of the discovery rule that created them, in the host list. Discovered hosts can be manually deleted. Discovered hosts will also be automatically deleted, based on the Keep lost resources period (in days) value of the discovery rule. Most of the configuration options are read-only, except for enabling/disabling the host and host inventory. Discovered hosts cannot have host prototypes of their own.

Ready-to-use templates

The default dataset in Zabbix offers several ready-to-use templates for monitoring VMware vCenter or directly ESX hypervisor. These templates contain pre-configured LLD rules as well as a number of built-in checks for monitoring virtual installations.

Templates for VMware vCenter and ESX hypervisor monitoring:

  • VMware - uses UUID data for corresponding macros;
  • VMware FQDN - uses FQDN data for corresponding macros.

In order for the VMware FQDN template to work correctly each monitored VM should have a unique OS name compliant with FQDN rules and VMware Tools must be installed on every machine. If these conditions are met, it is recommended to use VMware FQDN template. The creation of VMware FQDN template became possible after introducing the ability to create hosts with custom interfaces in Zabbix 5.2.

A classic VMware template is still available and can be used if FQDN requirements cannot be met. Please keep in mind, that the VMware template has a known issue. Hosts for discovered virtual machines will be created with the names saved in the vCenter (for example, VM1, VM2, etc.). If Zabbix agent active is installed on these hosts later with autoregistration enabled, the autoregistration process will read host names as they have been registered upon launch (for example, vm1.example.com, vm2.example.com, etc.) and create new hosts since no name matches have been found. As a result there will be two duplicate hosts for each machine with different names.

Templates used by discovery (normally, these templates should not be manually linked to a host):

  • VMware Hypervisor;
  • VMware Guest.

10. Virtual machine monitoring - 图5

Host configuration

To use VMware simple checks the host must have the following user macros defined:

  • {$VMWARE.URL} - VMware service (vCenter or ESX hypervisor) SDK URL (https://servername/sdk)
  • {$VMWARE.USERNAME} - VMware service user name
  • {$VMWARE.PASSWORD} - VMware service {$VMWARE.USERNAME} user password

Example

The following example demonstrates how to quickly setup VMware monitoring on Zabbix:

  • compile zabbix server with required options (—with-libxml2 and —with-libcurl)
  • set the StartVMwareCollectors option in Zabbix server configuration file to 1 or more
  • create a new host
  • set the host macros required for VMware authentication:
  1.     {{..:..:assets:en:manual:vm_monitoring:vm_host_macros.png|}}
  2.        * Link the host to the VMware service template: 
  3.            {{..:..:assets:en:manual:vm_monitoring:vm_host_templates.png|}}
  4.        * Click on the //Add// button to save the host

Extended logging

The data gathered by VMware collector can be logged for detailed debugging using debug level 5. This level can be set in server and proxy configuration files or using a runtime control option (-R log_level_increase="vmware collector,N", where N is a process number). The following examples demonstrate how extended logging can be started provided debug level 4 is already set:

  1. Increase log level of all vmware collectors:
  2.        shell> zabbix_server -R log_level_increase="vmware collector"
  4.        Increase log level of second vmware collector:
  5.        shell> zabbix_server -R log_level_increase="vmware collector,2"

If extended logging of VMware collector data is not required it can be stopped using the -R log_level_decrease option.

Troubleshooting

  • In case of unavailable metrics, please make sure if they are not made unavailable or turned off by default in recent VMware vSphere versions or if some limits are not placed on performance-metric database queries. See ZBX-12094 for additional details.

  • In case of ‘config.vpxd.stats.maxQueryMetrics’ is invalid or exceeds the maximum number of characters permitted\* error, add a config.vpxd.stats.maxQueryMetrics parameter to the vCenter Server settings. The value of this parameter should be the same as the value of maxQuerysize in VMware’s web.xml*. See this VMware knowledge base article for details.