Mutual TLS authentication between Kibana and Elasticsearch

Secure Sockets Layer (SSL) and Transport Layer Security (TLS) provide encryption for data-in-transit. While these terms are often used interchangeably, Kibana supports only TLS, which supersedes the old SSL protocols.

TLS requires X.509 certificates to authenticate the communicating parties and perform encryption of data-in-transit. Each certificate contains a public key and has and an associated — but separate — private key; these keys are used for cryptographic operations. Kibana supports certificates and private keys in PEM or PKCS#12 format.

In a standard TLS configuration, the server presents a signed certificate to authenticate itself to the client. In a mutual TLS configuration, the client also presents a signed certificate to authenticate itself to the server.

When Elasticsearch security features is enabled on your cluster, each request that Kibana (the client) makes to Elasticsearch (the server) must be authenticated. Most requests made by end users through Kibana to Elasticsearch are authenticated by using the credentials of the logged-in user. There are, however, a few internal requests that Kibana needs to make to Elasticsearch. For this reason, you must configure credentials for Kibana to use for those requests.

If Kibana has elasticsearch.username and elasticsearch.password configured, it will attempt to use these to authenticate to Elasticsearch via the native realm. However, Kibana also supports mutual TLS authentication with Elasticsearch via a Public Key Infrastructure (PKI) realm. To do so, Elasticsearch needs to verify the signature on the Kibana client certificate, and it also needs to map the client certificate’s distinguished name (DN) to the appropriate kibana_system role.

Using a PKI realm is a gold feature. For a comparison of the Elastic license levels, see the subscription page.

To configure Kibana and Elasticsearch to use mutual TLS authentication:

  1. Set up Kibana to work with Elastic Stack security features with a username and password.

  2. Set up TLS encryption between Kibana and Elasticsearch.

    This entails generating a “server certificate” for Elasticsearch to use on the HTTP layer.

  3. Obtain a client certificate and private key for Kibana.

    Kibana must this “client certificate” and corresponding private key when connecting to Elasticsearch.

    This is not the same as the server certificate that Kibana will present to web browsers.

    You may choose to generate a client certificate and private key using the elasticsearch-certutil tool. If you followed the Elasticsearch documentation for generating node certificates, then you likely have already set up a certificate authority (CA) to sign the Elasticsearch server certificate. You may choose to use the same CA to sign the Kibana client certificate. For example:

    1. bin/elasticsearch-certutil cert -ca elastic-stack-ca.p12 -name kibana-client -dns <your_kibana_hostname>

    This will generate a client certificate and private key in a PKCS#12 file named kibana-client.p12. In this example, the client certificate has a Common Name (CN) of "kibana-client" and a subject alternative name (SAN) of "<your_kibana_hostname>". The SAN may be required if you have hostname verification enabled on Elasticsearch.

  4. Obtain the certificate authority (CA) certificate chain for Kibana.

    Elasticsearch needs the appropriate CA certificate chain to properly establish trust when receiving connections from Kibana.

    If you followed the instructions to generate a client certificate, then you will have a PKCS#12 file for Kibana. You can extract the CA certificate chain from this file. For example:

    1. openssl pkcs12 -in kibana-client.p12 -cacerts -nokeys -out kibana-ca.crt

    This will produce a PEM-formatted file named kibana-ca.crt that contains the CA certificate from the PKCS#12 file.

  5. Configure Elasticsearch with a PKI realm and a native realm.

    By default, Elasticsearch provides a native realm for authenticating with a username and password. However, to support both a PKI realm (for Kibana) and a native realm (for end users), you must configure each realm in elasticsearch.yml:

    1. xpack.security.authc.realms.pki.realm1.order: 1
    2. xpack.security.authc.realms.pki.realm1.certificate_authorities: "/path/to/kibana-ca.crt"
    3. xpack.security.authc.realms.native.realm2.order: 2

  6. Configure Elasticsearch to request client certificates.

    By default, Elasticsearch will not request a client certificate when establishing a TLS connection. To change this, you must set up optional client certificate authentication in elasticsearch.yml:

    1. xpack.security.http.ssl.client_authentication: "optional"

  7. Restart Elasticsearch.

  8. Use Kibana to create a role mapping in Elasticsearch for the client certificate.

    This role mapping will assign the kibana_system role to any user that matches the included mapping rule, which is set to equal the client certificate’s DN attribute:

    Role mapping for the Kibana client certificate

    For more information, see role mappings.

  9. Configure Kibana to use the client certificate and private key.

    You need to specify the information required to access your client certificate and corresponding private key.

    1. If your certificate and private key are contained in a PKCS#12 file:

      Specify your PKCS#12 file in kibana.yml:

      1. elasticsearch.ssl.keystore.path: "/path/to/kibana-client.p12"

      If your PKCS#12 file is encrypted, add the decryption password to your Kibana keystore:

      1. bin/kibana-keystore add elasticsearch.ssl.keystore.password

      If your PKCS#12 file isn’t protected with a password, depending on how it was generated, you may need to set elasticsearch.ssl.keystore.password to an empty string.

    2. Otherwise, if your certificate and private key are in PEM format:

      Specify your certificate and private key in kibana.yml:

      1. elasticsearch.ssl.certificate: "/path/to/kibana-client.crt"
      2. elasticsearch.ssl.key: "/path/to/kibana-client.key"

      If your private key is encrypted, add the decryption password to your Kibana keystore:

      1. bin/kibana-keystore add elasticsearch.ssl.keyPassphrase

  1. Configure Kibana not to use a username and password for Elasticsearch.

    You must remove the elasticsearch.username and elasticsearch.password settings from kibana.yml. If these are present, Kibana will attempt to use them to authenticate to Elasticsearch via the native realm.

  2. Restart Kibana.

These steps enable Kibana to authenticate to Elasticsearch using a certificate. However, end users will only be able to authenticate to Kibana with a username and password. To allow end users to authenticate to Kibana using a client certificate, see Kibana PKI authentication.

Most Popular