Cross cluster search and security

Cross cluster search enables federated search across multiple clusters. When using cross cluster search with secured clusters, all clusters must have the Elasticsearch security features enabled.

The local cluster (the cluster used to initiate cross cluster search) must be allowed to connect to the remote clusters, which means that the CA used to sign the SSL/TLS key of the local cluster must be trusted by the remote clusters.

User authentication is performed on the local cluster and the user and user’s roles are passed to the remote clusters. A remote cluster checks the user’s roles against its local role definitions to determine which indices the user is allowed to access.

This feature was added as Beta in Elasticsearch v5.3 with further improvements made in 5.4 and 5.5. It requires gateway eligible nodes to be on v5.5 onwards.

To use cross cluster search with secured clusters:

  • Enable the Elasticsearch security features on every node in each connected cluster. For more information about the xpack.security.enabled setting, see Security Settings in Elasticsearch.
  • Enable encryption globally. To encrypt communications, you must enable enable SSL/TLS on every node.

  • Enable a trust relationship between the cluster used for performing cross cluster search (the local cluster) and all remote clusters. This can be done either by:

    • Using the same certificate authority to generate certificates for all connected clusters, or
    • Adding the CA certificate from the local cluster as a trusted CA in each remote cluster (see Transport TLS settings).

  • On the local cluster, ensure that users are assigned to (at least) one role that exists on the remote clusters. On the remote clusters, use that role to define which indices the user may access. (See User authorization).

  • Configure the local cluster to connect to remote clusters as described in Configuring Remote Clusters. For example, the following configuration adds two remote clusters to the local cluster:

    1. PUT _cluster/settings
    2. {
    3.   "persistent": {
    4.     "cluster": {
    5.       "remote": {
    6.         "cluster_one": {
    7.           "seeds": [ "10.0.1.1:9300" ]
    8.         },
    9.         "cluster_two": {
    10.           "seeds": [ "10.0.2.1:9300" ]
    11.         }
    12.       }
    13.     }
    14.   }
    15. }

Example Configuration of Cross Cluster Search

In the following example, we will configure the user alice to have permissions to search any data stream or index starting with logs- in cluster two from cluster one.

First, enable cluster one to perform cross cluster search on remote cluster two by running the following request as the superuser on cluster one:

  1. PUT _cluster/settings
  2. {
  3.   "persistent": {
  4.     "cluster.remote.cluster_two.seeds": [ "10.0.2.1:9300" ]
  5.   }
  6. }

Next, set up a role called cluster_two_logs on both cluster one and cluster two.

On cluster one, this role does not need any special privileges:

  1. POST /_security/role/cluster_two_logs
  2. {
  3. }

On cluster two, this role allows the user to query local indices called logs- from a remote cluster:

  1. POST /_security/role/cluster_two_logs
  2. {
  3.   "cluster": [],
  4.   "indices": [
  5.     {
  6.       "names": [
  7.         "logs-*"
  8.       ],
  9.       "privileges": [
  10.         "read",
  11.         "read_cross_cluster"
  12.       ]
  13.     }
  14.   ]
  15. }

Finally, create a user on cluster one and apply the cluster_two_logs role:

  1. POST /_security/user/alice
  2. {
  3.   "password" : "somepassword",
  4.   "roles" : [ "cluster_two_logs" ],
  5.   "full_name" : "Alice",
  6.   "email" : "alice@example.com",
  7.   "enabled": true
  8. }

With all of the above setup, the user alice is able to search indices in cluster two as follows:

  1. GET two:logs-2017.04/_search 
  2. {
  3.   "query": {
  4.     "match_all": {}
  5.   }
  6. }

Cross-cluster search and Kibana

When Kibana is used to search across multiple clusters, a two-step authorization process determines whether or not the user can access data streams and indices on a remote cluster:

  • First, the local cluster determines if the user is authorized to access remote clusters. (The local cluster is the cluster Kibana is connected to.)
  • If they are, the remote cluster then determines if the user has access to the specified data streams and indices.

To grant Kibana users access to remote clusters, assign them a local role with read privileges to indices on the remote clusters. You specify data streams and indices in a remote cluster as <remote_cluster_name>:<target>.

To enable users to actually read the remote data streams and indices, you must create a matching role on the remote clusters that grants the read_cross_cluster privilege and access to the appropriate data streams and indices.

For example, if Kibana is connected to the cluster where you’re actively indexing Logstash data (your local cluster) and you’re periodically offloading older time-based indices to an archive cluster (your remote cluster) and you want to enable Kibana users to search both clusters:

  1. On the local cluster, create a logstash_reader role that grants read and view_index_metadata privileges on the local logstash-* indices.

    If you configure the local cluster as another remote in Elasticsearch, the logstash_reader role on your local cluster also needs to grant the read_cross_cluster privilege.

  2. Assign your Kibana users a role that grants access to Kibana as well as your logstash_reader role.

  3. On the remote cluster, create a logstash_reader role that grants the read_cross_cluster privilege and read and view_index_metadata privileges for the logstash-* indices.