Data sources

OpenSearch data sources are the applications that OpenSearch can connect to and ingest data from. Once your data sources have been connected and your data has been ingested, it can be indexed, searched, and analyzed using REST APIs or the OpenSearch Dashboards UI.

This documentation focuses on using the OpenSeach Dashboards interface to connect and manage your data sources. For information about using an API to connect data sources, see the developer resources linked under Next steps.

Prerequisites

The first step in connecting your data sources to OpenSearch is to install OpenSearch and OpenSearch Dashboards on your system. Refer to the installation instructions for information.

Once you have installed OpenSearch and OpenSearch Dashboards, you can use Dashboards to connect your data sources to OpenSearch and then use Dashboards to manage data sources, create index patterns based on those data sources, run queries against a specific data source, and combine visualizations in one dashboard.

Configuration of the YAML files and installation of the dashboards-observability and opensearch-sql plugins is necessary. For more information, see OpenSearch plugins.

To securely store and encrypt data source connections in OpenSearch, you must add the following configuration to the opensearch.yml file on all the nodes:

plugins.query.datasources.encryption.masterkey: "YOUR_GENERATED_MASTER_KEY_HERE"

The key must be 16, 24, or 32 characters. You can use the following command to generate a 24-character key:

openssl rand -hex 12

Generating 12 bytes results in a hexadecimal string that is 12 * 2 = 24 characters.

Permissions

To work with data sources in OpenSearch Dashboards, you must be assigned the correct cluster-level data source permissions.

Types of data streams

To configure data sources through OpenSearch Dashboards, go to Management > Dashboards Management > Data sources. This flow can be used for OpenSearch data stream connections. See Configuring and using multiple data sources.

Alternatively, if you are running OpenSearch Dashboards 2.16 or later, go to Management > Data sources. This flow can be used to connect Amazon Simple Storage Service (Amazon S3) and Prometheus. See Connecting Amazon S3 to OpenSearch and Connecting Prometheus to OpenSearch for more information.

Next steps