Discovery

OPA can be configured to download bundles of policy and data, report status, and upload decision logs to remote endpoints. The discovery feature helps you centrally manage the OPA configuration for these features. You should use the discovery feature if you want to avoid managing OPA configuration updates in number of different locations.

When the discovery feature is enabled, OPA will periodically download a discovery bundle. Like regular bundles, the discovery bundle may contain JSON and Rego files. OPA will evaluate the data and policies contained in the discovery bundle to generate the rest of the configuration. There are two main ways to structure the discovery bundle:

  1. Include static JSON configuration files that define the OPA configuration.
  2. Include Rego files that can be evaluated to produce the OPA configuration.

If you need OPA to select which policy to download dynamically (e.g., based on environment variables like the region where OPA is running), use the second option.

If discovery is enabled, other features like bundle downloading and status reporting cannot be configured manually. Similarly, discovered configuration cannot override the original discovery settings in the configuration file that OPA was booted with.

See the Configuration Reference for configuration details.

Discovery Service API

OPA expects the service to expose an API endpoint that serves bundles.

  1. GET /<service_url>/<discovery.resource>HTTP/1.1

If the bundle exists, the server should respond with an HTTP 200 OK status followed by a gzipped tarball in the message body.

  1. HTTP/1.1 200 OK
  2. Content-Type: application/gzip

You can enable discovery with an OPA configuration file similar to the example below. In some places in the documentation, the initial configuration provided to OPA is referred to as the “boot configuration”.

  1. services:
  2.   - name: acmecorp
  3.     url: https://example.com/control-plane-api/v1
  4.     credentials:
  5.       bearer:
  6.         token: "bGFza2RqZmxha3NkamZsa2Fqc2Rsa2ZqYWtsc2RqZmtramRmYWxkc2tm"
  8. discovery:
  9.   name: example
  10.   resource: /configuration/example/discovery.tar.gz
  11.   service: acmecorp
  12.   signing:
  13.     keyid: my_global_key
  14.     scope: read

Using the boot configuration above, OPA will fetch discovery bundles from:

  1. https://example.com/control-plane-api/v1/configuration/example/discovery.tar.gz
  2. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  3. services[discovery.service].url          discovery.resource

The discovery.resource field defaults to bundles/<discovery.name>. The default is convenient if you want to serve discovery bundles and normal bundles from the same API endpoint. If only one service is defined, there is no need to set discovery.service.

The discovery.prefix configuration option is still available but has been deprecated in favor of discovery.resource. It will eventually be removed.

The optional discovery.signing field can be used to specify the keyid and scope that should be used for verifying the signature of the discovery bundle. See this section for details.

OPA generates it’s subsequent configuration by querying the Rego and JSON files contained inside the discovery bundle. The query is defined by the discovery.name field from the boot configuration: data.<discovery.name>. For example. with the boot configuration above, OPA executes the following query:

  1. data.example

As an alternative, you can also provide a decision field, to specify the name of the query. For example, with this configuration:

  1. services:
  2.   - name: acmecorp
  3.     url: https://example.com/control-plane-api/v1
  4.     credentials:
  5.       bearer:
  6.         token: "bGFza2RqZmxha3NkamZsa2Fqc2Rsa2ZqYWtsc2RqZmtramRmYWxkc2tm"
  7. discovery:
  8.   name: example
  9.   resource: /configuration/example/discovery.tar.gz
  10.   decision: example/discovery

OPA executes the following query:

  1. data.example.discovery

If the discovery bundle contained the following Rego file:

  1. package example
  3. discovery = {
  4.   "bundles": {
  5.     "main": {
  6.       "service": "acmecorp",
  7.       "resource": bundle_name
  8.     },
  9.   },
  10.   "default_decision": "acmecorp/httpauthz/allow"
  11. }
  13. bundle_name = "acmecorp/httpauthz"

The subsequent configuration would be:

  1. {
  2.   "bundles": {
  3.     "main": {
  4.       "service": "acmecorp",
  5.       "resource": "acmecorp/httpauthz"
  6.     },
  7.   },
  8.   "default_decision": "acmecorp/httpauthz/allow"
  9. }

The discovery bundle contents above are essentially static. The same result could be achieved by constructing the discovery bundle with a static JSON file:

  1. {
  2.   "example": {
  3.     "discovery": {
  4.       "bundles": {
  5.         "main": {
  6.           "service": "acmecorp",
  7.           "resource": "acmecorp/httpauthz"
  8.         },
  9.       },
  10.       "default_decision": "acmecorp/httpauthz/allow"
  11.     }
  12.   }
  13. }

For an example of how to configure OPA dynamically see the Example section below.

The subsequent configuration does not have to specify services or include a reference to a service in the bundle, status, or decision_log sections. If the either the services or references to services are missing, OPA will default them to the value from the boot configuration.

Example

Let’s see an example of how the discovery feature can be used to dynamically configure an OPA to download one of two bundles based on a label in the boot configuration. Let’s say the label region indicates the region in which the OPA is running and it’s value will decide the bundle to download.

Below is a policy file which generates an OPA configuration.

example.rego

  1. package example
  3. discovery = {
  4.   "bundles": {
  5.     "main": {
  6.       "service": "acmecorp",
  7.       "resource": bundle_name #line 7
  8.     }
  9.   }
  10. }
  12. rt = opa.runtime()
  13. region = rt.config.labels.region
  14. bundle_name = region_bundle[region]
  16. # region-bundle information
  17. region_bundle = {
  18.   "US": "example/test1/p",
  19.   "UK": "example/test2/p"
  20. }

The bundle_name variable in line 7 of the above policy will be dynamically selected based on the value of the label region. So if an OPA was started with region: "US", then the bundle_name will be example/test1/p.

Start an OPA with a boot configuration as shown below:

config.yaml

  1. services:
  2.   - name: acmecorp
  3.     url: https://example.com/control-plane-api/v1
  4.     credentials:
  5.       bearer:
  6.         token: "bGFza2RqZmxha3NkamZsa2Fqc2Rsa2ZqYWtsc2RqZmtramRmYWxkc2tm"
  8. discovery:
  9.   name: /example/discovery
  11. labels:
  12.   region: "US"

Run OPA:

  1. opa run -s -c config.yaml

You should see a log like below, which shows the bundle being downloaded. In this case, the bundle name is example/test1/p as region is US.

  1. INFO Bundle downloaded and activated successfully. name=example/test1/p plugin=bundle

Now start another OPA with a boot configuration as shown below. Notice the region is UK:

config.yaml

  1. services:
  2.   - name: acmecorp
  3.     url: https://example.com/control-plane-api/v1
  4.     credentials:
  5.       bearer:
  6.         token: "bGFza2RqZmxha3NkamZsa2Fqc2Rsa2ZqYWtsc2RqZmtramRmYWxkc2tm"
  8. discovery:
  9.   name: /example/discovery
  11. labels:
  12.   region: "UK"

Run OPA:

  1. opa run -s -c config.yaml

In this case, the bundle being downloaded is example/test2/p as region is UK.

  1. INFO Bundle downloaded and activated successfully. name=example/test2/p plugin=bundle

This shows how the discovery feature can help in centrally managing the bundle to be downloaded by an OPA based on a configuration label. You can use the same strategy to dynamically configure other plugins based on the running OPA’s configuration labels or environment variables.

Limitations

In practice, discovery services do not change frequently. These configuration sections are treated as immutable to avoid accidental configuration errors rendering OPA unable to discover a new configuration. If the discovered configuration changes the discovery or labels sections, those changes are ignored. If the discovered configuration changes the discovery service, an error will be logged.

Discovery Bundle Signature

Like regular bundles, if the discovery bundle contains a .signatures.json file, OPA will verify the discovery bundle before activating it. The format of the .signatures.json file and the verification steps are same as that for regular bundles. Since the discovered configuration ignores changes to the discovery section, any key used for signature verification of a discovery bundle CANNOT be modified via discovery.

🚨 We recommend that if you are using discovery you should be signing the discovery bundles because those bundles include the keys used to verify the non-discovery bundles. However, OPA does not enforce that recommendation. You may use unsigned discovery bundles that themselves require non-discovery bundles to be signed.