This page provides some performance benchmarks that give an idea of the overhead of using the OPA-Envoy plugin.

Test Setup

The setup uses the same example Go application that’s described in the standalone Envoy tutorial. Below are some more details about the setup:

• Platform: Minikube
• Kubernetes Version: 1.18.6
• Envoy Version: 1.17.0
• OPA-Envoy Version: 0.26.0-envoy

Benchmarks

The benchmark result below provides the percentile distribution of the latency observed by sending 100 requests/sec to the sample application. Each request makes a GET call to the /people endpoint exposed by the application.

The graph shows the latency distribution when the load test is performed under the following conditions:

• App Only

In this case, the graph documents the latency distribution observed when requests are sent directly to the application ie. no Envoy and OPA in the request path. This scenario is depicted by the blue curve.

• App and Envoy

In this case, the distribution is with Envoy External Authorization API disabled. This means OPA is not included in the request path but Envoy is. This scenario is depicted by the red curve.

• App, Envoy and OPA (NOP policy)

In the case, we will see the latency observed with Envoy External Authorization API enabled. This means Envoy will make a call to OPA on every incoming request. The graph explores the effect of loading the below NOP policy into OPA. This scenario is depicted by the green curve.

package envoy.authz
default allow = true

• App, Envoy and OPA (RBAC policy)

In the case, we will see the latency observed with Envoy External Authorization API enabled and explore the effect of loading the following RBAC policy into OPA. This scenario is depicted by the yellow curve.

package envoy.authz
import input.attributes.request.http as http_request
default allow = false
allow {
    roles_for_user[r]
    required_roles[r]
}
roles_for_user[r] {
    r := user_roles[user_name][_]
}
required_roles[r] {
    perm := role_perms[r][_]
    perm.method = http_request.method
    perm.path = http_request.path
}
user_name = parsed {
    [_, encoded] := split(http_request.headers.authorization, " ")
    [parsed, _] := split(base64url.decode(encoded), ":")
}
user_roles = {
    "alice": ["guest"],
    "bob": ["admin"]
}
role_perms = {
    "guest": [
        {"method": "GET",  "path": "/people"},
    ],
    "admin": [
        {"method": "GET",  "path": "/people"},
        {"method": "POST",  "path": "/people"},
    ],
}

The above four scenarios are replicated to measure the latency distribution now by sending 1000 requests/sec to the sample application. The following graph captures this result.

OPA Benchmarks

The table below captures the gRPC Server Handler and OPA Evaluation time with Envoy External Authorization API enabled and the RBAC policy described above loaded into OPA. All values are in microseconds.

OPA Evaluation

OPA Evaluation is the time taken to evaluate the policy.

gRPC Server Handler

gRPC Server Handler is the total time taken to prepare the input for the policy, evaluate the policy (OPA Evaluation) and prepare the result.

Resource Utilization

The following table records the CPU and memory usage for the OPA-Envoy container. These metrics were obtained using the kubectl top command. No resource limits were specified for the OPA-Envoy container.

In the analysis so far, the gRPC client used in Envoy’s External authorization filter configuration is the Google C++ gRPC client. The following graph displays the latency distribution for the same four conditions described previously (ie. App Only, App and Envoy, App, Envoy and OPA (NOP policy) and App, Envoy and OPA (RBAC policy)) by sending 100 requests/sec to the sample application but now using Envoy’s in-built gRPC client.

The below graph captures the latency distribution when 1000 requests/sec are sent to the sample application and Envoy’s in-built gRPC client is used.

The above graphs show that there is extra latency added when the OPA-Envoy plugin is used as an external authorization service. For example, in the previous graph, the latency for the App, Envoy and OPA (NOP policy) condition between the 90th and 99th percentile is at least double than that for App and Envoy.

The following graphs show the latency distribution for the App, Envoy and OPA (NOP policy) and App, Envoy and OPA (RBAC policy) condition and plot the latencies seen by using the Google C++ gRPC client and Envoy’s in-built gRPC client in the External authorization filter configuration. The first graph is when 100 requests/sec are sent to the application while the second one for 1000 requests/sec.