This guide demonstrates a client within the service mesh accessing destinations external to the mesh using OSM’s Egress policy API.

Prerequisites

  • Kubernetes cluster running Kubernetes v1.19.0 or greater.
  • Have OSM installed.
  • Have kubectl available to interact with the API server.
  • Have osm CLI available for managing the service mesh.

Demo

  1. Enable egress policy if not enabled:

    1. export osm_namespace=osm-system # Replace osm-system with the namespace where OSM is installed
    2. kubectl patch meshconfig osm-mesh-config -n "$osm_namespace" -p '{"spec":{"featureFlags":{"enableEgressPolicy":true}}}'  --type=merge

  2. Deploy the curl client into the curl namespace after enrolling its namespace to the mesh.

    1. # Create the curl namespace
    2. kubectl create namespace curl
    4. # Add the namespace to the mesh
    5. osm namespace add curl
    7. # Deploy curl client in the curl namespace
    8. kubectl apply -f https://raw.githubusercontent.com/openservicemesh/osm-docs/release-v1.0/manifests/samples/curl/curl.yaml -n curl

    Confirm the curl client pod is up and running.

    1. $ kubectl get pods -n curl
    2. NAME                    READY   STATUS    RESTARTS   AGE
    3. curl-54ccc6954c-9rlvp   2/2     Running   0          20s

HTTP Egress

  1. Confirm the curl client is unable make the HTTP request http://httpbin.org:80/get to the httpbin.org website on port 80.

    1. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/get
    2. command terminated with exit code 7

  2. Apply an Egress policy to allow the curl client’s ServiceAccount to access the httpbin.org website on port 80 serving the http protocol.

    1. kubectl apply -f - <<EOF
    2. kind: Egress
    3. apiVersion: policy.openservicemesh.io/v1alpha1
    4. metadata:
    5.   name: httpbin-80
    6.   namespace: curl
    7. spec:
    8.   sources:
    9.   - kind: ServiceAccount
    10.     name: curl
    11.     namespace: curl
    12.   hosts:
    13.   - httpbin.org
    14.   ports:
    15.   - number: 80
    16.     protocol: http
    17. EOF

  3. Confirm the curl client is able to make successful HTTP requests to http://httpbin.org:80/get.

    1. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/get
    2. HTTP/1.1 200 OK
    3. date: Thu, 13 May 2021 21:49:35 GMT
    4. content-type: application/json
    5. content-length: 335
    6. server: envoy
    7. access-control-allow-origin: *
    8. access-control-allow-credentials: true
    9. x-envoy-upstream-service-time: 168

  4. Confirm the curl client can no longer make successful HTTP requests to http://httpbin.org:80/get when the above policy is removed.

    1. kubectl delete egress httpbin-80 -n curl
    1. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/get
    2. command terminated with exit code 7

HTTPS Egress

Since HTTPS traffic is encrypted with TLS, OSM routes HTTPS based traffic by proxying it to its original destination as a TCP stream. The Server Name Indication (SNI) indicated by the HTTPS client application in the TLS handshake is matched against hosts specified in the Egress policy.

  1. Confirm the curl client is unable make the HTTPS request https://httpbin.org:443/get to the httpbin.org website on port 443.

    1. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI https://httpbin.org:443/get
    2. command terminated with exit code 7

  2. Apply an Egress policy to allow the curl client’s ServiceAccount to access the httpbin.org website on port 443 serving the https protocol.

    1. kubectl apply -f - <<EOF
    2. kind: Egress
    3. apiVersion: policy.openservicemesh.io/v1alpha1
    4. metadata:
    5.   name: httpbin-443
    6.   namespace: curl
    7. spec:
    8.   sources:
    9.   - kind: ServiceAccount
    10.     name: curl
    11.     namespace: curl
    12.   hosts:
    13.   - httpbin.org
    14.   ports:
    15.   - number: 443
    16.     protocol: https
    17. EOF

  3. Confirm the curl client is able to make successful HTTPS requests to https://httpbin.org:443/get.

    1. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI https://httpbin.org:443/get
    2. HTTP/2 200
    3. date: Thu, 13 May 2021 22:09:36 GMT
    4. content-type: application/json
    5. content-length: 260
    6. server: gunicorn/19.9.0
    7. access-control-allow-origin: *
    8. access-control-allow-credentials: true

  4. Confirm the curl client can no longer make successful HTTPS requests to https://httpbin.org:443/get when the above policy is removed.

    1. kubectl delete egress httpbin-443 -n curl
    1. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI https://httpbin.org:443/get
    2. command terminated with exit code 7

TCP Egress

TCP based Egress traffic is matched against the destination port and IP address ranges specified in an egress policy. If an IP address range is not specified, traffic will be matched only based on the destination port.

  1. Confirm the curl client is unable make the HTTPS request https://openservicemesh.io:443 to the openservicemesh.io website on port 443. Since HTTPS uses TCP as the underlying transport protocol, TCP based routing should implicitly enable access to any HTTP(s) host on the specified port.

    1. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI https://openservicemesh.io:443
    2. command terminated with exit code 7

  2. Apply an Egress policy to allow the curl client’s ServiceAccount to access the any destination on port 443 serving the tcp protocol.

    1. kubectl apply -f - <<EOF
    2. kind: Egress
    3. apiVersion: policy.openservicemesh.io/v1alpha1
    4. metadata:
    5.   name: tcp-443
    6.   namespace: curl
    7. spec:
    8.   sources:
    9.   - kind: ServiceAccount
    10.     name: curl
    11.     namespace: curl
    12.   ports:
    13.   - number: 443
    14.     protocol: tcp
    15. EOF

    Note: For server-first protocols such as MySQL, PostgreSQL, etc., where the server initiates the first bytes of data between the client and server, the protocol must be set to tcp-server-first to indicate to OSM to not perform protocol detection on the port. Protocol detection relies on inspecting the initial bytes of a connection, which is incompatible with server-first protocols. When the port’s protocol is set to tcp-server-first, protocol detection is skipped for that port number. It is also important to note that server-first port numbers must not be used for other application ports that require protocol detection to performed, which means the port numbers used for server-first protocols must not be used with other protocols such as HTTP and TCP that require protocol detection to be performed.

  3. Confirm the curl client is able to make successful HTTPS requests to https://openservicemesh.io:443.

    1. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI https://openservicemesh.io:443
    2. HTTP/2 200
    3. cache-control: public, max-age=0, must-revalidate
    4. content-length: 0
    5. content-type: text/html; charset=UTF-8
    6. date: Thu, 13 May 2021 22:35:07 GMT
    7. etag: "353ebaaf9573718bd1df6b817a472e47-ssl"
    8. strict-transport-security: max-age=31536000
    9. age: 0
    10. server: Netlify
    11. x-nf-request-id: 35a4f2dc-5356-45dc-9208-63e6fa162e0f-3350874

  4. Confirm the curl client can no longer make successful HTTPS requests to https://openservicemesh.io:443 when the above policy is removed.

    1. kubectl delete egress tcp-443 -n curl
    1. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI https://openservicemesh.io:443
    2. command terminated with exit code 7

HTTP Egress with SMI route matches

HTTP Egress policies can specify SMI HTTPRouteGroup matches for fine grained traffic control based on HTTP methods, headers and paths.

  1. Confirm the curl client is unable make HTTP requests to http://httpbin.org:80/get and http://httpbin.org:80/status/200 to the httpbin.org website on port 80.

    1. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/get
    2. command terminated with exit code 7
    3. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/status/200
    4. command terminated with exit code 7

  2. Apply an SMI HTTPRouteGroup resource to allow access to the HTTP path /get and an Egress policy to access the httpbin.org on website port 80 that matches on the SMI HTTPRouteGroup.

    1. kubectl apply -f - <<EOF
    2. apiVersion: specs.smi-spec.io/v1alpha4
    3. kind: HTTPRouteGroup
    4. metadata:
    5.   name: egress-http-route
    6.   namespace: curl
    7. spec:
    8.   matches:
    9.   - name: get
    10.     pathRegex: /get
    11. ---
    12. kind: Egress
    13. apiVersion: policy.openservicemesh.io/v1alpha1
    14. metadata:
    15.   name: httpbin-80
    16.   namespace: curl
    17. spec:
    18.   sources:
    19.   - kind: ServiceAccount
    20.     name: curl
    21.     namespace: curl
    22.   hosts:
    23.   - httpbin.org
    24.   ports:
    25.   - number: 80
    26.     protocol: http
    27.   matches:
    28.   - apiGroup: specs.smi-spec.io/v1alpha4
    29.     kind: HTTPRouteGroup
    30.     name: egress-http-route
    31. EOF

  3. Confirm the curl client is able to make successful HTTP requests to http://httpbin.org:80/get.

    1. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/get
    2. HTTP/1.1 200 OK
    3. date: Thu, 13 May 2021 21:49:35 GMT
    4. content-type: application/json
    5. content-length: 335
    6. server: envoy
    7. access-control-allow-origin: *
    8. access-control-allow-credentials: true
    9. x-envoy-upstream-service-time: 168

  4. Confirm the curl client is unable to make successful HTTP requests to http://httpbin.org:80/status/200.

    1. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/status/200
    2. HTTP/1.1 404 Not Found
    3. date: Fri, 14 May 2021 17:08:48 GMT
    4. server: envoy
    5. transfer-encoding: chunked

  5. Update the matching SMI HTTPRouteGroup resource to allow requests to HTTP paths matching the regex /status.*.

    1. kubectl apply -f - <<EOF
    2. apiVersion: specs.smi-spec.io/v1alpha4
    3. kind: HTTPRouteGroup
    4. metadata:
    5.   name: egress-http-route
    6.   namespace: curl
    7. spec:
    8.   matches:
    9.   - name: get
    10.     pathRegex: /get
    11.   - name: status
    12.     pathRegex: /status.*
    13. EOF

  6. Confirm the curl client can now make successful HTTP requests to http://httpbin.org:80/status/200.

    1. $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/status/200
    2. HTTP/1.1 200 OK
    3. date: Fri, 14 May 2021 17:10:48 GMT
    4. content-type: text/html; charset=utf-8
    5. content-length: 0
    6. server: envoy
    7. access-control-allow-origin: *
    8. access-control-allow-credentials: true
    9. x-envoy-upstream-service-time: 188