WebSockets

Requirements

Sandbox environment

Setup your sandbox environment with Docker and Docker Compose, and clone the Envoy repository with Git.

openssl

Generate SSL keys and certificates.

This example walks through some of the ways that Envoy can be configured to proxy WebSockets.

It demonstrates terminating a WebSocket connection with and without TLS, and provides some basic examples of proxying to encrypted and non-encrypted upstream sockets.

Warning

For the sake of simplicity, the examples provided here do not authenticate any client certificates, or validate any of the provided certificates.

When using TLS, you are strongly encouraged to validate all certificates wherever possible.

You should also authenticate clients where you control both sides of the connection, or relevant protocols are available.

Step 1: Create a certificate file for wss

Change directory to examples/websocket in the Envoy repository.

  1. $ pwd
  2. envoy/examples/websocket
  3. $ mkdir -p certs
  4. $ openssl req -batch -new -x509 -nodes -keyout certs/key.pem -out certs/cert.pem
  5. Generating a RSA private key
  6. ..................................................................................................................+++++
  7. ......+++++
  8. writing new private key to 'certs/key.pem'
  9. -----
  10. $ openssl pkcs12 -export -passout pass: -out certs/output.pkcs12 -inkey certs/key.pem -in certs/cert.pem

Step 2: Build and start the sandbox

This starts three proxies listening on localhost ports 10000-30000.

It also starts two upstream services, one ws and one wss.

The upstream services listen on the internal Docker network on ports 80 and 443 respectively.

The socket servers are very trivial implementations, that simply output [ws] HELO and [wss] HELO in response to any input.

  1. $ docker-compose pull
  2. $ docker-compose up --build -d
  3. $ docker-compose ps
  5.             Name                             Command               State            Ports
  6. ---------------------------------------------------------------------------------------------------
  7. websocket_proxy-ws_1                /docker-entrypoint.sh /usr ... Up      0.0.0.0:10000->10000/tcp
  8. websocket_proxy-wss_1               /docker-entrypoint.sh /usr ... Up      0.0.0.0:20000->10000/tcp
  9. websocket_proxy-wss-passthrough_1   /docker-entrypoint.sh /usr ... Up      0.0.0.0:30000->10000/tcp
  10. websocket_service-ws_1              websocat -E ws-listen:0.0. ... Up
  11. websocket_service-wss_1             websocat wss-listen:0.0.0. ... Up

Step 3: Test proxying ws -> ws

The proxy listening on port 10000 terminates the WebSocket connection without TLS and then proxies to an upstream socket, also without TLS.

In order for Envoy to terminate the WebSocket connection, the upgrade_configs in HttpConnectionManager must be set, as can be seen in the provided ws -> ws configuration:

  1.  1static_resources:
  2.  2  listeners:
  3.  3  - address:
  4.  4      socket_address:
  5.  5        address: 0.0.0.0
  6.  6        port_value: 10000
  7.  7    filter_chains:
  8.  8    - filters:
  9.  9      - name: envoy.filters.network.http_connection_manager
  10. 10        typed_config:
  11. 11          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
  12. 12          stat_prefix: ingress_ws_to_ws
  13. 13          upgrade_configs:
  14. 14          - upgrade_type: websocket
  15. 15          route_config:
  16. 16            name: local_route
  17. 17            virtual_hosts:
  18. 18            - name: app
  19. 19              domains:
  20. 20              - "*"
  21. 21              routes:
  22. 22              - match:
  23. 23                  prefix: "/"
  24. 24                route:
  25. 25                  cluster: service_ws
  26. 26          http_filters:
  27. 27          - name: envoy.filters.http.router
  28. 28            typed_config:
  29. 29              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

You can start an interactive session with the socket as follows:

  1. $ docker run -ti --network=host solsson/websocat ws://localhost:10000
  2. HELO
  3. [ws] HELO
  4. GOODBYE
  5. [ws] HELO

Type Ctrl-c to exit the socket session.

Step 4: Test proxying wss -> wss

The proxy listening on port 20000 terminates the WebSocket connection with TLS and then proxies to an upstream TLS WebSocket.

In addition to the upgrade_configs in HttpConnectionManager, the wss -> wss configuration adds a TLS transport_socket to both the listener and the cluster.

You can start an interactive session with the socket as follows:

  1. $ docker run -ti --network=host solsson/websocat --insecure wss://localhost:20000
  2. HELO
  3. [wss] HELO
  4. GOODBYE
  5. [wss] HELO

Type Ctrl-c to exit the socket session.

Step 5: Test proxying wss passthrough

The proxy listening on port 30000 passes through all TCP traffic to an upstream TLS WebSocket.

The wss passthrough configuration requires no TLS or HTTP setup, and instead uses a simple tcp_proxy.

You can start an interactive session with the socket as follows:

  1. $ docker run -ti --network=host solsson/websocat --insecure wss://localhost:30000
  2. HELO
  3. [wss] HELO
  4. GOODBYE
  5. [wss] HELO

Type Ctrl-c to exit the socket session.

See also

Securing Envoy quick start guide

Outline of key concepts for securing Envoy.

Double proxy sandbox

An example of securing traffic between proxies with validation and mutual authentication using mTLS with non-HTTP traffic.

TLS sandbox

Examples of various TLS termination patterns with Envoy.