Configure mTLS for client to APISIX

mTLS is a method for mutual authentication. Suppose in your network environment, only trusted clients are required to access the server. In that case, you can enable mTLS to verify the client’s identity and ensure the server API’s security. This article mainly introduces how to configure mutual authentication (mTLS) between the client and Apache APISIX.

Configuration

This example includes the following procedures:

  1. Generate certificates;
  2. Configure the certificate in APISIX;
  3. Create and configure routes in APISIX;
  4. Test verification.

To make the test results clearer, the examples mentioned in this article pass some information about the client credentials upstream, including: serial, fingerprint and common name.

Generate certificates

We need to generate three test certificates: the root, server, and client. Just use the following command to generate the test certificates we need via OpenSSL.

  1. # For ROOT CA
  2. openssl genrsa -out ca.key 2048
  3. openssl req -new -sha256 -key ca.key -out ca.csr -subj "/CN=ROOTCA"
  4. openssl x509 -req -days 36500 -sha256 -extensions v3_ca -signkey ca.key -in ca.csr -out ca.cer
  6. # For server certificate
  7. openssl genrsa -out server.key 2048
  8. # Note: The `test.com` in the CN value is the domain name/hostname we want to test
  9. openssl req -new -sha256 -key server.key -out server.csr -subj "/CN=test.com"
  10. openssl x509 -req -days 36500 -sha256 -extensions v3_req  -CA  ca.cer -CAkey ca.key  -CAserial ca.srl  -CAcreateserial -in server.csr -out server.cer
  12. # For client certificate
  13. openssl genrsa -out client.key 2048
  14. openssl req -new -sha256 -key client.key  -out client.csr -subj "/CN=CLIENT"
  15. openssl x509 -req -days 36500 -sha256 -extensions v3_req  -CA  ca.cer -CAkey ca.key  -CAserial ca.srl  -CAcreateserial -in client.csr -out client.cer
  17. # Convert client certificate to pkcs12 for Windows usage (optional)
  18. openssl pkcs12 -export -clcerts -in client.cer -inkey client.key -out client.p12

Configure the certificate in APISIX

Use the curl command to request APISIX Admin API to set up SSL for specific SNI.

Configure mTLS for client to APISIX - 图1note

Note that the newline character in the certificate needs to be replaced with its escape character \n.

  1. curl -X PUT 'http://127.0.0.1:9180/apisix/admin/ssls/1' \
  2. --header 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' \
  3. --header 'Content-Type: application/json' \
  4. --data-raw '{
  5.     "sni": "test.com",
  6.     "cert": "<content of server.cer>",
  7.     "key": "<content of server.key>",
  8.     "client": {
  9.         "ca": "<content of ca.cer>"
  10.     }
  11. }'
  • sni: Specify the domain name (CN) of the certificate. When the client tries to handshake with APISIX via TLS, APISIX will match the SNI data in ClientHello with this field and find the corresponding server certificate for handshaking.
  • cert: The server certificate.
  • key: The private key of the server certificate.
  • client.ca: The CA (certificate authority) file to verfiy the client certificate. For demonstration purposes, the same CA is used here.

Configure the route in APISIX

Use the curl command to request the APISIX Admin API to create a route.

  1. curl -X PUT 'http://127.0.0.1:9180/apisix/admin/routes/1' \
  2. --header 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' \
  3. --header 'Content-Type: application/json' \
  4. --data-raw '{
  5.     "uri": "/anything",
  6.     "plugins": {
  7.         "proxy-rewrite": {
  8.             "headers": {
  9.                 "X-Ssl-Client-Fingerprint": "$ssl_client_fingerprint",
  10.                 "X-Ssl-Client-Serial": "$ssl_client_serial",
  11.                 "X-Ssl-Client-S-DN": "$ssl_client_s_dn"
  12.             }
  13.         }
  14.     },
  15.     "upstream": {
  16.         "nodes": {
  17.             "httpbin.org":1
  18.         },
  19.         "type":"roundrobin"
  20.     }
  21. }'

APISIX automatically handles the TLS handshake based on the SNI and the SSL resource created in the previous step, so we do not need to specify the hostname in the route (but it is possible to specify the hostname if you need it).

Also, in the curl command above, we enabled the proxy-rewrite plugin, which will dynamically update the request header information. The source of the variable values in the example are the NGINX variables, and you can find them here: http://nginx.org/en/docs/http/ngx_http_ssl_module.html#variables.

Test

Since we are using the domain test.com as the test domain, we have to add the test domain to your DNS or local hosts file before we can start the verification.

  1. If we don’t use hosts and just want to test the results, then you can do so directly using the following command.
  1. curl --resolve "test.com:9443:127.0.0.1" https://test.com:9443/anything -k --cert ./client.cer --key ./client.key
  1. If you need to modify hosts, please read the following example (for Ubuntu).

  • Modify the /etc/hosts file

    1. # 127.0.0.1 localhost
    2. 127.0.0.1 test.com

  • Verify that the test domain name is valid

    1. ping test.com
    3. PING test.com (127.0.0.1) 56(84) bytes of data.
    4. 64 bytes from localhost.localdomain (127.0.0.1): icmp_seq=1 ttl=64 time=0.028 ms
    5. 64 bytes from localhost.localdomain (127.0.0.1): icmp_seq=2 ttl=64 time=0.037 ms
    6. 64 bytes from localhost.localdomain (127.0.0.1): icmp_seq=3 ttl=64 time=0.036 ms
    7. 64 bytes from localhost.localdomain (127.0.0.1): icmp_seq=4 ttl=64 time=0.031 ms
    8. ^C
    9. --- test.com ping statistics ---
    10. 4 packets transmitted, 4 received, 0% packet loss, time 3080ms
    11. rtt min/avg/max/mdev = 0.028/0.033/0.037/0.003 ms

  • Test results

    1. curl https://test.com:9443/anything -k --cert ./client.cer --key ./client.key

    You will then receive the following response body.

    1. {
    2.   "args": {},
    3.   "data": "",
    4.   "files": {},
    5.   "form": {},
    6.   "headers": {
    7.     "Accept": "*/*",
    8.     "Host": "test.com",
    9.     "User-Agent": "curl/7.81.0",
    10.     "X-Amzn-Trace-Id": "Root=1-63256343-17e870ca1d8f72dc40b2c5a9",
    11.     "X-Forwarded-Host": "test.com",
    12.     "X-Ssl-Client-Fingerprint": "c1626ce3bca723f187d04e3757f1d000ca62d651",
    13.     "X-Ssl-Client-S-Dn": "CN=CLIENT",
    14.     "X-Ssl-Client-Serial": "5141CC6F5E2B4BA31746D7DBFE9BA81F069CF970"
    15.   },
    16.   "json": null,
    17.   "method": "GET",
    18.   "origin": "127.0.0.1",
    19.   "url": "http://test.com/anything"
    20. }

Since we configured the proxy-rewrite plugin in the example, we can see that the response body contains the request body received upstream, containing the correct data.

MTLS bypass based on regular expression matching against URI

APISIX allows configuring an URI whitelist to bypass MTLS. If the URI of a request is in the whitelist, then the client certificate will not be checked. Note that other URIs of the associated SNI will get HTTP 400 response instead of alert error in the SSL handshake phase, if the client certificate is missing or invalid.

Timing diagram

skip mtls

Example

Configure mTLS for client to APISIX - 图3note

You can fetch the admin_key from config.yaml and save to an environment variable with the following command:

  1. admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"//g')
  1. Configure route and ssl via admin API
  1. curl http://127.0.0.1:9180/apisix/admin/routes/1 \
  2. -H "X-API-KEY: $admin_key" -X PUT -d '
  3. {
  4.     "uri": "/*",
  5.     "upstream": {
  6.         "nodes": {
  7.             "httpbin.org": 1
  8.         }
  9.     }
  10. }'
  12. curl http://127.0.0.1:9180/apisix/admin/ssls/1 \
  13. -H "X-API-KEY: $admin_key" -X PUT -d '
  14. {
  15.     "cert": "'"$(<t/certs/mtls_server.crt)"'",
  16.     "key": "'"$(<t/certs/mtls_server.key)"'",
  17.     "snis": [
  18.         "*.apisix.dev"
  19.     ],
  20.     "client": {
  21.         "ca": "'"$(<t/certs/mtls_ca.crt)"'",
  22.         "depth": 10,
  23.         "skip_mtls_uri_regex": [
  24.             "/anything.*"
  25.         ]
  26.     }
  27. }'
  1. If the client certificate is missing and the URI is not in the whitelist, then you’ll get HTTP 400 response.
  1. curl https://admin.apisix.dev:9443/uuid -v \
  2. --resolve 'admin.apisix.dev:9443:127.0.0.1' --cacert t/certs/mtls_ca.crt
  3. * Added admin.apisix.dev:9443:127.0.0.1 to DNS cache
  4. * Hostname admin.apisix.dev was found in DNS cache
  5. *   Trying 127.0.0.1:9443...
  6. * TCP_NODELAY set
  7. * Connected to admin.apisix.dev (127.0.0.1) port 9443 (#0)
  8. * ALPN, offering h2
  9. * ALPN, offering http/1.1
  10. * successfully set certificate verify locations:
  11. *   CAfile: t/certs/mtls_ca.crt
  12.   CApath: /etc/ssl/certs
  13. * TLSv1.3 (OUT), TLS handshake, Client hello (1):
  14. * TLSv1.3 (IN), TLS handshake, Server hello (2):
  15. * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
  16. * TLSv1.3 (IN), TLS handshake, Request CERT (13):
  17. * TLSv1.3 (IN), TLS handshake, Certificate (11):
  18. * TLSv1.3 (IN), TLS handshake, CERT verify (15):
  19. * TLSv1.3 (IN), TLS handshake, Finished (20):
  20. * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
  21. * TLSv1.3 (OUT), TLS handshake, Certificate (11):
  22. * TLSv1.3 (OUT), TLS handshake, Finished (20):
  23. * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
  24. * ALPN, server accepted to use h2
  25. * Server certificate:
  26. *  subject: C=cn; ST=GuangDong; L=ZhuHai; CN=admin.apisix.dev; OU=ops
  27. *  start date: Dec  1 10:17:24 2022 GMT
  28. *  expire date: Aug 18 10:17:24 2042 GMT
  29. *  subjectAltName: host "admin.apisix.dev" matched cert's "admin.apisix.dev"
  30. *  issuer: C=cn; ST=GuangDong; L=ZhuHai; CN=ca.apisix.dev; OU=ops
  31. *  SSL certificate verify ok.
  32. * Using HTTP2, server supports multi-use
  33. * Connection state changed (HTTP/2 confirmed)
  34. * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
  35. * Using Stream ID: 1 (easy handle 0x56246de24e30)
  36. > GET /uuid HTTP/2
  37. > Host: admin.apisix.dev:9443
  38. > user-agent: curl/7.68.0
  39. > accept: */*
  40. >
  41. * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
  42. * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
  43. * old SSL session ID is stale, removing
  44. * Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
  45. < HTTP/2 400
  46. < date: Fri, 21 Apr 2023 07:53:23 GMT
  47. < content-type: text/html; charset=utf-8
  48. < content-length: 229
  49. < server: APISIX/3.2.0
  50. <
  51. <html>
  52. <head><title>400 Bad Request</title></head>
  53. <body>
  54. <center><h1>400 Bad Request</h1></center>
  55. <hr><center>openresty</center>
  56. <p><em>Powered by <a href="https://apisix.apache.org/">APISIX</a>.</em></p></body>
  57. </html>
  58. * Connection #0 to host admin.apisix.dev left intact
  1. Although the client certificate is missing, but the URI is in the whitelist, you get successful response.
  1. curl https://admin.apisix.dev:9443/anything/foobar -i \
  2. --resolve 'admin.apisix.dev:9443:127.0.0.1' --cacert t/certs/mtls_ca.crt
  3. HTTP/2 200
  4. content-type: application/json
  5. content-length: 416
  6. date: Fri, 21 Apr 2023 07:58:28 GMT
  7. access-control-allow-origin: *
  8. access-control-allow-credentials: true
  9. server: APISIX/3.2.0
  10. ...

Conclusion

If you don’t want to use curl or test on windows, you can read this gist for more details. APISIX mTLS for client to APISIX.

For more information about the mTLS feature of Apache APISIX, you can read Mutual TLS Authentication.