Authentication using mTLS

mTLS authentication overview

Mutual TLS (mTLS) is a mutual authentication mechanism. Not only servers have keys and certs that the client uses to verify the identity of servers, clients also have keys and certs that the server uses to verify the identity of clients.

The following figure illustrates how Pulsar processes mTLS authentication between clients and servers.

Pulsar mTLS authentication process

Enable mTLS authentication on brokers

To configure brokers to authenticate clients using mTLS, add the following parameters to the conf/broker.conf. If you use a standalone Pulsar, you need to add these parameters to the conf/standalone.conf file:

  1. # enable authentication
  2. authenticationEnabled=true
  3. # set mTLS authentication provider
  4. authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
  6. # configure TLS for client to connect brokers
  7. brokerClientTlsEnabled=true
  8. brokerClientTrustCertsFilePath=/path/to/ca.cert.pem
  9. brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
  10. brokerClientAuthenticationParameters={"tlsCertFile":"/path/to/broker_client.cert.pem","tlsKeyFile":"/path/to/broker_client.key-pk8.pem"}
  12. # configure TLS ports
  13. brokerServicePortTls=6651
  14. webServicePortTls=8081
  16. # configure CA certificate
  17. tlsTrustCertsFilePath=/path/to/ca.cert.pem
  18. # configure server certificate
  19. tlsCertificateFilePath=/path/to/server.cert.pem
  20. # configure server's private key
  21. tlsKeyFilePath=/path/to/server.key-pk8.pem
  23. # enable mTLS
  24. tlsRequireTrustedClientCertOnConnect=true
  25. tlsAllowInsecureConnection=false
  27. # Tls cert refresh duration in seconds (set 0 to check on every new connection)
  28. tlsCertRefreshCheckDurationSec=300

Enable mTLS authentication on proxies

To configure proxies to authenticate clients using mTLS, add the following parameters to the conf/proxy.conf file.

  1. # enable authentication
  2. authenticationEnabled=true
  3. # set mTLS authentication provider
  4. authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
  6. # configure TLS for client to connect proxies
  7. tlsEnabledWithBroker=true
  8. brokerClientTrustCertsFilePath=/path/to/ca.cert.pem
  9. brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
  10. brokerClientAuthenticationParameters={"tlsCertFile":"/path/to/proxy.cert.pem","tlsKeyFile":"/path/to/proxy.key-pk8.pem"}
  12. # configure TLS ports
  13. brokerServicePortTls=6651
  14. webServicePortTls=8081
  16. # configure CA certificate
  17. tlsTrustCertsFilePath=/path/to/ca.cert.pem
  18. # configure server certificate
  19. tlsCertificateFilePath=/path/to/server.cert.pem
  20. # configure server's private key
  21. tlsKeyFilePath=/path/to/server.key-pk8.pem
  23. # enable mTLS
  24. tlsRequireTrustedClientCertOnConnect=true
  25. tlsAllowInsecureConnection=false

Configure mTLS authentication in Pulsar clients

When using mTLS authentication, clients connect via TLS transport. You need to configure clients to use https:// and the 8443 port for the web service URL, use pulsar+ssl:// and the 6651 port for the broker service URL.

  • Java
  • Python
  • C++
  • Node.js
  • Go
  • C#
  1. import org.apache.pulsar.client.api.PulsarClient;
  3. PulsarClient client = PulsarClient.builder()
  4.     .serviceUrl("pulsar+ssl://broker.example.com:6651/")
  5.     .tlsTrustCertsFilePath("/path/to/ca.cert.pem")
  6.     .authentication("org.apache.pulsar.client.impl.auth.AuthenticationTls",
  7.                     "tlsCertFile:/path/to/client.cert.pem,tlsKeyFile:/path/to/client.key-pk8.pem")
  8.     .build();
  1. from pulsar import Client, AuthenticationTLS
  3. auth = AuthenticationTLS("/path/to/client.cert.pem", "/path/to/client.key-pk8.pem")
  4. client = Client("pulsar+ssl://broker.example.com:6651/",
  5.                 tls_trust_certs_file_path="/path/to/ca.cert.pem",
  6.                 tls_allow_insecure_connection=False,
  7.                 authentication=auth)
  1. #include <pulsar/Client.h>
  3. pulsar::ClientConfiguration config;
  4. config.setUseTls(true);
  5. config.setTlsTrustCertsFilePath("/path/to/ca.cert.pem");
  6. config.setTlsAllowInsecureConnection(false);
  8. pulsar::AuthenticationPtr auth = pulsar::AuthTls::create("/path/to/client.cert.pem",
  9.                                                          "/path/to/client.key-pk8.pem")
  10. config.setAuth(auth);
  12. pulsar::Client client("pulsar+ssl://broker.example.com:6651/", config);
  1. const Pulsar = require('pulsar-client');
  3. (async () => {
  4.   const auth = new Pulsar.AuthenticationTls({
  5.     certificatePath: '/path/to/client.cert.pem',
  6.     privateKeyPath: '/path/to/client.key-pk8.pem',
  7.   });
  9.   const client = new Pulsar.Client({
  10.     serviceUrl: 'pulsar+ssl://broker.example.com:6651/',
  11.     authentication: auth,
  12.     tlsTrustCertsFilePath: '/path/to/ca.cert.pem',
  13.   });
  14. })();
  1. client, err := pulsar.NewClient(ClientOptions{
  2.         URL:                   "pulsar+ssl://broker.example.com:6651/",
  3.         TLSTrustCertsFilePath: "/path/to/ca.cert.pem",
  4.         Authentication:        pulsar.NewAuthenticationTLS("/path/to/client.cert.pem", "/path/to/client.key-pk8.pem"),
  5.     })
  1. var clientCertificate = new X509Certificate2("admin.pfx");
  2. var client = PulsarClient.Builder()
  3.                          .AuthenticateUsingClientCertificate(clientCertificate)
  4.                          .Build();

Configure mTLS authentication in CLI tools

Command-line tools like pulsar-admin, pulsar-perf, and pulsar-client use the conf/client.conf config file in a Pulsar installation.

To use mTLS authentication with the CLI tools of Pulsar, you need to add the following parameters to the conf/client.conf file, alongside the configurations to enable mTLS encryption:

  1. webServiceUrl=https://localhost:8081/
  2. brokerServiceUrl=pulsar+ssl://localhost:6651/
  3. authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
  4. authParams=tlsCertFile:/path/to/admin.cert.pem,tlsKeyFile:/path/to/admin.key-pk8.pem

Configure mTLS authentication with KeyStore

Apache Pulsar supports TLS encryption and mTLS authentication between clients and Apache Pulsar service. By default, it uses PEM format file configuration.

To configure mTLS authentication with KeyStore, complete the following steps.

Step 1: Configure brokers

Configure the broker.conf file as follows.

  1. # Configuration to enable authentication
  2. authenticationEnabled=true
  3. authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
  5. # Enable KeyStore type
  6. tlsEnabledWithKeyStore=true
  8. # key store
  9. tlsKeyStoreType=JKS
  10. tlsKeyStore=/var/private/tls/broker.keystore.jks
  11. tlsKeyStorePassword=brokerpw
  13. # trust store
  14. tlsTrustStoreType=JKS
  15. tlsTrustStore=/var/private/tls/broker.truststore.jks
  16. tlsTrustStorePassword=brokerpw
  18. # internal client/admin-client config
  19. brokerClientTlsEnabled=true
  20. brokerClientTlsEnabledWithKeyStore=true
  21. brokerClientTlsTrustStoreType=JKS
  22. brokerClientTlsTrustStore=/var/private/tls/client.truststore.jks
  23. brokerClientTlsTrustStorePassword=clientpw
  24. # internal auth config
  25. brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls
  26. brokerClientAuthenticationParameters={"keyStoreType":"JKS","keyStorePath":"/var/private/tls/client.keystore.jks","keyStorePassword":"clientpw"}
  28. tlsRequireTrustedClientCertOnConnect=true
  29. tlsAllowInsecureConnection=false

Step 2: Configure clients

Besides configuring TLS encryption, you need to configure the KeyStore, which contains a valid CN as client role, for clients.

For example:

  1. for Command-line tools like pulsar-admin, pulsar-perf, and pulsar-client, set the conf/client.conf file in a Pulsar installation.

    1. webServiceUrl=https://broker.example.com:8443/
    2. brokerServiceUrl=pulsar+ssl://broker.example.com:6651/
    3. useKeyStoreTls=true
    4. tlsTrustStoreType=JKS
    5. tlsTrustStorePath=/var/private/tls/client.truststore.jks
    6. tlsTrustStorePassword=clientpw
    7. authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls
    8. authParams={"keyStoreType":"JKS","keyStorePath":"/var/private/tls/client.keystore.jks","keyStorePassword":"clientpw"}

  2. for Java client

    1. import org.apache.pulsar.client.api.PulsarClient;
    3. PulsarClient client = PulsarClient.builder()
    4.     .serviceUrl("pulsar+ssl://broker.example.com:6651/")
    5.     .useKeyStoreTls(true)
    6.     .tlsTrustStorePath("/var/private/tls/client.truststore.jks")
    7.     .tlsTrustStorePassword("clientpw")
    8.     .allowTlsInsecureConnection(false)
    9.     .enableTlsHostnameVerification(false)
    10.     .authentication(
    11.             "org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls",
    12.             "keyStoreType:JKS,keyStorePath:/var/private/tls/client.keystore.jks,keyStorePassword:clientpw")
    13.     .build();

  3. for Java admin client

    1.     PulsarAdmin amdin = PulsarAdmin.builder().serviceHttpUrl("https://broker.example.com:8443")
    2.         .useKeyStoreTls(true)
    3.         .tlsTrustStorePath("/var/private/tls/client.truststore.jks")
    4.         .tlsTrustStorePassword("clientpw")
    5.         .allowTlsInsecureConnection(false)
    6.         .enableTlsHostnameVerification(false)
    7.         .authentication(
    8.                "org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls",
    9.                "keyStoreType:JKS,keyStorePath:/var/private/tls/client.keystore.jks,keyStorePassword:clientpw")
    10.         .build();

Authentication using mTLS - 图2note

Configure tlsTrustStorePath when you set useKeyStoreTls to true.