The Pulsar proxy
Pulsar代理是一个可选的网关,你可以在Pulsar集群中的brokers之前运行它们。 You can run a Pulsar proxy in cases when direction connections between clients and Pulsar brokers are either infeasible, undesirable, or both, for example when you run Pulsar in a cloud environment or on Kubernetes or an analogous platform.
配置Proxy
Proxy 有办法来找到集群中 broker 的地址。 你可以通过配置代理来直接连接到服务发现,或者在配置中指定broker URL。
Option 1: Use service discovery
Pulsar uses ZooKeeper for service discovery. To connect the proxy to ZooKeeper, specify the following in conf/proxy.conf
.
zookeeperServers=zk-0,zk-1,zk-2
configurationStoreServers=zk-0:2184,zk-remote:2184
If you use service discovery, the network ACL must allow the proxy to talk to the ZooKeeper nodes on the zookeeper client port, which is usually 2181, and on the configuration store client port, which is 2184 by default. 打开网络ACLs意味着如果有人破坏了代理,那么他们可以完全访问ZooKeeper。 For this reason, using broker URLs to configure the proxy is more secure.
Option 2: Use broker URLs
配置代理的更安全的方法是指定连接到brokers的URL。
Authorization at the proxy requires access to ZooKeeper, so if you use these broker URLs to connect to the brokers, you should disable the Proxy level authorization. Brokers still authorize requests after the proxy forwards them.
您可以在 conf/proxy.conf
中配置broker URLs,如下所示。
brokerServiceURL=pulsar://brokers.example.com:6650
brokerWebServiceURL=http://brokers.example.com:8080
functionWorkerWebServiceURL=http://function-workers.example.com:8080
Or if you use TLS:
brokerServiceURLTLS=pulsar+ssl://brokers.example.com:6651
brokerWebServiceURLTLS=https://brokers.example.com:8443
functionWorkerWebServiceURL=https://function-workers.example.com:8443
提供的URLs中的主机名应该是指向多个 broker 的DNS条目,或由多个broker 的IP地址支持的虚拟IP,以便在单个 broker 不可用时 ,代理不会失去与 Pulsar 群集的连接。
The ports to connect to the brokers (6650 and 8080, or in the case of TLS, 6651 and 8443) should be open in the network ACLs.
Note that if you do not use functions, then you do not need to configure functionWorkerWebServiceURL
.
启动 Proxy
To start the proxy:
$ cd /path/to/pulsar/directory
$ bin/pulsar proxy
You can run as many instances of the Pulsar proxy in a cluster as you want.
停止 Proxy
The Pulsar proxy runs by default in the foreground. To stop the proxy, simply stop the process in which the proxy is running.
Proxy frontends
You can run the Pulsar proxy behind some kind of load-distributing frontend, such as an HAProxy load balancer.
Pulsar 客户端连接 Proxy
Once your Pulsar proxy is up and running, preferably behind a load-distributing frontend, clients can connect to the proxy via whichever address that the frontend uses. If the address is the DNS address pulsar.cluster.default
, for example, then the connection URL for clients is pulsar://pulsar.cluster.default:6650
.
Proxy configuration
You can configure the Pulsar proxy using the proxy.conf
configuration file. The following parameters are available in that file:
配置项 | Description | 默认值 |
---|---|---|
zookeeperServers | ZooKeeper quorum 连接字符串(以逗号分隔的列表) | |
configurationStoreServers | 配置存储连接字符串(以逗号分隔的列表) | |
zookeeperSessionTimeoutMs | ZooKeeper会话超时(以毫秒为单位) | 30000 |
servicePort | 用于服务器二进制Protobuf请求的端口 | 6650 |
servicePortTls | 用于服务器二进制Protobuf TLS请求的端口 | 6651 |
statusFilePath | 在响应服务发现健康检查时,用于确定代理实例的轮换状态的文件的路径 | |
advertisedAddress | Hostname or IP address the service advertises to the outside world. | InetAddress.getLocalHost().getHostname() |
authenticationEnabled | 是否为Pulsar代理启用身份验证 | false |
authenticateMetricsEndpoint | Whether the ‘/metrics’ endpoint requires authentication. Defaults to true. ‘authenticationEnabled’ must also be set for this to take effect. | true |
authenticationProviders | 身份验证提供者名称列表(以逗号分隔的类名列表) | |
authorizationEnabled | 是否由Pulsar代理强制执行授权 | false |
authorizationProvider | 授权提供程序的完全限定类名 | org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider |
brokerClientAuthenticationPlugin | Pulsar代理使用的身份验证插件,用于对Pulsar brokers进行身份验证 | |
brokerClientAuthenticationParameters | Pulsar代理用于对Pulsar Brokers进行身份验证的参数 | |
brokerClientTrustCertsFilePath | Pulsar代理用于对Pulsar Brokers进行身份验证的可信证书的路径 | |
superUserRoles | Role names that are treated as “super-users,” meaning that they are able to perform all admin | |
forwardAuthorizationCredentials | Whether client authorization credentials are forwared to the broker for re-authorization. Authentication must be enabled via authenticationEnabled=true for this to take effect. | false |
maxConcurrentInboundConnections | Max concurrent inbound connections. The proxy rejects requests beyond that. | 10000 |
maxConcurrentLookupRequests | Max concurrent outbound connections. The proxy errors out requests beyond that. | 50000 |
tlsEnabledInProxy | 是否为代理启用TLS | false |
tlsEnabledWithBroker | 与Pulsar Brokers通信时是否启用TLS | false |
tlsCertificateFilePath | TLS证书文件的路径 | |
tlsKeyFilePath | TLS私钥文件的路径 | |
tlsTrustCertsFilePath | 受信任的TLS证书pem文件的路径 | |
tlsHostnameVerificationEnabled | 当代理与brokers建立TLS连接时是否验证主机名 | false |
tlsRequireTrustedClientCertOnConnect | Whether client certificates are required for TLS. Connections are rejected if the client certificate is not trusted. | false |