Pulsar 配置可以通过 Pulsar 安装 目录下的 conf
文件夹里的一系列配置文件进行管理。
BookKeeper
BookKeeper 是一个冗余的日志存储系统,Pulsar 用它来持久化存储所有消息。
Name | Description | 默认值 |
---|---|---|
bookiePort | bookie 服务器监听的端口。 | 3181 |
allowLoopback | 是否允许 bookie 使用 loopback 接口作为它的主要接口(例如,用于确立身份的接口)。 默认情况下,loopback 接口不允许作为主接口。 使用 loopback 接口作为主接口通常意味着配置错误。 例如,在某些 VPS 设置中,通常不会配置主机名或使主机名解析为 127.0.0.1 。 如果情况如此,那么集群中的所有 bookie 都会将其身份设置为 127.0.0.1:3181 并且只有一个能成功加入集群。 对于像这样配置的 VPS,你应该显性地设置监听接口。 | false |
listeningInterface | The network interface on which the bookie listens. If not set, the bookie will listen on all interfaces. | eth0 |
journalDirectory | BookKeeper 输出 write-ahead 日志(WAL)的目录 | data/bookkeeper/journal |
ledgerDirectories | BookKeeper 输出 ledger 快照的目录。 这里可以定义多个目录来存储以逗号分隔的快照,比如 ledgerDirectories=/tmp/bk1-data,/tmp/bk2-data 。 最理想的情况是,ledger 目录和日志目录都是在不同的设备中,这减少了随机读写和顺序写入之间的争执。 可以用单个磁盘运行,但性能将显著降低。 | data/bookkeeper/ledgers |
ledgerManagerType | 用于管理 ledger 如何存储与管理,以及垃圾收集的 ledger 管理器类型。 查看 BookKeeper Internals 获取更多信息。 | hierarchical |
zkLedgersRootPath | The root ZooKeeper path used to store ledger metadata. This parameter is used by the ZooKeeper-based ledger manager as a root znode to store all ledgers. | /ledgers |
ledgerStorageClass | Ledger 存储实现类 | org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage |
entryLogFilePreallocationEnabled | 启用或禁用条目日志记录器预分配 | true |
logSizeLimit | Max file size of the entry logger, in bytes. A new entry log file will be created when the old one reaches the file size limitation. | 2147483648 |
minorCompactionThreshold | 次级压缩的阈值。 其剩余大小百分比低于此阈值的条目日志文件将在次级压缩中被压缩。 如果设置为零,次级压缩将被禁用。 | 0.2 |
minorCompactionInterval | Time interval to run minor compaction, in seconds. 如果设置为零,次级压缩将被禁用。 | 3600 |
majorCompactionThreshold | 首级压缩的阈值。 其剩余大小百分比低于此阈值的条目日志文件将在首级压缩中被压缩。 剩余大小百分比仍高于阈值的条目日志文件永远不会被压缩。 如果设置为零,次级压缩将被禁用。 | 0.5 |
majorCompactionInterval | The time interval to run major compaction, in seconds. If set to less than zero, the major compaction is disabled. | 86400 |
compactionMaxOutstandingRequests | 设置压缩过程中无需刷写的最大条目数。 在压缩时,条目被写入条目日志,新的偏移被缓存在内存中。 一旦条目日志被刷写,索引将被新的偏移量更新。 此参数控制在强制刷写之前添加到条目日志的条目数量。 此参数的值越狱高意味着将有更多内存用于偏移量。 每个偏移量由三个 long 型组成。 此参数不应被修改,除非你完全了解其后果。 | 100000 |
compactionRate | 压缩过程中读取条目的频率,以每秒添加数为单位。 | 1000 |
isThrottleByBytes | 以字节或条目为单位进行阈值压缩 | false |
compactionRateByEntries | 压缩过程中读取条目的频率,以每秒添加数为单位。 | 1000 |
compactionRateByBytes | Set the rate at which compaction will readd entries. The unit is bytes added per second. | 1000000 |
journalMaxSizeMB | Max file size of journal file, in megabytes. A new journal file will be created when the old one reaches the file size limitation. | 2048 |
journalMaxBackups | The max number of old journal filse to keep. Keeping a number of old journal files would help data recovery in special cases. | 5 |
journalPreAllocSizeMB | 每次在日志中预分配的空间大小。 | 16 |
journalWriteBufferSizeKB | 日志使用的写缓冲区大小。 | 64 |
journalRemoveFromPageCache | 强制写入后,页是否应从页缓存中删除。 | true |
journalAdaptiveGroupWrites | 是否将日志的强制写入进行分组,这能优化分组提交获得更高吞吐量。 | true |
journalMaxGroupWaitMSec | 实现分组写入日志的最大延迟。 | 1 |
journalAlignmentSize | 所有日志的写入和提交应与给定的大小对齐 | 4096 |
journalBufferedWritesThreshold | 要实现分组的最大写入缓存 | 524288 |
journalFlushWhenQueueEmpty | 是否在日志队列为空时刷写日志 | false |
numJournalCallbackThreads | 处理日志回调的线程数 | 8 |
rereplicationEntryBatchSize | 重复制时在 fragment 中保存的最大条目数 | 5000 |
openLedgerRereplicationGracePeriod | The grace period, in milliseconds, that the replication worker waits before fencing and replicating a ledger fragment that’s still being written to upon bookie failure. | 30000 |
gcWaitTime | 触发下一次垃圾收集的时间间隔(毫秒)。 由于垃圾收集工作在后台进行,过于频繁的垃圾收集工作将降低性能。 如果磁盘容量足够,最好用更大的 gc 时间间隔。 | 900000 |
gcOverreplicatedLedgerWaitTime | 触发下一次垃圾收集器收集过度复制的 ledger 的时间间隔(毫秒)。 这种情况不应经常发生,因为我们从 zookeeper 的 bookie 上读取所有 ledger 的元数据。 | 86400000 |
flushInterval | 刷新 ledger 索引页到磁盘的间隔,以毫秒计。 刷新索引文件会引入大量随机磁盘 I/O。 如果在不同的设备上将日志目录和 ledger 目录分开,刷写就不会影响性能。 但是,如果在同一设备上放置日志目录和 ledger 目录,性能会在频繁刷写的情况下大幅下降。 你可以考虑增加刷写间隔来获得更好的性能,但你需要在 bookie 服务器失败重启后花更多时间。 | 60000 |
bookieDeathWatchInterval | 查看 bookie 是否已死亡的时间间隔,以毫秒为单位 | 1000 |
zkServers | A list of one of more servers on which zookeeper is running. The server list can be comma separated values, for example: zkServers=zk1:2181,zk2:2181,zk3:2181. | localhost:2181 |
zkTimeout | ZooKeeper 客户端会话超时时间,以毫秒为单位。Bookie 服务器如果收到 SESSION_EXPIRED,将退出,因为它从 ZooKeeper 进行分区超过了会话超时时间。JVM 垃圾收集和磁盘 I/O 会导致 SESSION_EXPIRED。 增加这个值可以帮助避免这个问题 | 30000 |
serverTcpNoDelay | 此设置用于启用/禁用 Nagle 的算法,该算法能通过减少通过网络发送的数据包数量来提高 TCP/IP 网络效率。 如果你正在发送许多小消息,这样在单个 IP 数据包中就可以放入不止一个消息,设置 server.tcpnodelay 为 false 来启用 Nagle 算法可以提供更好的性能。 | true |
openFileLimit | 在 bookie 服务器上可以打开 ledger 索引文件的最大数量。如果 ledger 索引文件数量达到这个限制,bookie 服务器会开始将一些 ledger 从内存交换到磁盘。 过于频繁的交换会影响性能。 你可以根据你的要求调整这个数字以获得性能提升。 | 0 |
pageSize | ledger 缓存中索引页的大小,以字节为单位。一个大的索引页可以改善将页写入磁盘的性能。当你有少量有 ledger 并且这些 ledger 有着相似数量的条目时,会很高效。 如果你有大量的 ledger,而每个 ledger 都有较少的条目,较小的索引页面将会提高内存使用率。 | 8192 |
pageLimit | 在 ledger 缓存中提供多少索引页面。如果索引页面数量达到此限制,bookie 服务器会开始将一些 ledger 从内存交换到磁盘。 当你发现交换变得更频繁时,可以增加这个值。 但请确认 pageLimit*page 的大小不应超过 JVM 最大内存限制,否则你将会得到 OutOfMemoryException。 总的来说,使用更小的索引页面可以在 ledger 量大而条目数量少的情况下获得更好的性能。如果 pageLimit 为 -1, bookie 服务器会使用 1/3 的 JVM 内存来计算索引页面的限制数量。 | 0 |
readOnlyModeEnabled | 如果所有已配置的 ledger 目录已满,则只支持客户的读取请求。 如果 “readOnlyModeEnabled=true” 那么当所有的 ledger 磁盘满时,bookie 会被转换为只读模式并只处理读请求。 否则 bookie 将被关闭。 默认情况下,这会被禁用。 | true |
diskUsageThreshold | 对于每个 ledger 目录,可使用的最大磁盘空间。 默认值为 0.95f。 比如,最多只能使用 95% 的磁盘,此后不会将任何内容写入该分区。 如果所有 ledger 目录分区已满,并且设置了“readOnlyModeEnabled=true”,那么 bookie 将会转为只读模式。 有效值应在 0 和 1 之间(不包含端点)。 | 0.95 |
diskCheckInterval | 磁盘检查间隔,以毫秒为单位,检查 ledger 目录使用情况的时间间隔。 | 10000 |
auditorPeriodicCheckInterval | 审查者对集群内所有 ledger 进行核对的时间间隔。 默认情况下,每周运行一次。 间隔以秒为单位。 要完全禁用定期检查,请将此设置为 0。 请注意,定期检查会给集群带来额外的负荷,因此每天的运行次数不应超过一次。 | 604800 |
auditorPeriodicBookieCheckInterval | 审查者 bookie 检查的时间间隔。 审查者 bookie 检查,检查 ledger 的元数据来查看哪些 bookie 应该包含各个 ledger 的条目。 如果本应包含条目的 bookie 不可用,包含该条目的 ledger 会被标记去做恢复。 设置为 0 会禁用定期检查。 当 bookie 失效时,bookie 检查仍然会进行。 间隔以秒为单位。 | 86400 |
numAddWorkerThreads | number of threads that should handle write requests. if zero, the writes would be handled by netty threads directly. | 0 |
numReadWorkerThreads | number of threads that should handle read requests. if zero, the reads would be handled by netty threads directly. | 8 |
maxPendingReadRequestsPerThread | 如果启用了读 worker 线程,限制待处理请求的数量,以避免执行器队列无止境增长。 | 2500 |
readBufferSizeBytes | 用作 BufferedReadChannel 容量的字节数量。 | 4096 |
writeBufferSizeBytes | 用作写缓存容量的字节数 | 65536 |
useHostNameAsBookieID | Whether the bookie should use its hostname to register with the coordination service (e.g.: zookeeper service). When false, bookie will use its ipaddress for the registration. | false |
statsProviderClass | org.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider | |
prometheusStatsHttpPort | 8000 | |
dbStorage_writeCacheMaxSizeMb | Size of Write Cache. Memory is allocated from JVM direct memory. Write cache is used to buffer entries before flushing into the entry log For good performance, it should be big enough to hold a sub | 25% 的直接内存 |
dbStorage_readAheadCacheMaxSizeMb | Size of Read cache. Memory is allocated from JVM direct memory. This read cache is pre-filled doing read-ahead whenever a cache miss happens | 25% 的直接内存 |
dbStorage_readAheadCacheBatchSize | 当读缓存 miss 发生后预装填的条目数量 | 1000 |
dbStorage_rocksDB_blockCacheSize | Size of RocksDB block-cache. For best performance, this cache should be big enough to hold a significant portion of the index database which can reach ~2GB in some cases | 10% 的直接内存 |
dbStorage_rocksDB_writeBufferSizeMB | 64 | |
dbStorage_rocksDB_sstSizeInMB | 64 | |
dbStorage_rocksDB_blockSize | 65536 | |
dbStorage_rocksDB_bloomFilterBitsPerKey | 10 | |
dbStorage_rocksDB_numLevels | -1 | |
dbStorage_rocksDB_numFilesInLevel0 | 4 | |
dbStorage_rocksDB_maxSizeInLevel1MB | 256 | |
nettyMaxFrameSizeBytes | Set the maximum netty frame size in bytes. If the size of a received message is larger than the configured value, the message is rejected. | 1 GB |
Broker
Pulsar broker 负责处理从生产者发出的消息、向消费者派发消息、在集群间复制数据,等等。
<th>
Description
</th>
<th>
默认值
</th>
<td>
Specify multiple advertised listeners for the broker.<br><br>The format is <code><listener_name>:pulsar://<host>:<port></code>.<br><br>If there are multiple listeners, separate them with commas.<br><br><strong x-id="1">Note</strong>: do not use this configuration with <code>advertisedAddress</code> and <code>brokerServicePort</code>. If the value of this configuration is empty, the broker uses <code>advertisedAddress</code> and <code>brokerServicePort</code>
</td>
<td>
/
</td>
Name |
---|
advertisedListeners |
internalListenerName|Specify the internal listener name for the broker.
Note: the listener name must be contained in advertisedListeners
.
If the value of this configuration is empty, the broker uses the first listener as the internal listener.|/| |enablePersistentTopics| Whether persistent topics are enabled on the broker |true| |enableNonPersistentTopics| Whether non-persistent topics are enabled on the broker |true| |functionsWorkerEnabled| Whether the Pulsar Functions worker service is enabled in the broker |false| |zookeeperServers| Zookeeper quorum connection string || |zooKeeperCacheExpirySeconds|ZooKeeper cache expiry time in seconds|300 |configurationStoreServers| Configuration store connection string (as a comma-separated list) || |brokerServicePort| Broker data port |6650| |brokerServicePortTls| Broker data port for TLS |6651| |webServicePort| Port to use to server HTTP request |8080| |webServicePortTls| Port to use to server HTTPS request |8443| |webSocketServiceEnabled| Enable the WebSocket API service in broker |false| |bindAddress| Hostname or IP address the service binds on, default is 0.0.0.0. |0.0.0.0| |advertisedAddress| Hostname or IP address the service advertises to the outside world. If not set, the value of InetAddress.getLocalHost().getHostName()
is used. || |clusterName| Name of the cluster to which this broker belongs to || |brokerDeduplicationEnabled| Sets the default behavior for message deduplication in the broker. If enabled, the broker will reject messages that were already stored in the topic. This setting can be overridden on a per-namespace basis. |false| |brokerDeduplicationMaxNumberOfProducers| The maximum number of producers for which information will be stored for deduplication purposes. |10000| |brokerDeduplicationEntriesInterval| The number of entries after which a deduplication informational snapshot is taken. A larger interval will lead to fewer snapshots being taken, though this would also lengthen the topic recovery time (the time required for entries published after the snapshot to be replayed). |1000| |brokerDeduplicationProducerInactivityTimeoutMinutes| The time of inactivity (in minutes) after which the broker will discard deduplication information related to a disconnected producer. |360| |dispatchThrottlingRatePerReplicatorInMsg| The default messages per second dispatch throttling-limit for every replicator in replication. The value of 0
means disabling replication message dispatch-throttling| 0 | |dispatchThrottlingRatePerReplicatorInByte| The default bytes per second dispatch throttling-limit for every replicator in replication. The value of 0
means disabling replication message-byte dispatch-throttling| 0 | |zooKeeperSessionTimeoutMillis| Zookeeper session timeout in milliseconds |30000| |brokerShutdownTimeoutMs| Time to wait for broker graceful shutdown. After this time elapses, the process will be killed |60000| |skipBrokerShutdownOnOOM| Flag to skip broker shutdown when broker handles Out of memory error. |false| |backlogQuotaCheckEnabled| Enable backlog quota check. Enforces action on topic when the quota is reached |true| |backlogQuotaCheckIntervalInSeconds| How often to check for topics that have reached the quota |60| |backlogQuotaDefaultLimitGB| The default per-topic backlog quota limit | -1 | |allowAutoTopicCreation| Enable topic auto creation if a new producer or consumer connected |true| |allowAutoTopicCreationType| The topic type (partitioned or non-partitioned) that is allowed to be automatically created. |Partitioned| |allowAutoSubscriptionCreation| Enable subscription auto creation if a new consumer connected |true| |defaultNumPartitions| The number of partitioned topics that is allowed to be automatically created if allowAutoTopicCreationType
is partitioned |1| |brokerDeleteInactiveTopicsEnabled| Enable the deletion of inactive topics |true| |brokerDeleteInactiveTopicsFrequencySeconds| How often to check for inactive topics |60| | brokerDeleteInactiveTopicsMode | Set the mode to delete inactive topics.
delete_when_no_subscriptions
: 删除没有订阅或活动生产者的主题。delete_when_subscriptions_caught_up
: delete the topic whose subscriptions have no backlogs and which has no active producers or consumers. |delete_when_no_subscriptions
| | brokerDeleteInactiveTopicsMaxInactiveDurationSeconds | Set the maximum duration for inactive topics. If it is not specified, thebrokerDeleteInactiveTopicsFrequencySeconds
parameter is adopted. | N/A | |messageExpiryCheckIntervalInMinutes| 主动检查与清除过期消息的频率 |5| |brokerServiceCompactionMonitorIntervalInSeconds| 检查带压缩策略的主题是否需要压缩的时间间隔 |60| |activeConsumerFailoverDelayTimeMillis| 当活动消费者更换时,执行游标倒转和消息派发的延迟时间。 |1000| |clientLibraryVersionCheckEnabled| 启用客户端库版本最低要求检查 |false| |clientLibraryVersionCheckAllowUnversioned| 允许无版本信息的客户端库 |true| |statusFilePath| broker 响应发现服务健康检查时用于决定 rotation 状态的文件路径 || |preferLaterVersions| 如果为 true,(且启用了 ModularLoadManagerImpl),载入管理器会尝试仅使用运行最新软件版本的 broker(来最小化对 bundle 的影响) |false| |maxNumPartitionsPerPartitionedTopic| 每个分区主题的最大分区数。 使用 0 或负数来禁用检查 |0| |tlsEnabled| 启用 TLS |false| |tlsCertificateFilePath| TLS 证书文件路径 || |tlsKeyFilePath| TLS 私钥文件路径 || |tlsTrustCertsFilePath| 信任的 TLS 证书文件路径 || |tlsAllowInsecureConnection| 接受来自客户端的不信任 TLS 证书 |false| |tlsProtocols| 指定 broker 在 TLS 握手时使用的协议。 Multiple values can be specified, separated by commas. Example:-TLSv1.2
,TLSv1.1
,TLSv1
|| |tlsCiphers|Specify the tls cipher the broker will use to negotiate during TLS Handshake. Multiple values can be specified, separated by commas. Example:-TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
|| |tlsEnabledWithKeyStore| Enable TLS with KeyStore type configuration in broker |false| |tlsProvider| TLS Provider for KeyStore type || |tlsKeyStoreType| LS KeyStore type configuration in broker: JKS, PKCS12 |JKS| |tlsKeyStore| TLS KeyStore path in broker || |tlsKeyStorePassword| TLS KeyStore password for broker || |brokerClientTlsEnabledWithKeyStore| Whether internal client use KeyStore type to authenticate with Pulsar brokers |false| |brokerClientSslProvider| The TLS Provider used by internal client to authenticate with other Pulsar brokers || |brokerClientTlsTrustStoreType| TLS TrustStore type configuration for internal client: JKS, PKCS12, used by the internal client to authenticate with Pulsar brokers |JKS| |brokerClientTlsTrustStore| TLS TrustStore path for internal client, used by the internal client to authenticate with Pulsar brokers || |brokerClientTlsTrustStorePassword| TLS TrustStore password for internal client, used by the internal client to authenticate with Pulsar brokers || |brokerClientTlsCiphers| Specify the tls cipher the internal client will use to negotiate during TLS Handshake. (一个逗号分隔的密码列表) 例如:[TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256]|| |brokerClientTlsProtocols| 指定 TLS 握手期间 broker 使用的 tls 协议。 (逗号分隔的协议名称列表)。 例如:[TLSv1.2, TLSv1.1, TLSv1] || |ttlDurationDefaultInSeconds| 命名空间的默认 ttl(当命名空间策略中未配置 ttl 时)。 |0| |tokenSecretKey| 配置用于验证认证令牌的密钥。 The key can be specified like:tokenSecretKey=data:;base64,xxxxxxxxx
ortokenSecretKey=file:///my/secret.key
|| |tokenPublicKey| Configure the public key to be used to validate auth tokens. The key can be specified like:tokenPublicKey=data:;base64,xxxxxxxxx
ortokenPublicKey=file:///my/secret.key
|| |tokenPublicAlg| Configure the algorithm to be used to validate auth tokens. 这可以是Java JWT 支持的任意一种非对称算法 (https://github.com/jwtk/jjwt#signature-algorithms-keys) |RS256| |tokenAuthClaim| Specify which of the token’s claims will be used as the authentication “principal” or “role”. 如果这里留空,则默认的”sub” claim 会被使用 || |tokenAudienceClaim| 令牌受众的 “claim” 名称,比如”aud”会用于从令牌获取受众。 If not set, audience will not be verified. || |tokenAudience| 该 broker 的令牌受众。 The fieldtokenAudienceClaim
of a valid token, need contains this. || |maxUnackedMessagesPerConsumer| 消费者在共享订阅上接收消息时允许的未确认消息最大数量。 Broker will stop sending messages to consumer once, this limit reaches until consumer starts acknowledging messages back. 使用 0 值,禁用 unackeMessage 限制检查,消费者可以不受任何限制地接收消息 |50000| |maxUnackedMessagesPersubscription| 每个共享订阅允许的未确认消息的最大数量。 Broker will stop dispatching messages to all consumers of the subscription once this limit reaches until consumer starts acknowledging messages back and unack count reaches to limit/2. 使用 0 值,禁用 unackedMessage-limit 检查且调度器可以不受任何限制地发送消息 |200000| |subscriptionRedeliveryTrackerEnabled| 启用订阅消息重送跟踪 |true| subscriptionExpirationTimeMinutes | 删除上次消费后非活动订阅的时间。Setting this configuration to a value greater than 0 deletes inactive subscriptions automatically.
Setting this configuration to 0 does not delete inactive subscriptions automatically.Since this configuration takes effect on all topics, if there is even one topic whose subscriptions should not be deleted automatically, you need to set it to 0.
Instead, you can set a subscription expiration time for each namespace using thepulsar-admin namespaces set-subscription-expiration-time options
command. | 0 | |maxConcurrentLookupRequest| Max number of concurrent lookup request broker allows to throttle heavy incoming lookup traffic |50000| |maxConcurrentTopicLoadRequest| Max number of concurrent topic loading request broker allows to control number of zk-operations |5000| |authenticationEnabled| Enable authentication |false| |authenticationProviders| Autentication provider name list, which is comma separated list of class names || |authorizationEnabled| Enforce authorization |false| |superUserRoles| Role names that are treated as “super-user”, meaning they will be able to do all admin operations and publish/consume from all topics || |brokerClientAuthenticationPlugin| Authentication settings of the broker itself. 当 broker 连接到其它 broker 时使用,不管是相同集群还是其它集群中的 || |brokerClientAuthenticationParameters||| |athenzDomainNames| 用于认证的受支持的 Athenz 提供者域名(逗号分隔) || |exposePreciseBacklogInPrometheus| 暴露出精确的积压统计信息,设置为 false 来使用 published counter 和 consumed counter 来进行计算,这样更高效但可能会有误差。 |false| |bookkeeperMetadataServiceUri| bookkeeper 用来载入相应元数据驱动及解析其元数据服务位置的元数据服务 uri。 This value can be fetched usingbookkeeper shell whatisinstanceid
command in BookKeeper cluster. 例如: zk+hierarchical://localhost:2181/ledgers。 元数据服务 uri 列表也可以是一个分号间隔的值,如下所示: zk+hierarchical://zk1:2181;zk2:2181;zk3:2181/ledgers || |bookkeeperClientAuthenticationPlugin| 连接到 bookies 时使用的认证插件 || |bookkeeperClientAuthenticationParametersName| BookKeeper 认证插件的实现指定的名字与值 || |bookkeeperClientAuthenticationParameters||| |bookkeeperClientTimeoutInSeconds| BookKeeper 添加与读取操作的超时时间 |30| |bookkeeperClientSpeculativeReadTimeoutInMillis| 当一个读请求无法在一定时间内完成时启用 speculative 读取。采用 0 值时,禁用 speculative 读取。 |0| |bookkeeperClientHealthCheckEnabled| 启用 bookie 健康检查。 Bookies that have more than the configured number of failure within the interval will be quarantined for some time. 在此期间,这些 bookies 上不会创建新的 ledger |true| |bookkeeperClientHealthCheckIntervalSeconds||60| |bookkeeperClientHealthCheckErrorThresholdPerInterval||5| |bookkeeperClientHealthCheckQuarantineTimeInSeconds ||1800| |bookkeeperClientRackawarePolicyEnabled| 启用 rack-aware bookie 选择政策。 当生成新的 bookie ensemble 时 BK 会从不同的 rack 中选取 bookie |true| |bookkeeperClientRegionawarePolicyEnabled| 启用 region-aware bookie 选择政策。 BK will chose bookies from different regions and racks when forming a new bookie ensemble. 如果启用,bookkeeperClientRackawarePolicyEnabled 属性的值会被忽略 |false| |bookkeeperClientReorderReadSequenceEnabled| 启用/禁用读取条目的读取序列重排序。 |false| |bookkeeperClientIsolationGroups| 通过指定一个要选择的 bookie 组列表来启用 bookie 隔离。 指定组外的任何 bookie 都不会被 broker 使用 || |bookkeeperClientSecondaryIsolationGroups| 当 bookkeeperClientIsolationGroups 没有足够多的可用 bookie 时启用 bookie 的 secondary-isolation 组。 || |bookkeeperClientMinAvailableBookiesInIsolationGroups| bookkeeperClientIsolationGroups 中应可用的最少 bookie 数,否则 broker 会将 bookkeeperClientSecondaryIsolationGroups 的 bookie 囊括在隔离列表中。 || |bookkeeperClientGetBookieInfoIntervalSeconds| 设置定期查询 bookie 信息的间隔 |86400| |bookkeeperClientGetBookieInfoRetryIntervalSeconds| 设置当查询 bookie 信息失败后的重试间隔 |60| |bookkeeperEnableStickyReads | 启用/禁用 使对一个 ledger 的读请求变得与单个 broker 粘滞(sticky)。 If this flag is enabled, the client will use one single bookie (by preference) to read all entries for a ledger. | true | |managedLedgerDefaultEnsembleSize| 当创建一个 ledger 时使用的 bookie 数量 |2| |managedLedgerDefaultWriteQuorum| 每个消息存储的拷贝数量 |2| |managedLedgerDefaultAckQuorum| 得到保证的拷贝数量 (写操作完成前需要等待的 ack) |2| |managedLedgerCacheSizeMB| 在托管的 ledger 中用于缓存数据负载的内存大小。 This memory is allocated from JVM direct memory and it’s shared across all the topics running in the same broker. 默认地,使用可用的直接内存的 1/5 || |managedLedgerCacheCopyEntries| 当插入缓存时是否应该创建条目负载的拷贝 | false| |managedLedgerCacheEvictionWatermark| 当驱逐触发时降低缓存等级的阈值 |0.9| |managedLedgerCacheEvictionFrequency| 为托管的 ledger 缓存配置缓存驱逐频率 (驱逐/秒) | 100.0 | |managedLedgerCacheEvictionTimeThresholdMillis| 所有在缓存中停留超过配置时长的条目会被驱逐 | 1000 | |managedLedgerCursorBackloggedThreshold| 设置当游标应被当作“积压的” 且应被设置为非活动的阈值 (以条目数为单位)。 | 1000| |managedLedgerDefaultMarkDeleteRateLimit| 限制由消费者确认消息产生的每秒写入频率 |1.0| |managedLedgerMaxEntriesPerLedger| 在触发倒转前追加至 ledger 的最大条目数。 A ledger rollover is triggered on these conditions:- Either the max rollover time has been reached
- or max entries have been written to the ledged and at least min-time has passed
|50000| |managedLedgerMinLedgerRolloverTimeMinutes| Minimum time between ledger rollover for a topic |10| |managedLedgerMaxLedgerRolloverTimeMinutes| Maximum time before forcing a ledger rollover for a topic |240| |managedLedgerCursorMaxEntriesPerLedger| Max number of entries to append to a cursor ledger |50000| |managedLedgerCursorRolloverTimeInSeconds| Max time before triggering a rollover on a cursor ledger |14400| |managedLedgerMaxUnackedRangesToPersist| Max number of “acknowledgment holes” that are going to be persistently stored. When acknowledging out of order, a consumer will leave holes that are supposed to be quickly filled by acking all the messages. The information of which messages are acknowledged is persisted by compressing in “ranges” of messages that were acknowledged. After the max number of ranges is reached, the information will only be tracked in memory and messages will be redelivered in case of crashes. |1000| |autoSkipNonRecoverableData| Skip reading non-recoverable/unreadable data-ledger under managed-ledger’s list.It helps when data-ledgers gets corrupted at bookkeeper and managed-cursor is stuck at that ledger. |false| |loadBalancerEnabled| Enable load balancer |true| |loadBalancerPlacementStrategy| Strategy to assign a new bundle weightedRandomSelection || |loadBalancerReportUpdateThresholdPercentage| Percentage of change to trigger load report update |10| |loadBalancerReportUpdateMaxIntervalMinutes| maximum interval to update load report |15| |loadBalancerHostUsageCheckIntervalMinutes| Frequency of report to collect |1| |loadBalancerSheddingIntervalMinutes| Load shedding interval. Broker periodically checks whether some traffic should be offload from some over-loaded broker to other under-loaded brokers |30| |loadBalancerSheddingGracePeriodMinutes| Prevent the same topics to be shed and moved to other broker more that once within this timeframe |30| |loadBalancerBrokerMaxTopics| Usage threshold to allocate max number of topics to broker |50000| |loadBalancerBrokerUnderloadedThresholdPercentage| Usage threshold to determine a broker as under-loaded |1| |loadBalancerBrokerOverloadedThresholdPercentage| Usage threshold to determine a broker as over-loaded |85| |loadBalancerResourceQuotaUpdateIntervalMinutes| Interval to update namespace bundle resource quotat |15| |loadBalancerBrokerComfortLoadLevelPercentage| Usage threshold to determine a broker is having just right level of load |65| |loadBalancerAutoBundleSplitEnabled| enable/disable namespace bundle auto split |false| |loadBalancerNamespaceBundleMaxTopics| maximum topics in a bundle, otherwise bundle split will be triggered |1000| |loadBalancerNamespaceBundleMaxSessions| maximum sessions (producers + consumers) in a bundle, otherwise bundle split will be triggered |1000| |loadBalancerNamespaceBundleMaxMsgRate| maximum msgRate (in + out) in a bundle, otherwise bundle split will be triggered |1000| |loadBalancerNamespaceBundleMaxBandwidthMbytes| maximum bandwidth (in + out) in a bundle, otherwise bundle split will be triggered |100| |loadBalancerNamespaceMaximumBundles| maximum number of bundles in a namespace |128| |replicationMetricsEnabled| Enable replication metrics |true| |replicationConnectionsPerBroker| Max number of connections to open for each broker in a remote cluster More connections host-to-host lead to better throughput over high-latency links. |16| |replicationProducerQueueSize| Replicator producer queue size |1000| |replicatorPrefix| Replicator prefix used for replicator producer name and cursor name pulsar.repl|| |replicationTlsEnabled| Enable TLS when talking with other clusters to replicate messages |false| |defaultRetentionTimeInMinutes| Default message retention time || |defaultRetentionSizeInMB| Default retention size |0| |keepAliveIntervalSeconds| How often to check whether the connections are still alive |30| |loadManagerClassName| Name of load manager to use |org.apache.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl| |supportedNamespaceBundleSplitAlgorithms| Supported algorithms name for namespace bundle split |[range_equally_divide,topic_count_equally_divide]| |defaultNamespaceBundleSplitAlgorithm| Default algorithm name for namespace bundle split |range_equally_divide| |managedLedgerOffloadDriver| Driver to use to offload old data to long term storage (Possible values: S3) || |managedLedgerOffloadMaxThreads| Maximum number of thread pool threads for ledger offloading |2| |managedLedgerUnackedRangesOpenCacheSetEnabled| Use Open Range-Set to cache unacknowledged messages |true| |managedLedgerOffloadDeletionLagMs|Delay between a ledger being successfully offloaded to long term storage and the ledger being deleted from bookkeeper | 14400000| |managedLedgerOffloadAutoTriggerSizeThresholdBytes|The number of bytes before triggering automatic offload to long term storage |-1 (disabled)| |s3ManagedLedgerOffloadRegion| For Amazon S3 ledger offload, AWS region || |s3ManagedLedgerOffloadBucket| For Amazon S3 ledger offload, Bucket to place offloaded ledger into || |s3ManagedLedgerOffloadServiceEndpoint| For Amazon S3 ledger offload, Alternative endpoint to connect to (useful for testing) || |s3ManagedLedgerOffloadMaxBlockSizeInBytes| For Amazon S3 ledger offload, Max block size in bytes. (默认为 64MB,最小 5MB) |67108864| |s3ManagedLedgerOffloadReadBufferSizeInBytes| 对于 Amazon S3 ledger 卸载,以字节计的读缓存大小 (默认为 1MB) |1048576| |s3ManagedLedgerOffloadRole| 对于 Amazon S3 ledger 卸载,在写入 s3 前提供一个假定角色 || |s3ManagedLedgerOffloadRoleSessionName| 对于 Amazon S3 ledger 卸载,当使用一个角色时提供一个角色会话名 |pulsar-s3-offload| | acknowledgmentAtBatchIndexLevelEnabled | 启用或禁用批量下标确认。 | false | | maxMessageSize | 设置消息的最大大小。 | 5 MB | | preciseTopicPublishRateLimiterEnable | 启用精确主题发布率限制。 | false |
Client
The
pulsar-client
CLI tool can be used to publish messages to Pulsar and consume messages from Pulsar topics. This tool can be used in lieu of a client library.Name Description 默认值 webServiceUrl 群集的 web URL。 http://localhost:8080/ brokerServiceUrl 集群的Pulsar 协议地址。 pulsar://localhost:6650/ authPlugin 身份认证插件。 authParams 群集的身份认证参数, 逗号分隔的字符串。 useTls 是否在群集中强制执行 TLS 验证。 false tlsAllowInsecureConnection tlsTrustCertsFilePath Service discovery
Name Description 默认值 zookeeperServers Zookeeper quorum connection string (comma-separated) zooKeeperCacheExpirySeconds ZooKeeper 缓存过期时间(秒) 300 configurationStoreServers 配置存储连接字符串(以逗号分隔的列表) zookeeperSessionTimeoutMs ZooKeeper session timeout 30000 servicePort Port to use to server binary-proto request 6650 servicePortTls Port to use to server binary-proto-tls request 6651 webServicePort Port that discovery service listen on 8080 webServicePortTls Port to use to server HTTPS request 8443 bindOnLocalhost Control whether to bind directly on localhost rather than on normal hostname false authenticationEnabled Enable authentication false authenticationProviders Authentication provider name list, which is comma separated list of class names (comma-separated) authorizationEnabled Enforce authorization false superUserRoles Role names that are treated as “super-user”, meaning they will be able to do all admin operations and publish/consume from all topics (comma-separated) tlsEnabled Enable TLS false tlsCertificateFilePath TLS证书文件的路径 tlsKeyFilePath TLS私钥文件的路径 Log4j
Name 默认值 pulsar.root.logger WARN,CONSOLE pulsar.log.dir logs pulsar.log.file pulsar.log log4j.rootLogger ${pulsar.root.logger} log4j.appender.CONSOLE org.apache.log4j.ConsoleAppender log4j.appender.CONSOLE.Threshold DEBUG log4j.appender.CONSOLE.layout org.apache.log4j.PatternLayout log4j.appender.CONSOLE.layout.ConversionPattern %d{ISO8601} - %-5p - [%t:%C{1}@%L] - %m%n log4j.appender.ROLLINGFILE org.apache.log4j.DailyRollingFileAppender log4j.appender.ROLLINGFILE.Threshold DEBUG log4j.appender.ROLLINGFILE.File ${pulsar.log.dir}/${pulsar.log.file} log4j.appender.ROLLINGFILE.layout org.apache.log4j.PatternLayout log4j.appender.ROLLINGFILE.layout.ConversionPattern %d{ISO8601} - %-5p [%t:%C{1}@%L] - %m%n log4j.appender.TRACEFILE org.apache.log4j.FileAppender log4j.appender.TRACEFILE.Threshold TRACE log4j.appender.TRACEFILE.File pulsar-trace.log log4j.appender.TRACEFILE.layout org.apache.log4j.PatternLayout log4j.appender.TRACEFILE.layout.ConversionPattern %d{ISO8601} - %-5p [%t:%C{1}@%L][%x] - %m%n Log4j shell
Name 默认值 bookkeeper.root.logger ERROR,CONSOLE log4j.rootLogger ${bookkeeper.root.logger} log4j.appender.CONSOLE org.apache.log4j.ConsoleAppender log4j.appender.CONSOLE.Threshold DEBUG log4j.appender.CONSOLE.layout org.apache.log4j.PatternLayout log4j.appender.CONSOLE.layout.ConversionPattern %d{ABSOLUTE} %-5p %m%n log4j.logger.org.apache.zookeeper ERROR log4j.logger.org.apache.bookkeeper ERROR log4j.logger.org.apache.bookkeeper.bookie.BookieShell INFO Standalone
Name Description 默认值 zookeeperServers The quorum connection string for local ZooKeeper zooKeeperCacheExpirySeconds ZooKeeper 缓存过期时间(秒) 300 configurationStoreServers 配置存储连接字符串(以逗号分隔的列表) brokerServicePort The port on which the standalone broker listens for connections 6650 webServicePort THe port used by the standalone broker for HTTP requests 8080 bindAddress The hostname or IP address on which the standalone service binds 0.0.0.0 advertisedAddress The hostname or IP address that the standalone service advertises to the outside world. If not set, the value of InetAddress.getLocalHost().getHostName()
is used.clusterName The name of the cluster that this broker belongs to. standalone zooKeeperSessionTimeoutMillis The ZooKeeper session timeout, in milliseconds. 30000 brokerShutdownTimeoutMs The time to wait for graceful broker shutdown. After this time elapses, the process will be killed. 60000 skipBrokerShutdownOnOOM Flag to skip broker shutdown when broker handles Out of memory error. false backlogQuotaCheckEnabled Enable the backlog quota check, which enforces a specified action when the quota is reached. true backlogQuotaCheckIntervalInSeconds How often to check for topics that have reached the backlog quota. 60 backlogQuotaDefaultLimitGB The default per-topic backlog quota limit. 10 ttlDurationDefaultInSeconds The default ttl for namespaces if ttl is not configured at namespace policies. 0 brokerDeleteInactiveTopicsEnabled Enable the deletion of inactive topics. true brokerDeleteInactiveTopicsFrequencySeconds How often to check for inactive topics, in seconds. 60 messageExpiryCheckIntervalInMinutes How often to proactively check and purged expired messages. 5 activeConsumerFailoverDelayTimeMillis How long to delay rewinding cursor and dispatching messages when active consumer is changed. 1000 clientLibraryVersionCheckEnabled Enable checks for minimum allowed client library version. false clientLibraryVersionCheckAllowUnversioned Allow client libraries with no version information true statusFilePath The path for the file used to determine the rotation status for the broker when responding to service discovery health checks /usr/local/apache/htdocs maxUnackedMessagesPerConsumer The maximum number of unacknowledged messages allowed to be received by consumers on a shared subscription. The broker will stop sending messages to a consumer once this limit is reached or until the consumer begins acknowledging messages. A value of 0 disables the unacked message limit check and thus allows consumers to receive messages without any restrictions. 50000 maxUnackedMessagesPerSubscription The same as above, except per subscription rather than per consumer. 200000 maxNumPartitionsPerPartitionedTopic Max number of partitions per partitioned topic. Use 0 or negative number to disable the check 0 authenticationEnabled Enable authentication for the broker. false authenticationProviders A comma-separated list of class names for authentication providers. false authorizationEnabled Enforce authorization in brokers. false superUserRoles Role names that are treated as “superusers.” Superusers are authorized to perform all admin tasks. brokerClientAuthenticationPlugin The authentication settings of the broker itself. Used when the broker connects to other brokers either in the same cluster or from other clusters. brokerClientAuthenticationParameters The parameters that go along with the plugin specified using brokerClientAuthenticationPlugin. athenzDomainNames Supported Athenz authentication provider domain names as a comma-separated list. exposePreciseBacklogInPrometheus Enable expose the precise backlog stats, set false to use published counter and consumed counter to calculate, this would be more efficient but may be inaccurate. false bookkeeperClientAuthenticationPlugin Authentication plugin to be used when connecting to bookies (BookKeeper servers). bookkeeperClientAuthenticationParametersName BookKeeper authentication plugin implementation parameters and values. bookkeeperClientAuthenticationParameters Parameters associated with the bookkeeperClientAuthenticationParametersName bookkeeperClientTimeoutInSeconds Timeout for BookKeeper add and read operations. 30 bookkeeperClientSpeculativeReadTimeoutInMillis Speculative reads are initiated if a read request doesn’t complete within a certain time. A value of 0 disables speculative reads. 0 bookkeeperClientHealthCheckEnabled Enable bookie health checks. true bookkeeperClientHealthCheckIntervalSeconds The time interval, in seconds, at which health checks are performed. New ledgers are not created during health checks. 60 bookkeeperClientHealthCheckErrorThresholdPerInterval Error threshold for health checks. 5 bookkeeperClientHealthCheckQuarantineTimeInSeconds If bookies have more than the allowed number of failures within the time interval specified by bookkeeperClientHealthCheckIntervalSeconds 1800 bookkeeperClientRackawarePolicyEnabled true bookkeeperClientRegionawarePolicyEnabled false bookkeeperClientReorderReadSequenceEnabled false bookkeeperClientIsolationGroups bookkeeperClientSecondaryIsolationGroups Enable bookie secondary-isolation group if bookkeeperClientIsolationGroups doesn’t have enough bookie available. bookkeeperClientMinAvailableBookiesInIsolationGroups Minimum bookies that should be available as part of bookkeeperClientIsolationGroups else broker will include bookkeeperClientSecondaryIsolationGroups bookies in isolated list. managedLedgerDefaultEnsembleSize 1 managedLedgerDefaultWriteQuorum 1 managedLedgerDefaultAckQuorum 1 managedLedgerCacheSizeMB 1024 managedLedgerCacheCopyEntries Whether we should make a copy of the entry payloads when inserting in cache false managedLedgerCacheEvictionWatermark 0.9 managedLedgerCacheEvictionFrequency Configure the cache eviction frequency for the managed ledger cache (evictions/sec) 100.0 managedLedgerCacheEvictionTimeThresholdMillis All entries that have stayed in cache for more than the configured time, will be evicted 1000 managedLedgerCursorBackloggedThreshold Configure the threshold (in number of entries) from where a cursor should be considered ‘backlogged’ and thus should be set as inactive. 1000 managedLedgerUnackedRangesOpenCacheSetEnabled Use Open Range-Set to cache unacknowledged messages true managedLedgerDefaultMarkDeleteRateLimit 0.1 managedLedgerMaxEntriesPerLedger 50000 managedLedgerMinLedgerRolloverTimeMinutes 10 managedLedgerMaxLedgerRolloverTimeMinutes 240 managedLedgerCursorMaxEntriesPerLedger 50000 managedLedgerCursorRolloverTimeInSeconds 14400 autoSkipNonRecoverableData false loadBalancerEnabled false loadBalancerPlacementStrategy weightedRandomSelection loadBalancerReportUpdateThresholdPercentage 10 loadBalancerReportUpdateMaxIntervalMinutes 15 loadBalancerHostUsageCheckIntervalMinutes 1 loadBalancerSheddingIntervalMinutes 30 loadBalancerSheddingGracePeriodMinutes 30 loadBalancerBrokerMaxTopics 50000 loadBalancerBrokerUnderloadedThresholdPercentage 1 loadBalancerBrokerOverloadedThresholdPercentage 85 loadBalancerResourceQuotaUpdateIntervalMinutes 15 loadBalancerBrokerComfortLoadLevelPercentage 65 loadBalancerAutoBundleSplitEnabled false loadBalancerNamespaceBundleMaxTopics 1000 loadBalancerNamespaceBundleMaxSessions 1000 loadBalancerNamespaceBundleMaxMsgRate 1000 loadBalancerNamespaceBundleMaxBandwidthMbytes 100 loadBalancerNamespaceMaximumBundles 128 replicationMetricsEnabled true replicationConnectionsPerBroker 16 replicationProducerQueueSize 1000 defaultRetentionTimeInMinutes 0 defaultRetentionSizeInMB 0 keepAliveIntervalSeconds 30 WebSocket
Name Description 默认值 configurationStoreServers zooKeeperSessionTimeoutMillis 30000 zooKeeperCacheExpirySeconds ZooKeeper 缓存过期时间(秒) 300 serviceUrl serviceUrlTls brokerServiceUrl brokerServiceUrlTls webServicePort 8080 webServicePortTls 8443 bindAddress 0.0.0.0 clusterName authenticationEnabled false authenticationProviders authorizationEnabled false superUserRoles brokerClientAuthenticationPlugin brokerClientAuthenticationParameters tlsEnabled false tlsAllowInsecureConnection false tlsCertificateFilePath tlsKeyFilePath tlsTrustCertsFilePath Pulsar proxy
The Pulsar proxy can be configured in the
conf/proxy.conf
file.Name Description 默认值 zookeeperServers ZooKeeper quorum 连接字符串(以逗号分隔的列表) configurationStoreServers 配置存储连接字符串(以逗号分隔的列表) zookeeperSessionTimeoutMs ZooKeeper会话超时(以毫秒为单位) 30000 zooKeeperCacheExpirySeconds ZooKeeper 缓存过期时间(秒) 300 servicePort 用于服务器二进制Protobuf请求的端口 6650 servicePortTls 用于服务器二进制Protobuf TLS请求的端口 6651 statusFilePath 在响应服务发现健康检查时,用于确定代理实例的轮换状态的文件的路径 advertisedAddress Hostname or IP address the service advertises to the outside world. InetAddress.getLocalHost().getHostname()
authenticationEnabled 是否为Pulsar代理启用身份验证 false authenticateMetricsEndpoint Whether the ‘/metrics’ endpoint requires authentication. Defaults to true. ‘authenticationEnabled’ must also be set for this to take effect. true authenticationProviders 身份验证提供者名称列表(以逗号分隔的类名列表) authorizationEnabled 是否由Pulsar代理强制执行授权 false authorizationProvider 授权提供程序的完全限定类名 org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider brokerClientAuthenticationPlugin Pulsar代理使用的身份验证插件,用于对Pulsar brokers进行身份验证 brokerClientAuthenticationParameters Pulsar代理用于对Pulsar Brokers进行身份验证的参数 brokerClientTrustCertsFilePath Pulsar代理用于对Pulsar Brokers进行身份验证的可信证书的路径 superUserRoles “超级用户”的角色名,这意味着它们将能够执行所有管理 forwardAuthorizationCredentials Whether client authorization credentials are forwared to the broker for re-authorization. Authentication must be enabled via authenticationEnabled=true for this to take effect. false maxConcurrentInboundConnections Max concurrent inbound connections. The proxy will reject requests beyond that. 10000 maxConcurrentLookupRequests Max concurrent outbound connections. The proxy will error out requests beyond that. 50000 tlsEnabledInProxy 是否为代理启用TLS false tlsEnabledWithBroker 与Pulsar Brokers通信时是否启用TLS false tlsCertificateFilePath TLS证书文件的路径 tlsKeyFilePath TLS私钥文件的路径 tlsTrustCertsFilePath 受信任的TLS证书pem文件的路径 tlsHostnameVerificationEnabled 当代理与brokers建立TLS连接时是否验证主机名 false tlsRequireTrustedClientCertOnConnect Whether client certificates are required for TLS. Connections are rejected if the client certificate isn’t trusted. false tlsProtocols Specify the tls protocols the broker will use to negotiate during TLS Handshake. Multiple values can be specified, separated by commas. Example:- TLSv1.2
,TLSv1.1
,TLSv1
tlsCiphers Specify the tls cipher the broker will use to negotiate during TLS Handshake. Multiple values can be specified, separated by commas. Example:- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
tokenSecretKey Configure the secret key to be used to validate auth tokens. The key can be specified like: tokenSecretKey=data:;base64,xxxxxxxxx
ortokenSecretKey=file:///my/secret.key
tokenPublicKey Configure the public key to be used to validate auth tokens. The key can be specified like: tokenPublicKey=data:;base64,xxxxxxxxx
ortokenPublicKey=file:///my/secret.key
tokenPublicAlg Configure the algorithm to be used to validate auth tokens. This can be any of the asymettric algorithms supported by Java JWT (https://github.com/jwtk/jjwt#signature-algorithms-keys) RS256 tokenAuthClaim Specify the token claim that will be used as the authentication “principal” or “role”. The “subject” field will be used if this is left blank proxyLogLevel Set the Pulsar Proxy log level. - If the value is set to 0, no TCP channel information is logged.
- If the value is set to 1, only the TCP channel information and command information (without message body) are parsed and logged.
- If the value is set to 2, all TCP channel information, command information, and message body are parsed and logged.
0 ZooKeeper
ZooKeeper handles a broad range of essential configuration- and coordination-related tasks for Pulsar. The default configuration file for ZooKeeper is in the
conf/zookeeper.conf
file in your Pulsar installation. The following parameters are available:Name Description 默认值 tickTime The tick is the basic unit of time in ZooKeeper, measured in milliseconds and used to regulate things like heartbeats and timeouts. tickTime is the length of a single tick. 2000 initLimit The maximum time, in ticks, that the leader ZooKeeper server allows follower ZooKeeper servers to successfully connect and sync. The tick time is set in milliseconds using the tickTime parameter. 10 syncLimit The maximum time, in ticks, that a follower ZooKeeper server is allowed to sync with other ZooKeeper servers. The tick time is set in milliseconds using the tickTime parameter. 5 dataDir The location where ZooKeeper will store in-memory database snapshots as well as the transaction log of updates to the database. data/zookeeper clientPort The port on which the ZooKeeper server will listen for connections. 2181 autopurge.snapRetainCount In ZooKeeper, auto purge determines how many recent snapshots of the database stored in dataDir to retain within the time interval specified by autopurge.purgeInterval (while deleting the rest). 3 autopurge.purgeInterval The time interval, in hours, by which the ZooKeeper database purge task is triggered. Setting to a non-zero number will enable auto purge; setting to 0 will disable. Read this guide before enabling auto purge. 1 maxClientCnxns The maximum number of client connections. Increase this if you need to handle more ZooKeeper clients. 60 In addition to the parameters in the table above, configuring ZooKeeper for Pulsar involves adding a
server.N
line to theconf/zookeeper.conf
file for each node in the ZooKeeper cluster, whereN
is the number of the ZooKeeper node. Here’s an example for a three-node ZooKeeper cluster:server.1=zk1.us-west.example.com:2888:3888
server.2=zk2.us-west.example.com:2888:3888
server.3=zk3.us-west.example.com:2888:3888
We strongly recommend consulting the ZooKeeper Administrator’s Guide for a more thorough and comprehensive introduction to ZooKeeper configuration