Migration
Pulsar has 3 types of broker load balancers, that is, simple, modular, and extensible.
You can perform the following migrations.
Migration | When to use |
---|---|
Migrate from simple to modular | If you want to use the bundle unloading strategy of OverloadShedder, ThresholdShedder, or UniformLoadShedder. |
Migrate from modular to extensible | If you want to use the bundle unloading strategy of TransferShedder. |
Migrate from extensible to modular | If you want to use the bundle unloading strategy of OverloadShedder, ThresholdShedder, or UniformLoadShedder. |
note
It is not recommended to migrate from the modular or extensible to the simple broker load balancer since the simple broker load balancer is deprecated and no longer in use.
Considerations
Before migrating from one broker load balancer type to another, review the relationship between broker load balancer type, Pulsar version, and bundle unloading strategy. You may need to upgrade Pulsar versions or update the bundle unloading strategy. Below are brief summaries.
This broker load balancer type | is available in this Pulsar version |
---|---|
Simple | All versions |
Modular | 1.7 and later |
Extensible | 3.0 and later |
This bundle unloading strategy | is available for this broker load balancer type | in available this Pulsar version |
---|---|---|
OverloadShedder | Modular | 1.18 and later |
ThresholdShedder | Modular | 2.6 and later |
UniformLoadShedder | Modular | 2.10 and later |
TransferShedder | Extensible | 3.0 and later |
Migrate from simple to modular broker load balancer
You can migrate from the simple to the modular broker load balancer, by manually changing the configuration settings in the broker.conf file or by using the pulsar-admin tool.
Change broker.conf file
Access to the broker.conf file.
vim apache-pulsar-3.3.2/conf/broker.conf
Change the broker load balancer by setting loadManagerClassName to
ModularLoadManagerImpl
in the broker.conf file.loadManagerClassName=org.apache.pulsar.broker.loadbalance.extensions.ModularLoadManagerImpl
Restart the Pulsar cluster. The new setting will take effect after the restart.
Use pulsar-admin tool
Access the pulsar-admin tool.
cd apache-pulsar-3.3.2/bin
Set
--config
toloadManagerClassName
and--value
toorg.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl
.pulsar-admin brokers update-dynamic-config \
--config loadManagerClassName \
--value org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl
You do not need to restart the Pulsar cluster. The new settings will take effect after 1 to 2 minutes.
Migrate from modular to extensible broker load balancer
You can migrate from the modular to the extensible broker load balancer, by manually changing settings in the broker.conf file. During the migration, the lookup and assignment will be redirected to the brokers with the extensible load balancer.
note
The pulsar-admin tool is not supported for this migration.
Change broker.conf file
Upgrade the Pulsar cluster to 3.0.0 or later versions.
Access to the broker.conf file.
vim apache-pulsar-3.3.2/conf/broker.conf
Change the following settings in the broker.conf file:
Update broker load balancer by setting loadManagerClassName to
ExtensibleLoadManagerImpl
.Update bundle unloading strategy by setting
loadBalancerLoadSheddingStrategy
toTransferShedder
.
loadManagerClassName=org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImpl
loadBalancerLoadSheddingStrategy=org.apache.pulsar.broker.loadbalance.extensions.scheduler.TransferShedder
note
- The extensible broker load balancer is available in Pulsar 3.0.0 or later.
- The TransferShedder unloading strategy is only available in the extensible load balancer.
Restart the Pulsar cluster. The new settings will take effect after the restart.
Migrate from extensible to modular broker load balancer
You can migrate from the extensible to the modular broker load balancer, by manually changing the setting in the broker.conf file. During the migration, the lookup and assignment will be redirected to the brokers with the modular load balancer.
note
The pulsar-admin tool is not supported for this migration.
Change broker.conf file
Access to the broker.conf file.
vim apache-pulsar-3.3.2/conf/broker.conf
Change the following settings in the broker.conf file:
Update broker load balancer by setting loadManagerClassName to ModularLoadManagerImpl
Update bundle unloading strategy to OverloadShedder, ThresholdShedder, or UniformLoadShedder based on your needs.
loadManagerClassName=org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl
loadBalancerLoadSheddingStrategy=org.apache.pulsar.broker.loadbalance.impl.ThresholdShedder
note
TransferShedder is only supported in the extensible broker load balancer, so you need to change TransferShedder to other bundle unloading strategies.
Restart the Pulsar cluster. The new settings will take effect after the restart.
Related topics
To get a comprehensive understanding and discover the key insights, see Broker load balancing | Overview.
To discover different usage scenarios, see Broker load balancing | Use cases.
To explore functionalities, see Broker load balancing | Features.
To understand advantages, see Broker load balancing | Benefits.
To learn essential fundamentals, see Broker load balancing | Concepts.
To review various versions of broker load balancers, see Broker load balancing | Types.