Consumer Thread Model, Provider Thread Model

Usage of Dubbo consumer thread pool model

Consumer Thread Model

For Dubbo applications prior to version 2.7.5, especially some consumer applications, when faced with consuming a large number of services with a high concurrency scenario (typical in gateway scenarios), there often arises the issue of excessive thread allocation on the consumer side. For specific discussions, refer to Need a limited Threadpool in consumer side #2013.

The improved consumer thread pool model effectively addresses this issue by reusing the blocked business threads on the service side.

Old Thread Pool Model

Consumer Thread Pool.png

We focus on the Consumer part:

  1. Business threads issue a request and obtain a Future instance.
  2. The business thread immediately calls future.get to block while waiting for the business result to return.
  3. Once the business data returns, it is processed by a dedicated Consumer thread pool for deserialization, and future.set is called to place the deserialized business result back.
  4. The business thread directly returns the result.

Current Thread Pool Model

New Consumer Thread Pool.png

  1. Business threads issue a request and obtain a Future instance.
  2. Before calling future.get(), it first calls ThreadlessExecutor.wait(), which makes the business thread wait in a blocking queue until an element is added to the queue.
  3. When the business data returns, a Runnable Task is created and placed into the ThreadlessExecutor queue.
  4. The business thread retrieves the Task and executes it in its thread: deserializes the business data and sets it to Future.
  5. The business thread directly returns the result.

This way, compared to the old thread pool model, the business thread is responsible for monitoring and parsing return results, saving the overhead of an additional consumer thread pool.

Provider Thread Model

The current thread models for the Dubbo protocol and the Triple protocol have not yet been aligned. Below, the thread models for the Triple protocol and Dubbo protocol will be introduced separately.

Dubbo Protocol

Before introducing the Provider-side thread model of the Dubbo protocol, it is important to first explain that Dubbo abstracts channel operations into five behaviors:

  • Establish connection: connected, responsible for recording read and write times on the channel, as well as processing callback logic after establishing a connection.
  • Disconnect: disconnected, responsible for removing read and write times on the channel and processing callback logic after disconnecting.
  • Send message: sent, including sending requests and responses, and recording write times.
  • Receive message: received, including receiving requests and responses, and recording read times.
  • Exception capture: caught, used to handle various exceptions occurring on the channel.

The thread model of the Dubbo framework is closely related to these five behaviors, and the Provider thread model of the Dubbo protocol can be divided into five types: AllDispatcher, DirectDispatcher, MessageOnlyDispatcher, ExecutionDispatcher, ConnectionOrderedDispatcher.

Configuration Method

Thread ModelConfiguration Value
All Dispatcherall
Direct Dispatcherdirect
Execution Dispatcherexecution
Message Only Dispatchermessage
Connection Ordered Dispatcherconnection

Taking the configuration method in application.yaml as an example: configuring dispatcher: all under protocol can adjust the thread model of the Dubbo protocol to All Dispatcher

  1. dubbo:
  2.   application:
  3.     name: dubbo-springboot-demo-provider
  4.   protocol:
  5.     name: dubbo
  6.     port: -1
  7.     dispatcher: all
  8.   registry:
  9.     id: zk-registry
  10.     address: zookeeper://127.0.0.1:2181

All Dispatcher

The following figure illustrates the thread model of the All Dispatcher:

dubbo-provider-alldispatcher

  • Operations executed in IO threads:
    1. The sent operation is executed in the IO thread.
    2. The serialization response is executed in the IO thread.
  • Operations executed in Dubbo threads:
    1. Received, connected, disconnected, and caught are executed in the Dubbo thread.
    2. The behavior of deserializing requests is done in Dubbo.

Direct Dispatcher

The following figure illustrates the thread model of the Direct Dispatcher:

dubbo-provider-directDispatcher

  • Operations executed in IO threads:
    1. Received, connected, disconnected, caught, and sent operations are executed in the IO thread.
    2. The deserialization of requests and the serialization of responses occur in the IO thread.
  • There are no operations performed in the Dubbo thread.

Execution Dispatcher

The following figure illustrates the thread model of the Execution Dispatcher:

dubbo-provider-ExecutionDispatcher

  • Operations executed in IO threads:
    1. Sent, connected, disconnected, and caught operations are executed in the IO thread.
    2. The serialization response is executed in the IO thread.
  • Operations executed in Dubbo threads:
    1. Received is executed in the Dubbo thread.
    2. The behavior of deserializing requests is done in Dubbo.

Message Only Dispatcher

On the Provider side, the thread model of Message Only Dispatcher is consistent with that of Execution Dispatcher, so the following figure is the same as the Execution Dispatcher.

The following figure illustrates the thread model of the Message Only Dispatcher:

dubbo-provider-ExecutionDispatcher

  • Operations executed in IO threads:
    1. Sent, connected, disconnected, and caught operations are executed in the IO thread.
    2. The serialization response is executed in the IO thread.
  • Operations executed in Dubbo threads:
    1. Received is executed in the Dubbo thread.
    2. The behavior of deserializing requests is done in Dubbo.

Connection Ordered Dispatcher

The following figure illustrates the thread model of the Connection Ordered Dispatcher:

dubbbo-provider-connectionOrderedDispatcher

  • Operations executed in IO threads:
    1. The sent operation is executed in the IO thread.
    2. The serialization response is executed in the IO thread.
  • Operations executed in Dubbo threads:
    1. Received, connected, disconnected, and caught are executed in the Dubbo thread; however, the connected and disconnected behaviors are isolated from the other two behaviors through a thread pool and provide link limitation and alarm capabilities in the Dubbo connected thread pool.
    2. The behavior of deserializing requests is done in Dubbo.

Triple Protocol

The following figure shows the thread model of the Triple protocol Provider side.

triple-provider

The thread model for Triple protocol Provider is currently quite simple, with both serialization and deserialization operations working in the Dubbo thread, while the IO thread does not carry these tasks.

Thread Pool Isolation

A new thread pool management method isolates the thread pools of each service within the provider application, making them independent from each other. If a service’s thread pool resources are exhausted, it will not affect other normal services. Thread pool configuration is supported, allowing users to specify their desired configuration.

Thread pool isolation ensures that the threads used for invoking remote methods by Dubbo are separate from the threads used by the microservice to execute its tasks. This helps improve system performance and stability by preventing thread blocking or competition.

Currently, configuration can be done via API, XML, or Annotation.

Configuration Parameters

  • ApplicationConfig adds String executor-management-mode parameter with values default and isolation, defaulting to default.
    • executor-management-mode = default uses the original thread pool management method shared between services granular by protocol port
    • executor-management-mode = isolation uses the new thread pool management method isolated between services granular by service tuple
  • ServiceConfig adds Executor executor parameter, for the thread pool isolated between services, which can be user-configurable. If not specified, a default thread pool for service isolation will be built using the protocol’s configuration (ProtocolConfig).

The new Executor executor configuration parameter in ServiceConfig is only valid if executor-management-mode = isolation is specified.

API

  1.     public void test() {
  2.         // provider app
  3.         DubboBootstrap providerBootstrap = DubboBootstrap.newInstance();
  5.         ServiceConfig serviceConfig1 = new ServiceConfig();
  6.         serviceConfig1.setInterface(DemoService.class);
  7.         serviceConfig1.setRef(new DemoServiceImpl());
  8.         serviceConfig1.setVersion(version1);
  9.         // Set executor1 for serviceConfig1, max threads is 10
  10.         NamedThreadFactory threadFactory1 = new NamedThreadFactory("DemoService-executor");
  11.         ExecutorService executor1 = Executors.newFixedThreadPool(10, threadFactory1);
  12.         serviceConfig1.setExecutor(executor1);
  14.         ServiceConfig serviceConfig2 = new ServiceConfig();
  15.         serviceConfig2.setInterface(HelloService.class);
  16.         serviceConfig2.setRef(new HelloServiceImpl());
  17.         serviceConfig2.setVersion(version2);
  18.         // Set executor2 for serviceConfig2, max threads is 100
  19.         NamedThreadFactory threadFactory2 = new NamedThreadFactory("HelloService-executor");
  20.         ExecutorService executor2 = Executors.newFixedThreadPool(100, threadFactory2);
  21.         serviceConfig2.setExecutor(executor2);
  23.         ServiceConfig serviceConfig3 = new ServiceConfig();
  24.         serviceConfig3.setInterface(HelloService.class);
  25.         serviceConfig3.setRef(new HelloServiceImpl());
  26.         serviceConfig3.setVersion(version3);
  27.         // Because executor is not set for serviceConfig3, the default executor of serviceConfig3 is built using
  28.         // the threadpool parameter of the protocolConfig ( FixedThreadpool , max threads is 200)
  29.         serviceConfig3.setExecutor(null);
  31.         // It takes effect only if [executor-management-mode=isolation] is configured
  32.         ApplicationConfig applicationConfig = new ApplicationConfig("provider-app");
  33.         applicationConfig.setExecutorManagementMode("isolation");
  35.         providerBootstrap
  36.         .application(applicationConfig)
  37.         .registry(registryConfig)
  38.         // export with tri and dubbo protocol
  39.         .protocol(new ProtocolConfig("tri", 20001))
  40.         .protocol(new ProtocolConfig("dubbo", 20002))
  41.         .service(serviceConfig1)
  42.         .service(serviceConfig2)
  43.         .service(serviceConfig3);
  45.         providerBootstrap.start();
  46.     }

XML

  1. <beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  2.        xmlns:dubbo="http://dubbo.apache.org/schema/dubbo"
  3.        xmlns="http://www.springframework.org/schema/beans"
  4.        xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.3.xsd
  5.        http://dubbo.apache.org/schema/dubbo http://dubbo.apache.org/schema/dubbo/dubbo.xsd">
  7.   <!-- NOTE: we need config executor-management-mode="isolation" -->
  8.   <dubbo:application name="demo-provider" executor-management-mode="isolation">
  9.   </dubbo:application>
  11.   <dubbo:config-center address="zookeeper://127.0.0.1:2181"/>
  12.   <dubbo:metadata-report address="zookeeper://127.0.0.1:2181"/>
  13.   <dubbo:registry id="registry1" address="zookeeper://127.0.0.1:2181?registry-type=service"/>
  15.   <dubbo:protocol name="dubbo" port="-1"/>
  16.   <dubbo:protocol name="tri" port="-1"/>
  18.   <!-- expose three service with dubbo and tri protocol-->
  19.   <bean id="demoServiceV1" class="org.apache.dubbo.config.spring.impl.DemoServiceImpl"/>
  20.   <bean id="helloServiceV2" class="org.apache.dubbo.config.spring.impl.HelloServiceImpl"/>
  21.   <bean id="helloServiceV3" class="org.apache.dubbo.config.spring.impl.HelloServiceImpl"/>
  23.   <!-- customized thread pool -->
  24.   <bean id="executor-demo-service"
  25.         class="org.apache.dubbo.config.spring.isolation.spring.support.DemoServiceExecutor"/>
  26.   <bean id="executor-hello-service"
  27.         class="org.apache.dubbo.config.spring.isolation.spring.support.HelloServiceExecutor"/>
  29.   <!-- this service use [executor="executor-demo-service"] as isolated thread pool-->
  30.   <dubbo:service executor="executor-demo-service"
  31.                  interface="org.apache.dubbo.config.spring.api.DemoService" version="1.0.0" group="Group1"
  32.                  timeout="3000" ref="demoServiceV1" registry="registry1" protocol="dubbo,tri"/>
  34.   <!-- this service use [executor="executor-hello-service"] as isolated thread pool-->
  35.   <dubbo:service executor="executor-hello-service"
  36.                  interface="org.apache.dubbo.config.spring.api.HelloService" version="2.0.0" group="Group2"
  37.                  timeout="5000" ref="helloServiceV2" registry="registry1" protocol="dubbo,tri"/>
  39.   <!-- not set executor for this service, the default executor built using threadpool parameter of the protocolConfig -->
  40.   <dubbo:service interface="org.apache.dubbo.config.spring.api.HelloService" version="3.0.0" group="Group3"
  41.                  timeout="5000" ref="helloServiceV3" registry="registry1" protocol="dubbo,tri"/>
  43. </beans>

Annotation

  1. @Configuration
  2. @EnableDubbo(scanBasePackages = "org.apache.dubbo.config.spring.isolation.spring.annotation.provider")
  3. public class ProviderConfiguration {
  4.     @Bean
  5.     public RegistryConfig registryConfig() {
  6.         RegistryConfig registryConfig = new RegistryConfig();
  7.         registryConfig.setAddress("zookeeper://127.0.0.1:2181");
  8.         return registryConfig;
  9.     }
  11.     // NOTE: we need config executor-management-mode="isolation"
  12.     @Bean
  13.     public ApplicationConfig applicationConfig() {
  14.         ApplicationConfig applicationConfig = new ApplicationConfig("provider-app");
  16.         applicationConfig.setExecutorManagementMode("isolation");
  17.         return applicationConfig;
  18.     }
  20.     // expose services with dubbo protocol
  21.     @Bean
  22.     public ProtocolConfig dubbo() {
  23.         ProtocolConfig protocolConfig = new ProtocolConfig("dubbo");
  24.         return protocolConfig;
  25.     }
  27.     // expose services with tri protocol
  28.     @Bean
  29.     public ProtocolConfig tri() {
  30.         ProtocolConfig protocolConfig = new ProtocolConfig("tri");
  31.         return protocolConfig;
  32.     }
  34.     // customized thread pool
  35.     @Bean("executor-demo-service")
  36.     public Executor demoServiceExecutor() {
  37.         return new DemoServiceExecutor();
  38.     }
  40.     // customized thread pool
  41.     @Bean("executor-hello-service")
  42.     public Executor helloServiceExecutor() {
  43.         return new HelloServiceExecutor();
  44.     }
  45. }
  1. // custom thread pool
  2. public class DemoServiceExecutor extends ThreadPoolExecutor {
  3.     public DemoServiceExecutor() {
  4.         super(10, 10, 60, TimeUnit.SECONDS, new LinkedBlockingDeque<>(),
  5.             new NamedThreadFactory("DemoServiceExecutor"));
  6.     }
  7. }
  1. // custom thread pool
  2. public class HelloServiceExecutor extends ThreadPoolExecutor {
  3.     public HelloServiceExecutor() {
  4.         super(100, 100, 60, TimeUnit.SECONDS, new LinkedBlockingDeque<>(),
  5.             new NamedThreadFactory("HelloServiceExecutor"));
  6.     }
  7. }
  1. // "executor-hello-service" is beanName
  2. @DubboService(executor = "executor-demo-service", version = "1.0.0", group = "Group1")
  3. public class DemoServiceImplV1 implements DemoService {
  5.   @Override
  6.   public String sayName(String name) {
  7.     return "server name";
  8.   }
  10.   @Override
  11.   public Box getBox() {
  12.     return null;
  13.   }
  14. }
  1. // not set executor for this service, the default executor built using threadpool parameter of the protocolConfig
  2. @DubboService(version = "3.0.0", group = "Group3")
  3. public class HelloServiceImplV2 implements HelloService {
  4.     private static final Logger logger = LoggerFactory.getLogger(HelloServiceImplV2.class);
  6.     @Override
  7.     public String sayHello(String name) {
  8.         return "server hello";
  9.     }
  10. }
  1. @DubboService(executor = "executor-hello-service", version = "2.0.0", group = "Group2")
  2. public class HelloServiceImplV3 implements HelloService {
  3.     private static final Logger logger = LoggerFactory.getLogger(HelloServiceImplV3.class);
  5.     @Override
  6.     public String sayHello(String name) {
  7.         return "server hello";
  8.     }
  9. }

Thread Pool Status Export

Dubbo automatically exports thread stacks through Jstack to preserve the scene for troubleshooting.

Default strategy

  • Export path: user.home identifies the user’s home directory
  • Export interval: the shortest interval allows exports every 10 minutes
  • Export switch: defaults to open

When the business thread pool is full, we need to know what resources and conditions the threads are waiting for to find bottlenecks or exceptions in the system.

Export Switch Control

  1. # dubbo.properties
  2. dubbo.application.dump.enable=true
  1. <dubbo:application name="demo-provider" dump-enable="false"/>
  1. dubbo:
  2.   application:
  3.     name: dubbo-springboot-demo-provider
  4.     dump-enable: false

Export Path

  1. # dubbo.properties
  2. dubbo.application.dump.directory=/tmp
  1. <dubbo:application name="demo-provider" dump-directory="/tmp"/>
  1. dubbo:
  2.   application:
  3.     name: dubbo-springboot-demo-provider
  4.     dump-directory: /tmp

Feedback

Was this page helpful?

Yes No

Last modified September 30, 2024: Update & Translate Overview Docs (#3040) (d37ebceaea7)