Changing an actor system’s configuration

An actor system is the basis of YDB. Each component of the system is represented by one or more actors.
Each actor is allocated to a specific ExecutorPool corresponding to the actor’s task.
Changing the configuration will help you more accurately distribute the number of reserved cores for each type of task.

Actor system config description

The actor system configuration contains an enumeration of ExecutorPools, their mapping to task types, and the actor system scheduler configurations.

The following task types and their respective pools are currently supported:

  • System: Designed to perform fast internal YDB operations.
  • User: Includes the entire user load for handling and executing incoming requests.
  • Batch: Tasks that have no strict time limit for execution, mainly the execution of background operations.
  • IO: Responsible for performing any tasks with blocking operations (for example, writing logs to a file).
  • IC: Interconnect, includes all the load associated with communication between nodes.

Each pool is described by the Executor field as shown in the example below.

  1. Executor {
  2.   Type: BASIC
  3.   Threads: 9
  4.   SpinThreshold: 1
  5.   Name: "System"
  6. }

Updating configuration of the actor system - 图1

A summary of the main fields:

  • Type: Currently, two types are supported, such as BASIC and IO. All pools, except IO, are of the BASIC type.
  • Threads: The number of threads (concurrently running actors) in this pool.
  • SpinThreshold: The number of CPU cycles before going to sleep if there are no tasks, which a thread running as an actor will take (affects the CPU usage and request latency under low loads).
  • Name: The pool name to be displayed for the node in Monitoring.

Mapping pools to task types is done by setting the pool sequence number in special fields. Pool numbering starts from 0. Multiple task types can be set for a single pool.

List of fields with their respective tasks:

  • SysExecutor: System
  • UserExecutor: User
  • BatchExecutor: Batch
  • IoExecutor: IO

Example:

  1. SysExecutor: 0
  2. UserExecutor: 1
  3. BatchExecutor: 2
  4. IoExecutor: 3

Updating configuration of the actor system - 图2

The IC pool is set in a different way, via ServiceExecutor, as shown in the example below.

  1. ServiceExecutor {
  2.   ServiceName: "Interconnect"
  3.   ExecutorId: 4
  4. }

Updating configuration of the actor system - 图3

The actor system scheduler is responsible for the delivery of deferred messages exchanged by actors and is set with the following parameters:

  • Resolution: The minimum time offset step in microseconds.
  • SpinThreshold: Similar to the pool parameter, the number of CPU cycles before going to sleep if there are no messages.
  • ProgressThreshold: The maximum time offset step in microseconds.

If, for an unknown reason, the scheduler thread is stuck, it will send messages according to the lagging time, offsetting it by the ProgressThreshold value each time.

We do not recommend changing the scheduler config. You should only change the number of threads in the pool configs.

Example of the default actor system configuration:

  1. Executor {
  2.   Type: BASIC
  3.   Threads: 9
  4.   SpinThreshold: 1
  5.   Name: "System"
  6. }
  7. Executor {
  8.   Type: BASIC
  9.   Threads: 16
  10.   SpinThreshold: 1
  11.   Name: "User"
  12. }
  13. Executor {
  14.   Type: BASIC
  15.   Threads: 7
  16.   SpinThreshold: 1
  17.   Name: "Batch"
  18. }
  19. Executor {
  20.   Type: IO
  21.   Threads: 1
  22.   Name: "IO"
  23. }
  24. Executor {
  25.   Type: BASIC
  26.   Threads: 3
  27.   SpinThreshold: 10
  28.   Name: "IC"
  29.   TimePerMailboxMicroSecs: 100
  30. }
  31. SysExecutor: 0
  32. UserExecutor: 1
  33. IoExecutor: 3
  34. BatchExecutor: 2
  35. ServiceExecutor {
  36.   ServiceName: "Interconnect"
  37.   ExecutorId: 4
  38. }

Updating configuration of the actor system - 图4

On static nodes

Static nodes take the configuration of the actor system from the kikimr/cfg/sys.txt file.

After changing the configuration, restart the node.

On dynamic nodes

Dynamic nodes take the configuration from the CMS. To change it, you can use the following command:

  1. ConfigureRequest {
  2.   Actions {
  3.     AddConfigItem {
  4.       ConfigItem {
  5.         // UsageScope: { ... }
  6.         Config {
  7.           ActorSystemConfig {
  8.             <actor system config>
  9.           }  
  10.         }
  11.         MergeStrategy: 3
  12.       }
  13.     }
  14.   }
  15. }
  17. ```bash
  18. kikimr -s <endpoint> admin console execute --domain=<domain> --retry=10 actorsystem.txt

Updating configuration of the actor system - 图5