Concurrency Options

This is a walkthrough of different concurrency options available to control the number of concurrent workers that ghz utilizes to make requests to the server. All examples are done using a simple unary gRPC call.

Many of these options are similar to the load control options, but independently control the concurrent workers utilized.

Step Up Concurrency

  1. ./dist/ghz --insecure --async --proto /protos/helloworld.proto \
  2.   --call helloworld.Greeter/SayHello \
  3.   -n 10000 --rps 200 \
  4.   --concurrency-schedule=step --concurrency-start=5 --concurrency-step=5 --concurrency-end=50 --concurrency-step-duration=5s \
  5.   -d '{"name":"{{.WorkerID}}"}' 0.0.0.0:50051
  7. Summary:
  8.   Count:    10000
  9.   Total:    50.05 s
  10.   Slowest:  52.04 ms
  11.   Fastest:  50.19 ms
  12.   Average:  50.59 ms
  13.   Requests/sec: 199.79
  15. Response time histogram:
  16.   50.187 [1]    |
  17.   50.373 [1786] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  18.   50.558 [3032] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  19.   50.743 [2822] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  20.   50.929 [1536] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  21.   51.114 [562]  |∎∎∎∎∎∎∎
  22.   51.299 [194]  |∎∎∎
  23.   51.485 [42]   |∎
  24.   51.670 [15]   |
  25.   51.855 [6]    |
  26.   52.041 [4]    |
  28. Latency distribution:
  29.   10 % in 50.33 ms
  30.   25 % in 50.42 ms
  31.   50 % in 50.57 ms
  32.   75 % in 50.73 ms
  33.   90 % in 50.89 ms
  34.   95 % in 51.01 ms
  35.   99 % in 51.24 ms
  37. Status code distribution:

This test performs a constant load at 200 RPS, starting with 5 workers, and increasing concurrency by 5 workers every 5s until we have 50 workers. At that point all 50 workers will be used to sustain the constant 200 RPS until 10000 total request limit is reached. Worker count over time would look something like:

Step Up Concurrency Constant Load

Step Down Concurrency

  1. ./dist/ghz --insecure --async --proto /protos/helloworld.proto \
  2.   --call helloworld.Greeter/SayHello \
  3.   -n 10000 --rps 200 \
  4.   --concurrency-schedule=step --concurrency-start=50 --concurrency-step=-5 \
  5.   --concurrency-step-duration=5s --concurrency-max-duration=30s \
  6.   -d '{"name":"{{.WorkerID}}"}' 0.0.0.0:50051
  8. Summary:
  9.   Count:    10000
  10.   Total:    50.05 s
  11.   Slowest:  52.13 ms
  12.   Fastest:  50.15 ms
  13.   Average:  50.63 ms
  14.   Requests/sec: 199.79
  16. Response time histogram:
  17.   50.152 [1]    |
  18.   50.350 [1145] |∎∎∎∎∎∎∎∎∎∎∎∎∎
  19.   50.548 [2476] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  20.   50.746 [3491] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  21.   50.943 [2202] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  22.   51.141 [490]  |∎∎∎∎∎∎
  23.   51.339 [148]  |∎∎
  24.   51.536 [30]   |
  25.   51.734 [10]   |
  26.   51.932 [4]    |
  27.   52.130 [3]    |
  29. Latency distribution:
  30.   10 % in 50.34 ms
  31.   25 % in 50.47 ms
  32.   50 % in 50.63 ms
  33.   75 % in 50.77 ms
  34.   90 % in 50.89 ms
  35.   95 % in 50.99 ms
  36.   99 % in 51.24 ms
  38. Status code distribution:
  39.   [OK]   10000 responses

This test performs a constant load at 200 RPS, starting with 50 workers, and decreasing concurrency by 5 workers every 5s until 30s has elapsed. At that point all remaining workers will be used to sustain the constant 200 RPS until 10000 total request limit is reached. Worker count over time would look something like:

Step Down Concurrency Constant Load

Linear increase of concurrency

  1. ./dist/ghz --insecure --async --proto /protos/helloworld.proto \
  2.   --call helloworld.Greeter/SayHello \
  3.   -n 10000 --rps 200 \
  4.   --concurrency-schedule=line --concurrency-start=20 --concurrency-step=2 --concurrency-max-duration=30s \
  5.   -d '{"name":"{{.WorkerID}}"}' 0.0.0.0:50051
  7. Summary:
  8.   Count:    10000
  9.   Total:    50.05 s
  10.   Slowest:  58.54 ms
  11.   Fastest:  50.16 ms
  12.   Average:  50.60 ms
  13.   Requests/sec: 199.79
  15. Response time histogram:
  16.   50.157 [1]    |
  17.   50.995 [9515] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  18.   51.834 [477]  |∎∎
  19.   52.672 [3]    |
  20.   53.510 [0]    |
  21.   54.349 [0]    |
  22.   55.187 [1]    |
  23.   56.025 [0]    |
  24.   56.864 [0]    |
  25.   57.702 [1]    |
  26.   58.540 [2]    |
  28. Latency distribution:
  29.   10 % in 50.31 ms
  30.   25 % in 50.40 ms
  31.   50 % in 50.60 ms
  32.   75 % in 50.75 ms
  33.   90 % in 50.89 ms
  34.   95 % in 50.99 ms
  35.   99 % in 51.25 ms
  37. Status code distribution:
  38.   [OK]   10000 responses

This test performs a constant load at 200 RPS, starting with 20 workers, and increasing concurrency linearly every 1s by 2 workers until 30s has elapsed. At that point all remaining workers will be used to sustain the constant 200 RPS until 10000 total request limit is reached.

Lene Up Concurrency Constant Load