Using YDB in Kubernetes

Monitoring

For convenience, YDB provides standard mechanisms for collecting logs and metrics.

Logging is done to standard stdout and stderr streams and can be redirected using popular solutions. We recommend using a combination of Fluentd and Elastic Stack.

To collect metrics, ydb-controller provides resources like ServiceMonitor. They can be handled using kube-prometheus-stack.

Description of YDB controller resources

Storage resource

  1. apiVersion: ydb.tech/v1alpha1
  2. kind: Storage
  3. metadata:
  4.   # make sure you specify this name when creating a database
  5.   name: storage-sample
  6. spec:
  7.   # you can specify either the YDB version or the container name 
  8.   # image:
  9.   #   name: "cr.yandex/ydb/ydb:stable-21-4-14"
  10.   version: 21.4.30
  11.   # the number of cluster pods
  12.   nodes: 8
  14. # https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims
  15. # specifying disk resources for cluster pods
  16. dataStore:
  17.   volumeMode: Block
  18.   accessModes:
  19.     - ReadWriteOnce
  20.   resources:
  21.     requests:
  22.       storage: 80Gi
  24. # https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
  25. # Limiting the YDB cluster pod resources
  26. resources:
  27.   limits:
  28.     cpu: 2
  29.     memory: 8Gi
  30.   requests:
  31.     cpu: 2
  32.     memory: 8Gi
  34. # https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector
  35. # selector of the cluster nodes that the YDB pods can run on
  36. # nodeSelector:
  37. #   network: fast
  39. # https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
  40. # indicating which cluster instances' disadvantages can be ignored when assigning YDB pods
  41. # tolerations:
  42. #   - key: "example-key"
  43. #     operator: "Exists"
  44. #     effect: "NoSchedule"
  46. # https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity
  47. # instructing the scheduler to distribute the YDB pods evenly across the nodes
  48. # affinity:
  49. #   podAntiAffinity:
  50. #      preferredDuringSchedulingIgnoredDuringExecution:
  51. #      - weight: 100
  52. #        podAffinityTerm:
  53. #          labelSelector:
  54. #            matchExpressions:
  55. #            - key: app.kubernetes.io/instance
  56. #              operator: In
  57. #              values:
  58. #              - ydb
  59. #          topologyKey: kubernetes.io/hostname

Use - 图1

Database resource

  1. apiVersion: ydb.tech/v1alpha1
  2. kind: Database
  3. metadata:
  4.   # the name to be used when creating a database => `/root/database-sample`
  5.   name: database-sample
  6. spec:
  7.   # you can specify either the YDB version or the container name 
  8.   # image:
  9.   #   name: "cr.yandex/ydb/ydb:stable-21-4-14"
  10.   version: 21.4.30
  11.   # the number of database pods
  12.   nodes: 6
  14.   # the pointer of the YDB cluster storage nodes, corresponds to the Helm release name
  15.   storageClusterRef:
  16.     name: ydb
  17.     namespace: default
  19.   # https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
  20.   # Limiting the YDB dynamic pod resources
  21.   resources:
  22.     limits:
  23.       cpu: 2
  24.       memory: 8Gi
  25.     requests:
  26.       cpu: 2
  27.       memory: 8Gi
  29.   # https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector
  30.   # selector of the cluster nodes that the YDB pods can run on
  31.   # nodeSelector:
  32.   #   network: fast
  34.   # https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
  35.   # indicating which cluster instances' disadvantages can be ignored when assigning YDB pods
  36.   # tolerations:
  37.   #   - key: "example-key"
  38.   #     operator: "Exists"
  39.   #     effect: "NoSchedule"
  41.   # https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity
  42.   # instructing the scheduler to distribute the YDB pods evenly across the nodes
  43.   # affinity:
  44.   #   podAntiAffinity:
  45.   #      preferredDuringSchedulingIgnoredDuringExecution:
  46.   #      - weight: 100
  47.   #        podAffinityTerm:
  48.   #          labelSelector:
  49.   #            matchExpressions:
  50.   #            - key: app.kubernetes.io/instance
  51.   #              operator: In
  52.   #              values:
  53.   #              - ydb
  54.   #          topologyKey: kubernetes.io/hostname

Use - 图2

Allocating resources

You can limit resource consumption for each YDB pod. If you leave the limit values empty, a pod can use the entire CPU time and VM RAM. This may cause undesirable effects. We recommend that you always specify the resource limits explicitly.

To learn more about resource allocation and limits, see the Kubernetes documentation.