本文讲解了如何使您的集群符合互联网安全中心发布的 Kubernetes 安全基准，保护集群中节点的安全。安装 Kubernetes 之前，请按照本指南进行操作。加固指南旨在与特定版本的 CIS Kubernetes Benchmark，Kubernetes 和 Rancher 一起使用。

加固指南旨在与特定版本的 CIS Kubernetes Benchmark，Kubernetes 和 Rancher 一起使用：

加固指南版本Rancher 版本CIS Benchmark 版本Kubernetes 版本
加固指南 v2.4.0Rancher v2.4.0Benchmark v1.5Kubernetes v1.15

点击这里下载 PDF 版本的加固指南

概览

下面的安全加固指南是针对在生产环境的 Rancher v2.4.0 中使用 Kubernetes v1.15 版本的集群。它概述了如何满足互联网安全中心（CIS）提出的 Kubernetes 安全标准。

有关如果根据官方 CIS 基准评估集群的更多详细信息，请参阅CIS Benchmark Rancher 自测指南 - Rancher v2.4.0。

已知问题

如果注册自定义节点时只提供了公共 IP，在 CIS 1.5 加固设置中，将无法正常在 Rancher UI 中使用执行命令行和查看日志功能。

  • 如果注册自定义节点时只提供了公共 IP，在 CIS 1.5 加固设置中，将无法正常在 Rancher UI 中使用执行命令行和查看日志功能。如果想要使用上述两个功能，请在注册自定义节点时提供私有 IP 地址。
  • default_pod_security_policy_template_id:为 restricted时，Rancher 在默认的 service account 中创建角色绑定和集群角色绑定。CIS 1.5 要求默认 service account 没有绑定任何角色，不提供 service account 的 token，不分配特定的权限。

配置内核运行时参数

对于集群中的所有类型的节点，都建议使用以下的sysctl配置。在/etc/sysctl.d/90-kubelet.conf中设置以下参数：

  1. vm.overcommit_memory=1
  2. vm.panic_on_oom=0
  3. kernel.panic=10
  4. kernel.panic_on_oops=1
  5. kernel.keys.root_maxbytes=25000000

执行sysctl -p /etc/sysctl.d/90-kubelet.conf来启用配置。

配置etcd用户和组

在安装 RKE 之前，需要设置etcd服务的用户帐户和组。etcd用户的uid和gid将在 RKE 的config.yml中使用，以在安装期间为文件和目录设置适当的权限。 .

创建etcd用户和组

要创建etcd组，请运行以下控制台命令。

  1. addgroup --gid 52034 etcd
  2. useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd

使用etcd用户的uid和gid更新 RKE 的 config.yml文件：

  1. services:
  2.   etcd:
  3.     gid: 52034
  4.     uid: 52034

将default服务账号的automountServiceAccountToken设置为 false。

Kubernetes 提供了一个默认服务账号（Service Account），如果集群的工作负载中没有为 Pod 分配任何特定服务账号，那么它将会使用这个default的服务账号。在需要从 Pod 访问 Kubernetes API 的情况下，应为该 Pod 创建一个特定的服务账号，并向该服务账号授予权限。这个default的服务账户应该被设置为不提供服务账号令牌（service account token）和任何权限。将automountServiceAccountToken设置为 false 之后，Kubernetes 在启动 Pod 时，将不会自动注入default服务账户。

每个命名空间中的default服务账号必须包含以下值：

  1. automountServiceAccountToken: false

把下面的 yaml 另存为account_update.yaml

  1. apiVersion: v1
  2. kind: ServiceAccount
  3. metadata:
  4.   name: default
  5. automountServiceAccountToken: false

创建一个名称为account_update.sh的脚本。通过运行chmod +x account_update.sh，使这个脚本有执行权限。

  1. #!/bin/bash -e
  3. for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
  4.   kubectl patch serviceaccount default -n ${namespace} -p "$(cat account_update.yaml)"
  5. done

确保所有命名空间均已定义网络策略

在同一个 Kubernetes 集群上运行不同的应用程序会产生一个风险，那就是应用可能受到相邻应用程序的攻击。为了确保容器只能与预期的容器进行通信，网络细分是必不可少的。通过设置网络策略（Network Policy），可以设置哪些 Pod 之间可以通信，以及是否可以和其他网络端点进行通信。

网络策略是作用于命名空间范围的。将网络策略应用于给定命名空间时，所有不被这个策略允许的流量将被拒绝。然而，如果命名空间中没有设置网络策略，那么进出这个命名空间中 Pod 的所有流量都将被允许。要使用网络策略，必须启用 CNI（容器网络接口）插件。本指南使用canal提供策略实施。您可以在这里找到有关 CNI 插件的其他信息。

在集群上启用 CN​​I 插件后，您可以设置一个默认的网络策略。下面是一个宽松的网络策略示例，仅供参考。如果您想要允许到某个命名空间内所有 Pod 的流量（即使已经添加了一些策略，使得一些 Pods 被隔离了），您可以创建一个策略，明确允许该命名空间中的所有流量。将以下yaml另存为 default-allow-all.yaml。额外关于网络策略的信息，请查看Kubernetes 官方文档。

:::important 重要 这个NetworkPolicy示例不建议在生产环境中使用。 :::

  1. ---
  2. apiVersion: networking.k8s.io/v1
  3. kind: NetworkPolicy
  4. metadata:
  5.   name: default-allow-all
  6. spec:
  7.   podSelector: {}
  8.   ingress:
  9.     - {}
  10.   egress:
  11.     - {}
  12.   policyTypes:
  13.     - Ingress
  14.     - Egress

创建一个名称为apply_networkPolicy_to_all_ns.sh的脚本。通过运行chmod +x apply_networkPolicy_to_all_ns.sh，使这个脚本有执行权限

  1. #!/bin/bash -e
  3. for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
  4.   kubectl apply -f default-allow-all.yaml -n ${namespace}
  5. done

运行脚本，以使全部的命名空间使用这个default-allow-all.yaml文件中的宽松的NetworkPolicy。

加固的 RKE cluster.yml 配置参考

您可以用这个供您参考的cluster.yml，通过 RKE CLI 来创建安全加固的 Rancher Kubernetes Engine（RKE）集群。有关每个配置的详细信息，请参阅RKE 文档。

  1. # If you intend to deploy Kubernetes in an air-gapped environment,
  2. # please consult the documentation on how to configure custom RKE images.
  3. kubernetes_version: "v1.15.9-rancher1-1"
  4. enable_network_policy: true
  5. default_pod_security_policy_template_id: "restricted"
  6. services:
  7.   etcd:
  8.     uid: 52034
  9.     gid: 52034
  10.   kube-api:
  11.     pod_security_policy: true
  12.     secrets_encryption_config:
  13.       enabled: true
  14.     audit_log:
  15.       enabled: true
  16.     admission_configuration:
  17.     event_rate_limit:
  18.       enabled: true
  19.   kube-controller:
  20.     extra_args:
  21.       feature-gates: "RotateKubeletServerCertificate=true"
  22.   scheduler:
  23.     image: ""
  24.     extra_args: {}
  25.     extra_binds: []
  26.     extra_env: []
  27.   kubelet:
  28.     generate_serving_certificate: true
  29.     extra_args:
  30.       feature-gates: "RotateKubeletServerCertificate=true"
  31.       protect-kernel-defaults: "true"
  32.       tls-cipher-suites: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256"
  33.     extra_binds: []
  34.     extra_env: []
  35.     cluster_domain: ""
  36.     infra_container_image: ""
  37.     cluster_dns_server: ""
  38.     fail_swap_on: false
  39.   kubeproxy:
  40.     image: ""
  41.     extra_args: {}
  42.     extra_binds: []
  43.     extra_env: []
  44. network:
  45.   plugin: ""
  46.   options: {}
  47.   mtu: 0
  48.   node_selector: {}
  49. authentication:
  50.   strategy: ""
  51.   sans: []
  52.   webhook: null
  53. addons: |
  54.   ---
  55.   apiVersion: v1
  56.   kind: Namespace
  57.   metadata:
  58.     name: ingress-nginx
  59.   ---
  60.   apiVersion: rbac.authorization.k8s.io/v1
  61.   kind: Role
  62.   metadata:
  63.     name: default-psp-role
  64.     namespace: ingress-nginx
  65.   rules:
  66.   - apiGroups:
  67.     - extensions
  68.     resourceNames:
  69.     - default-psp
  70.     resources:
  71.     - podsecuritypolicies
  72.     verbs:
  73.     - use
  74.   ---
  75.   apiVersion: rbac.authorization.k8s.io/v1
  76.   kind: RoleBinding
  77.   metadata:
  78.     name: default-psp-rolebinding
  79.     namespace: ingress-nginx
  80.   roleRef:
  81.     apiGroup: rbac.authorization.k8s.io
  82.     kind: Role
  83.     name: default-psp-role
  84.   subjects:
  85.   - apiGroup: rbac.authorization.k8s.io
  86.     kind: Group
  87.     name: system:serviceaccounts
  88.   - apiGroup: rbac.authorization.k8s.io
  89.     kind: Group
  90.     name: system:authenticated
  91.   ---
  92.   apiVersion: v1
  93.   kind: Namespace
  94.   metadata:
  95.     name: cattle-system
  96.   ---
  97.   apiVersion: rbac.authorization.k8s.io/v1
  98.   kind: Role
  99.   metadata:
  100.     name: default-psp-role
  101.     namespace: cattle-system
  102.   rules:
  103.   - apiGroups:
  104.     - extensions
  105.     resourceNames:
  106.     - default-psp
  107.     resources:
  108.     - podsecuritypolicies
  109.     verbs:
  110.     - use
  111.   ---
  112.   apiVersion: rbac.authorization.k8s.io/v1
  113.   kind: RoleBinding
  114.   metadata:
  115.     name: default-psp-rolebinding
  116.     namespace: cattle-system
  117.   roleRef:
  118.     apiGroup: rbac.authorization.k8s.io
  119.     kind: Role
  120.     name: default-psp-role
  121.   subjects:
  122.   - apiGroup: rbac.authorization.k8s.io
  123.     kind: Group
  124.     name: system:serviceaccounts
  125.   - apiGroup: rbac.authorization.k8s.io
  126.     kind: Group
  127.     name: system:authenticated
  128.   ---
  129.   apiVersion: policy/v1beta1
  130.   kind: PodSecurityPolicy
  131.   metadata:
  132.     name: restricted
  133.   spec:
  134.     requiredDropCapabilities:
  135.     - NET_RAW
  136.     privileged: false
  137.     allowPrivilegeEscalation: false
  138.     defaultAllowPrivilegeEscalation: false
  139.     fsGroup:
  140.       rule: RunAsAny
  141.     runAsUser:
  142.       rule: MustRunAsNonRoot
  143.     seLinux:
  144.       rule: RunAsAny
  145.     supplementalGroups:
  146.       rule: RunAsAny
  147.     volumes:
  148.     - emptyDir
  149.     - secret
  150.     - persistentVolumeClaim
  151.     - downwardAPI
  152.     - configMap
  153.     - projected
  154.   ---
  155.   apiVersion: rbac.authorization.k8s.io/v1
  156.   kind: ClusterRole
  157.   metadata:
  158.     name: psp:restricted
  159.   rules:
  160.   - apiGroups:
  161.     - extensions
  162.     resourceNames:
  163.     - restricted
  164.     resources:
  165.     - podsecuritypolicies
  166.     verbs:
  167.     - use
  168.   ---
  169.   apiVersion: rbac.authorization.k8s.io/v1
  170.   kind: ClusterRoleBinding
  171.   metadata:
  172.     name: psp:restricted
  173.   roleRef:
  174.     apiGroup: rbac.authorization.k8s.io
  175.     kind: ClusterRole
  176.     name: psp:restricted
  177.   subjects:
  178.   - apiGroup: rbac.authorization.k8s.io
  179.     kind: Group
  180.     name: system:serviceaccounts
  181.   - apiGroup: rbac.authorization.k8s.io
  182.     kind: Group
  183.     name: system:authenticated
  184.   ---
  185.   apiVersion: v1
  186.   kind: ServiceAccount
  187.   metadata:
  188.     name: tiller
  189.     namespace: kube-system
  190.   ---
  191.   apiVersion: rbac.authorization.k8s.io/v1
  192.   kind: ClusterRoleBinding
  193.   metadata:
  194.     name: tiller
  195.   roleRef:
  196.     apiGroup: rbac.authorization.k8s.io
  197.     kind: ClusterRole
  198.     name: cluster-admin
  199.   subjects:
  200.   - kind: ServiceAccount
  201.     name: tiller
  202.     namespace: kube-system
  204. addons_include: []
  205. system_images:
  206.   etcd: ""
  207.   alpine: ""
  208.   nginx_proxy: ""
  209.   cert_downloader: ""
  210.   kubernetes_services_sidecar: ""
  211.   kubedns: ""
  212.   dnsmasq: ""
  213.   kubedns_sidecar: ""
  214.   kubedns_autoscaler: ""
  215.   coredns: ""
  216.   coredns_autoscaler: ""
  217.   kubernetes: ""
  218.   flannel: ""
  219.   flannel_cni: ""
  220.   calico_node: ""
  221.   calico_cni: ""
  222.   calico_controllers: ""
  223.   calico_ctl: ""
  224.   calico_flexvol: ""
  225.   canal_node: ""
  226.   canal_cni: ""
  227.   canal_flannel: ""
  228.   canal_flexvol: ""
  229.   weave_node: ""
  230.   weave_cni: ""
  231.   pod_infra_container: ""
  232.   ingress: ""
  233.   ingress_backend: ""
  234.   metrics_server: ""
  235.   windows_pod_infra_container: ""
  236. ssh_key_path: ""
  237. ssh_cert_path: ""
  238. ssh_agent_auth: false
  239. authorization:
  240.   mode: ""
  241.   options: {}
  242. ignore_docker_version: false
  243. private_registries: []
  244. ingress:
  245.   provider: ""
  246.   options: {}
  247.   node_selector: {}
  248.   extra_args: {}
  249.   dns_policy: ""
  250.   extra_envs: []
  251.   extra_volumes: []
  252.   extra_volume_mounts: []
  253. cluster_name: ""
  254. prefix_path: ""
  255. addon_job_timeout: 0
  256. bastion_host:
  257.   address: ""
  258.   port: ""
  259.   user: ""
  260.   ssh_key: ""
  261.   ssh_key_path: ""
  262.   ssh_cert: ""
  263.   ssh_cert_path: ""
  264. monitoring:
  265.   provider: ""
  266.   options: {}
  267.   node_selector: {}
  268. restore:
  269.   restore: false
  270.   snapshot_name: ""
  271. dns: null

安全加固的 RKE 模板配置参考

这个 RKE 参考模板提供了安装安全加固的 Kubenetes 所需的配置。RKE 模板用于配置 Kubernetes 和定义 Rancher 设置。请参阅Rancher 文档获得更多安装和 RKE 模板的详细信息。

  1. #
  2. # Cluster Config
  3. #
  4. default_pod_security_policy_template_id: restricted
  5. docker_root_dir: /var/lib/docker
  6. enable_cluster_alerting: false
  7. enable_cluster_monitoring: false
  8. enable_network_policy: true
  9. #
  10. # Rancher Config
  11. #
  12. rancher_kubernetes_engine_config:
  13.   addon_job_timeout: 30
  14.   addons: |-
  15.     ---
  16.     apiVersion: v1
  17.     kind: Namespace
  18.     metadata:
  19.       name: ingress-nginx
  20.     ---
  21.     apiVersion: rbac.authorization.k8s.io/v1
  22.     kind: Role
  23.     metadata:
  24.       name: default-psp-role
  25.       namespace: ingress-nginx
  26.     rules:
  27.     - apiGroups:
  28.       - extensions
  29.       resourceNames:
  30.       - default-psp
  31.       resources:
  32.       - podsecuritypolicies
  33.       verbs:
  34.       - use
  35.     ---
  36.     apiVersion: rbac.authorization.k8s.io/v1
  37.     kind: RoleBinding
  38.     metadata:
  39.       name: default-psp-rolebinding
  40.       namespace: ingress-nginx
  41.     roleRef:
  42.       apiGroup: rbac.authorization.k8s.io
  43.       kind: Role
  44.       name: default-psp-role
  45.     subjects:
  46.     - apiGroup: rbac.authorization.k8s.io
  47.       kind: Group
  48.       name: system:serviceaccounts
  49.     - apiGroup: rbac.authorization.k8s.io
  50.       kind: Group
  51.       name: system:authenticated
  52.     ---
  53.     apiVersion: v1
  54.     kind: Namespace
  55.     metadata:
  56.       name: cattle-system
  57.     ---
  58.     apiVersion: rbac.authorization.k8s.io/v1
  59.     kind: Role
  60.     metadata:
  61.       name: default-psp-role
  62.       namespace: cattle-system
  63.     rules:
  64.     - apiGroups:
  65.       - extensions
  66.       resourceNames:
  67.       - default-psp
  68.       resources:
  69.       - podsecuritypolicies
  70.       verbs:
  71.       - use
  72.     ---
  73.     apiVersion: rbac.authorization.k8s.io/v1
  74.     kind: RoleBinding
  75.     metadata:
  76.       name: default-psp-rolebinding
  77.       namespace: cattle-system
  78.     roleRef:
  79.       apiGroup: rbac.authorization.k8s.io
  80.       kind: Role
  81.       name: default-psp-role
  82.     subjects:
  83.     - apiGroup: rbac.authorization.k8s.io
  84.       kind: Group
  85.       name: system:serviceaccounts
  86.     - apiGroup: rbac.authorization.k8s.io
  87.       kind: Group
  88.       name: system:authenticated
  89.     ---
  90.     apiVersion: policy/v1beta1
  91.     kind: PodSecurityPolicy
  92.     metadata:
  93.       name: restricted
  94.     spec:
  95.       requiredDropCapabilities:
  96.       - NET_RAW
  97.       privileged: false
  98.       allowPrivilegeEscalation: false
  99.       defaultAllowPrivilegeEscalation: false
  100.       fsGroup:
  101.         rule: RunAsAny
  102.       runAsUser:
  103.         rule: MustRunAsNonRoot
  104.       seLinux:
  105.         rule: RunAsAny
  106.       supplementalGroups:
  107.         rule: RunAsAny
  108.       volumes:
  109.       - emptyDir
  110.       - secret
  111.       - persistentVolumeClaim
  112.       - downwardAPI
  113.       - configMap
  114.       - projected
  115.     ---
  116.     apiVersion: rbac.authorization.k8s.io/v1
  117.     kind: ClusterRole
  118.     metadata:
  119.       name: psp:restricted
  120.     rules:
  121.     - apiGroups:
  122.       - extensions
  123.       resourceNames:
  124.       - restricted
  125.       resources:
  126.       - podsecuritypolicies
  127.       verbs:
  128.       - use
  129.     ---
  130.     apiVersion: rbac.authorization.k8s.io/v1
  131.     kind: ClusterRoleBinding
  132.     metadata:
  133.       name: psp:restricted
  134.     roleRef:
  135.       apiGroup: rbac.authorization.k8s.io
  136.       kind: ClusterRole
  137.       name: psp:restricted
  138.     subjects:
  139.     - apiGroup: rbac.authorization.k8s.io
  140.       kind: Group
  141.       name: system:serviceaccounts
  142.     - apiGroup: rbac.authorization.k8s.io
  143.       kind: Group
  144.       name: system:authenticated
  145.     ---
  146.     apiVersion: v1
  147.     kind: ServiceAccount
  148.     metadata:
  149.       name: tiller
  150.       namespace: kube-system
  151.     ---
  152.     apiVersion: rbac.authorization.k8s.io/v1
  153.     kind: ClusterRoleBinding
  154.     metadata:
  155.       name: tiller
  156.     roleRef:
  157.       apiGroup: rbac.authorization.k8s.io
  158.       kind: ClusterRole
  159.       name: cluster-admin
  160.     subjects:
  161.     - kind: ServiceAccount
  162.       name: tiller
  163.       namespace: kube-system
  164.   ignore_docker_version: true
  165.   kubernetes_version: v1.15.9-rancher1-1
  166.   #
  167.   #   If you are using calico on AWS
  168.   #
  169.   #    network:
  170.   #      plugin: calico
  171.   #      calico_network_provider:
  172.   #        cloud_provider: aws
  173.   #
  174.   # # To specify flannel interface
  175.   #
  176.   #    network:
  177.   #      plugin: flannel
  178.   #      flannel_network_provider:
  179.   #      iface: eth1
  180.   #
  181.   # # To specify flannel interface for canal plugin
  182.   #
  183.   #    network:
  184.   #      plugin: canal
  185.   #      canal_network_provider:
  186.   #        iface: eth1
  187.   #
  188.   network:
  189.     mtu: 0
  190.     plugin: canal
  191.   #
  192.   #    services:
  193.   #      kube-api:
  194.   #        service_cluster_ip_range: 10.43.0.0/16
  195.   #      kube-controller:
  196.   #        cluster_cidr: 10.42.0.0/16
  197.   #        service_cluster_ip_range: 10.43.0.0/16
  198.   #      kubelet:
  199.   #        cluster_domain: cluster.local
  200.   #        cluster_dns_server: 10.43.0.10
  201.   #
  202.   services:
  203.     etcd:
  204.       backup_config:
  205.         enabled: false
  206.         interval_hours: 12
  207.         retention: 6
  208.         safe_timestamp: false
  209.       creation: 12h
  210.       extra_args:
  211.         election-timeout: "5000"
  212.         heartbeat-interval: "500"
  213.       gid: 52034
  214.       retention: 72h
  215.       snapshot: false
  216.       uid: 52034
  217.     kube_api:
  218.       always_pull_images: false
  219.       audit_log:
  220.         enabled: true
  221.       event_rate_limit:
  222.         enabled: true
  223.       pod_security_policy: true
  224.       secrets_encryption_config:
  225.         enabled: true
  226.       service_node_port_range: 30000-32767
  227.     kube_controller:
  228.       extra_args:
  229.         address: 127.0.0.1
  230.         feature-gates: RotateKubeletServerCertificate=true
  231.         profiling: "false"
  232.         terminated-pod-gc-threshold: "1000"
  233.     kubelet:
  234.       extra_args:
  235.         anonymous-auth: "false"
  236.         event-qps: "0"
  237.         feature-gates: RotateKubeletServerCertificate=true
  238.         make-iptables-util-chains: "true"
  239.         protect-kernel-defaults: "true"
  240.         streaming-connection-idle-timeout: 1800s
  241.         tls-cipher-suites: >-
  242.           TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
  243.       fail_swap_on: false
  244.       generate_serving_certificate: true
  245.     scheduler:
  246.       extra_args:
  247.         address: 127.0.0.1
  248.         profiling: "false"
  249.   ssh_agent_auth: false
  250. windows_prefered_cluster: false

安全加固的 Ubuntu 18.04 LTS cloud-config参考配置

这个供您参考的cloud-config通常被用于云基础架构环境中，来进行计算实例的配置管理。这个参考配置了在安装 kubernetes 之前需要的 Ubuntu 操作系统级别的设置。

  1. #cloud-config
  2. packages:
  3.   - curl
  4.   - jq
  5. runcmd:
  6.   - sysctl -w vm.overcommit_memory=1
  7.   - sysctl -w kernel.panic=10
  8.   - sysctl -w kernel.panic_on_oops=1
  9.   - curl https://releases.rancher.com/install-docker/18.09.sh | sh
  10.   - usermod -aG docker ubuntu
  11.   - return=1; while [ $return != 0 ]; do sleep 2; docker ps; return=$?; done
  12.   - addgroup --gid 52034 etcd
  13.   - useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd
  14. write_files:
  15.   - path: /etc/sysctl.d/kubelet.conf
  16.     owner: root:root
  17.     permissions: "0644"
  18.     content: |
  19.       vm.overcommit_memory=1
  20.       kernel.panic=10
  21.       kernel.panic_on_oops=1