dubbo-pilot Control Plane 部署

1 总体目标

  • 控制面编译和镜像构建
  • 使用 istioctl 在 kubernetes 环境部署
  • 如何对控制面程序 debug

2 基本流程

这个例子将演示如何在编译 dubbo-pilot 控制平面并在 kubernetes 环境下如何使用 istioctl 进行部署

  1. 本地启动控制平面，对 dubbo-pilot 进行启动和 debug
  2. 使用 istioctl 在 k8s 环境启动和 debug

3 详细步骤

3.1 环境要求

  • Golang
  • Docker
  • Minikube/Kind
  • Kubectl
  • Dlv

3.2 本地部署

3.2.1 编译

  1. 编译 docker-builder
  1. cd dubbo-go-pixiu/tools/docker-builder && go install
  3. docker-builder -h:
  5. Builds Istio docker images
  7. Usage:
  8.    [flags]
  10. Flags:
  11.       --architecures strings   architectures to build (default [linux/amd64])
  12.       --base-version string    base version to use (default "latest")
  13.       --builder string         type of builder to use. options are crane or docker (default "docker")
  14.   -h, --help                   help for this command
  15.       --hub strings            docker hub(s) (default [localhost:5000])
  16.       --istio-version string   istio version to use (default "1.14-dev")
  17.       --kind-load              kind cluster to load into
  18.       --no-cache               disable caching
  19.       --no-clobber             do not allow pushing images that already exist
  20.       --proxy-version string   proxy version to use (default "7ae8e27f274b33dc2f4d83100aea5971ed6698d3")
  21.       --push                   push targets to registry
  22.       --save                   save targets to tar.gz
  23.       --tag strings            docker tag(s) (default [latest])
  24.       --targets strings        targets to build (default [app,app_sidecar_centos_7,app_sidecar_debian_11,app_sidecar_ubuntu_jammy,app_sidecar_ubuntu_xenial,ext-authz,install-cni,istioctl,operator,pilot,proxyv2])
  25.       --variants strings       variants to build (default [default])
  26.       --version                show build version
  1. 使用 docker-builder 自动编译 && 构建镜像

编译 istioctl

  1. docker-builder --targets istioctl
  3. 编译完成：
  5. ls  out/linux_amd64/
  6. istioctl  logs  pilot-agent  pilot-discovery

编译 dubbo-pilot 并推送到私有镜像仓库

  1. tools/docker-builder/docker-builder --targets pilot --hub docker.io/bobtthp --push

3.2.2 本地部署

本地启动方式：

  1. ./out/linux_amd64/pilot-discovery
  3. 启动日志：
  4. 2022-09-24T15:31:56.751245Z    info    FLAG: --caCertFile=""
  5. 2022-09-24T15:31:56.751277Z    info    FLAG: --clusterAliases="[]"
  6. 2022-09-24T15:31:56.751280Z    info    FLAG: --clusterID="Kubernetes"
  7. 2022-09-24T15:31:56.751282Z    info    FLAG: --clusterRegistriesNamespace="istio-system"
  8. 2022-09-24T15:31:56.751284Z    info    FLAG: --configDir=""
  9. 2022-09-24T15:31:56.751286Z    info    FLAG: --ctrlz_address="localhost"
  10. 2022-09-24T15:31:56.751289Z    info    FLAG: --ctrlz_port="9876"
  11. 2022-09-24T15:31:56.751291Z    info    FLAG: --domain="cluster.local"
  12. ...
  13. 2022-09-24T15:31:56.753814Z    info    initializing mesh configuration ./etc/istio/config/mesh

3.3 容器部署

3.3.1 镜像构建

构建远程 debug 镜像

  1. 下载 dlv
  1. git clone https://github.com/go-delve/delve.git
  2. make install
  4. which dlv
  5. /root/go/bin/dlv
  1. Dockerfile 增加dlv dubbo-go-pixiu/pilot/docker/Dockerfile.pilot:
  1. COPY ${TARGETARCH:-amd64}/dlv /usr/local/bin/dlv
  1. 拷贝 dlv 至镜像挂载目录中：
  1. cp /root/go/bin/dlv out/linux_amd64/dockerx_build/build.docker.pilot/amd64/
  1. debug 镜像构建并推送:
  1. docker-builder --targets pilot --hub docker.io/bobtthp --push --tag debug
  1. 本地也可以查看镜像构建情况：
  1. [root~master-1] /tmp/dubbo-go-pixiu/tools/docker-builder> docker images
  2. REPOSITORY                                                        TAG                                               IMAGE ID       CREATED             SIZE
  3. bobtthp/pilot                                                     latest                                            7b1aadd55120   13 minutes ago      262MB

3.3.2 部署

  1. 使用刚构建的镜像部署:
  1. out/linux_amd64/istioctl --set .values.pilot.image=bobtthp/pilot:debug install
  1. 查看部署情况：
  1. [root~master] /tmp/dubbo-go-pixiu/tools/docker-builder> kubectl  get po -n istio-system
  2. NAME                    READY   STATUS    RESTARTS   AGE
  3. istiod-fd5d9f77-2ncjq   1/1     Running   0          18m
  1. 进入容器远程 debug:
  1. [root~master-1] /tmp/dubbo-go-pixiu> kubectl  exec -it -n istio-system istiod-fd5d9f77-2ncjq bash
  2. kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
  1. 启动 dlv:
  1. istio-proxy@istiod-fd5d9f77-2ncjq:/$ dlv --listen=:8015 --headless=true --api-version=2 --log attach `ps -ef |grep pilot-discovery| awk '{print $2}'`
  2. 2022-11-04T15:43:14Z error layer=debugger could not create config directory: mkdir /home/istio-proxy/.config: read-only file system
  3. API server listening at: [::]:8015
  4. 2022-11-04T15:43:14Z warning layer=rpc Listening for remote connections (connections are not authenticated nor encrypted)
  5. 2022-11-04T15:43:14Z info layer=debugger attaching to pid 1
  1. 对外暴露端口：
  1. [root~master-1] /tmp> kubectl  port-forward -n istio-system istiod-fd5d9f77-2ncjq 8015:8015
  2. Forwarding from 127.0.0.1:8015 -> 8015
  3. Forwarding from [::1]:8015 -> 8015
  1. 可以进行远程调试

最后修改 December 16, 2022: Fix check (#1736) (97972c1)