Securing and Exposing the Registry

You are viewing documentation for a release that is no longer supported. The latest supported version of version 3 is [3.11]. For the most recent version 4, see [4]

You are viewing documentation for a release that is no longer supported. The latest supported version of version 3 is [3.11]. For the most recent version 4, see [4]

Overview

By default, the OKD registry is secured during cluster installation so that it serves traffic via TLS. A passthrough route is also created by default to expose the service externally.

If for any reason your registry has not been secured or exposed, see the following sections for steps on how to manually do so.

Manually Securing the Registry

To manually secure the registry to serve traffic via TLS:

  1. Deploy the registry.

  2. Fetch the service IP and port of the registry:

    1. $ oc get svc/docker-registry
    2. NAME              LABELS                                    SELECTOR                  IP(S)            PORT(S)
    3. docker-registry   docker-registry=default                   docker-registry=default   172.30.124.220   5000/TCP

  3. You can use an existing server certificate, or create a key and server certificate valid for specified IPs and host names, signed by a specified CA. To create a server certificate for the registry service IP and the docker-registry.default.svc.cluster.local host name, run the following command from the first master listed in the Ansible host inventory file, by default /etc/ansible/hosts:

    1. $ oc adm ca create-server-cert \
    2.     --signer-cert=/etc/origin/master/ca.crt \
    3.     --signer-key=/etc/origin/master/ca.key \
    4.     --signer-serial=/etc/origin/master/ca.serial.txt \
    5.     --hostnames='docker-registry.default.svc.cluster.local,docker-registry.default.svc,172.30.124.220' \
    6.     --cert=/etc/secrets/registry.crt \
    7.     --key=/etc/secrets/registry.key

    If the router will be exposed externally, add the public route host name in the --hostnames flag:

    1. --hostnames='mydocker-registry.example.com,docker-registry.default.svc.cluster.local,172.30.124.220 \

    See Redeploying Registry and Router Certificates for additional details on updating the default certificate so that the route is externally accessible.

    The oc adm ca create-server-cert command generates a certificate that is valid for two years. This can be altered with the —expire-days option, but for security reasons, it is recommended to not make it greater than this value.

  4. Create the secret for the registry certificates:

    1. $ oc create secret generic registry-certificates \
    2.     --from-file=/etc/secrets/registry.crt \
    3.     --from-file=/etc/secrets/registry.key

  5. Add the secret to the registry pod’s service accounts (including the default service account):

    1. $ oc secrets link registry registry-certificates
    2. $ oc secrets link default  registry-certificates

    Limiting secrets to only the service accounts that reference them is disabled by default. This means that if serviceAccountConfig.limitSecretReferences is set to false (the default setting) in the master configuration file, linking secrets to a service is not required.

  6. Pause the docker-registry service:

    1. $ oc rollout pause dc/docker-registry

  7. Add the secret volume to the registry deployment configuration:

    1. $ oc volume dc/docker-registry --add --type=secret \
    2.     --secret-name=registry-certificates -m /etc/secrets

  8. Enable TLS by adding the following environment variables to the registry deployment configuration:

    1. $ oc set env dc/docker-registry \
    2.     REGISTRY_HTTP_TLS_CERTIFICATE=/etc/secrets/registry.crt \
    3.     REGISTRY_HTTP_TLS_KEY=/etc/secrets/registry.key

    See the Configuring a registry section of the Docker documentation for more information.

  9. Update the scheme used for the registry’s liveness probe from HTTP to HTTPS:

    1. $ oc patch dc/docker-registry -p '{"spec": {"template": {"spec": {"containers":[{
    2.     "name":"registry",
    3.     "livenessProbe":  {"httpGet": {"scheme":"HTTPS"}}
    4.   }]}}}}'

  10. If your registry was initially deployed on OKD 1.1.2 or later, update the scheme used for the registry’s readiness probe from HTTP to HTTPS:

    1. $ oc patch dc/docker-registry -p '{"spec": {"template": {"spec": {"containers":[{
    2.    "name":"registry",
    3.    "readinessProbe":  {"httpGet": {"scheme":"HTTPS"}}
    4.  }]}}}}'

  11. Resume the docker-registry service:

    1. $ oc rollout resume dc/docker-registry

  12. Validate the registry is running in TLS mode. Wait until the latest docker-registry deployment completes and verify the Docker logs for the registry container. You should find an entry for listening on :5000, tls.

    1. $ oc logs dc/docker-registry | grep tls
    2. time="2015-05-27T05:05:53Z" level=info msg="listening on :5000, tls" instance.id=deeba528-c478-41f5-b751-dc48e4935fc2

  13. Copy the CA certificate to the Docker certificates directory. This must be done on all nodes in the cluster:

    1. $ dcertsdir=/etc/docker/certs.d
    2. $ destdir_addr=$dcertsdir/172.30.124.220:5000
    3. $ destdir_name=$dcertsdir/docker-registry.default.svc.cluster.local:5000
    5. $ sudo mkdir -p $destdir_addr $destdir_name
    6. $ sudo cp ca.crt $destdir_addr    (1)
    7. $ sudo cp ca.crt $destdir_name
    1The ca.crt file is a copy of /etc/origin/master/ca.crt on the master.

  14. When using authentication, some versions of docker also require you to configure your cluster to trust the certificate at the OS level.

    1. Copy the certificate:

      1. $ cp /etc/origin/master/ca.crt /etc/pki/ca-trust/source/anchors/myregistrydomain.com.crt

    2. Run:

      1. $ update-ca-trust enable

  15. Remove the --insecure-registry option only for this particular registry in the /etc/sysconfig/docker file. Then, reload the daemon and restart the docker service to reflect this configuration change:

    1. $ sudo systemctl daemon-reload
    2. $ sudo systemctl restart docker

  16. Validate the docker client connection. Running docker push to the registry or docker pull from the registry should succeed. Make sure you have logged into the registry.

    1. $ docker tag|push <registry/image> <internal_registry/project/image>

    For example:

    1. $ docker pull busybox
    2. $ docker tag docker.io/busybox 172.30.124.220:5000/openshift/busybox
    3. $ docker push 172.30.124.220:5000/openshift/busybox
    4. ...
    5. cf2616975b4a: Image successfully pushed
    6. Digest: sha256:3662dd821983bc4326bee12caec61367e7fb6f6a3ee547cbaff98f77403cab55

Manually Exposing a Secure Registry

Instead of logging in to the OKD registry from within the OKD cluster, you can gain external access to it by first securing the registry and then exposing it with a route. This allows you to log in to the registry from outside the cluster using the route address, and to tag and push images using the route host.

  1. Each of the following prerequisite steps are performed by default during a typical cluster installation. If they have not been, perform them manually:

    1. Manually deploy the registry.

    2. Manually secure the registry.

    3. Manually deploy a router.

  2. A passthrough route should have been created by default for the registry during the initial cluster installation:

    1. Verify whether the route exists:

      1. $ oc get route/docker-registry -o yaml
      2. apiVersion: v1
      3. kind: Route
      4. metadata:
      5.   name: docker-registry
      6. spec:
      7.   host: <host> (1)
      8.   to:
      9.     kind: Service
      10.     name: docker-registry (2)
      11.   tls:
      12.     termination: passthrough (3)
      1The host for your route. You must be able to resolve this name externally via DNS to the router’s IP address.
      2The service name for your registry.
      3Specifies this route as a passthrough route.

      Re-encrypt routes are also supported for exposing the secure registry.

    2. If it does not exist, create the route via the oc create route passthrough command, specifying the registry as the route’s service. By default, the name of the created route is the same as the service name:

      1. Get the docker-registry service details:

        1. $ oc get svc
        2. NAME              CLUSTER_IP       EXTERNAL_IP   PORT(S)                 SELECTOR                  AGE
        3. docker-registry   172.30.69.167    <none>        5000/TCP                docker-registry=default   4h
        4. kubernetes        172.30.0.1       <none>        443/TCP,53/UDP,53/TCP   <none>                    4h
        5. router            172.30.172.132   <none>        80/TCP                  router=router             4h

      2. Create the route:

        1. $ oc create route passthrough    \
        2.     --service=docker-registry    \(1)
        3.     --hostname=<host>
        4. route "docker-registry" created     (2)
        1Specifies the registry as the route’s service.
        2The route name is identical to the service name.

  1. Next, you must trust the certificates being used for the registry on your host system to allow the host to push and pull images. The certificates referenced were created when you secured your registry.

    1. $ sudo mkdir -p /etc/docker/certs.d/<host>
    2. $ sudo cp <ca_certificate_file> /etc/docker/certs.d/<host>
    3. $ sudo systemctl restart docker

  2. Log in to the registry using the information from securing the registry. However, this time point to the host name used in the route rather than your service IP. When logging in to a secured and exposed registry, make sure you specify the registry in the docker login command:

    1. # docker login -e user@company.com \
    2.     -u f83j5h6 \
    3.     -p Ju1PeM47R0B92Lk3AZp-bWJSck2F7aGCiZ66aFGZrs2 \
    4.     <host>

  3. You can now tag and push images using the route host. For example, to tag and push a busybox image in a project called test:

    1. $ oc get imagestreams -n test
    2. NAME      DOCKER REPO   TAGS      UPDATED
    4. $ docker pull busybox
    5. $ docker tag busybox <host>/test/busybox
    6. $ docker push <host>/test/busybox
    7. The push refers to a repository [<host>/test/busybox] (len: 1)
    8. 8c2e06607696: Image already exists
    9. 6ce2e90b0bc7: Image successfully pushed
    10. cf2616975b4a: Image successfully pushed
    11. Digest: sha256:6c7e676d76921031532d7d9c0394d0da7c2906f4cb4c049904c4031147d8ca31
    13. $ docker pull <host>/test/busybox
    14. latest: Pulling from <host>/test/busybox
    15. cf2616975b4a: Already exists
    16. 6ce2e90b0bc7: Already exists
    17. 8c2e06607696: Already exists
    18. Digest: sha256:6c7e676d76921031532d7d9c0394d0da7c2906f4cb4c049904c4031147d8ca31
    19. Status: Image is up to date for <host>/test/busybox:latest
    21. $ oc get imagestreams -n test
    22. NAME      DOCKER REPO                       TAGS      UPDATED
    23. busybox   172.30.11.215:5000/test/busybox   latest    2 seconds ago

    Your image streams will have the IP address and port of the registry service, not the route name and port. See oc get imagestreams for details.

Manually Exposing a Non-Secure Registry

Instead of securing the registry in order to expose the registry, you can simply expose a non-secure registry for non-production OKD environments. This allows you to have an external route to the registry without using SSL certificates.

Only non-production environments should expose a non-secure registry to external access.

To expose a non-secure registry:

  1. Expose the registry:

    1. # oc expose service docker-registry --hostname=<hostname> -n default

    This creates the following JSON file:

    1. apiVersion: v1
    2. kind: Route
    3. metadata:
    4.   creationTimestamp: null
    5.   labels:
    6.     docker-registry: default
    7.   name: docker-registry
    8. spec:
    9.   host: registry.example.com
    10.   port:
    11.     targetPort: "5000"
    12.   to:
    13.     kind: Service
    14.     name: docker-registry
    15. status: {}

  2. Verify that the route has been created successfully:

    1. # oc get route
    2. NAME              HOST/PORT                    PATH      SERVICE           LABELS                    INSECURE POLICY   TLS TERMINATION
    3. docker-registry   registry.example.com            docker-registry   docker-registry=default

  3. Check the health of the registry:

    1. $ curl -v http://registry.example.com/healthz

    Expect an HTTP 200/OK message.

    After exposing the registry, update your /etc/sysconfig/docker file by adding the port number to the OPTIONS entry. For example:

    1. OPTIONS='--selinux-enabled --insecure-registry=172.30.0.0/16 --insecure-registry registry.example.com:80'

    The above options should be added on the client from which you are trying to log in.

    Also, ensure that Docker is running on the client.

When logging in to the non-secured and exposed registry, make sure you specify the registry in the docker login command. For example:

  1. # docker login -e user@company.com \
  2.     -u f83j5h6 \
  3.     -p Ju1PeM47R0B92Lk3AZp-bWJSck2F7aGCiZ66aFGZrs2 \
  4.     <host>