Upgrade Calico on OpenShift 4

About upgrading Calico

This page describes how to upgrade to v3.27 for OpenShift 4 from an existing Calico cluster.

Host Endpoints

caution

If your cluster has host endpoints with interfaceName: * you must prepare your cluster before upgrading. Failure to do so will result in an outage.

In versions of Calico prior to v3.14, all-interfaces host endpoints (host endpoints with interfaceName: *) only supported pre-DNAT policy. The default behavior of all-interfaces host endpoints, in the absence of any policy, was to allow all traffic.

Beginning from v3.14, all-interfaces host endpoints support normal policy in addition to pre-DNAT policy. The support for normal policy includes a change in default behavior for all-interfaces host endpoints: in the absence of policy the default behavior is to drop traffic. This default behavior is consistent with “named” host endpoints (which specify a named interface such as “eth0”); named host endpoints drop traffic in the absence of policy.

Before upgrading to v3.27, you must ensure that global network policies are in place that select existing all-interfaces host endpoints and explicitly allow existing traffic flows. As a starting point, you can create an allow-all policy that selects existing all-interfaces host endpoints. First, we’ll add a label to the existing host endpoints. Get a list of the nodes that have an all-interfaces host endpoint:

calicoctl get hep -owide | grep | awk '"print $1"'

With the names of the all-interfaces host endpoints, we can label each host endpoint with a new label (for example, host-endpoint-upgrade: “”):

calicoctl get hep -owide | grep '*' | awk '"print $1"' \| xargs -I {} oc exec -i -n kube-system calicoctl -- /calicoctl label hostendpoint {} host-endpoint-upgrade=

Now that the nodes with an all-interfaces host endpoint are labeled with host-endpoint-upgrade, we can create a policy to log and allow all traffic going into or out of the host endpoints temporarily:

cat > allow-all-upgrade.yaml <<EOF
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: allow-all-upgrade
spec:
  selector: has(host-endpoint-upgrade)
  types:
  - Ingress
  - Egress
  ingress:
  - action: Log
  - action: Allow
  egress:
  - action: Log
  - action: Allow
EOF

Apply the policy:

calicoctl apply -f - < allow-all-upgrade.yaml

After applying this policy, all-interfaces host endpoints will log and allow all traffic through them. This policy will allow all traffic not accounted for by other policies. After upgrading, please review syslog logs for traffic going through the host endpoints and update the policy as needed to secure traffic to the host endpoints.

Upgrading Calico on OpenShift 4

Make a manifests directory.

mkdir manifests

Download the Calico manifests for OpenShift and add them to the generated manifests directory:

mkdir calico
wget -qO- https://github.com/projectcalico/calico/releases/download/v3.27.4/ocp.tgz | tar xvz --strip-components=1 -C calico
cp calico/* manifests/

Apply the updated manifests.

oc apply -f manifests/

You can now monitor the upgrade progress with the following command:

watch oc get tigerastatus

If you were upgrading from a version of Calico prior to v3.14 and followed the pre-upgrade steps for host endpoints above, review traffic logs from the temporary policy, add any global network policies needed to allow traffic, and delete the temporary network policy allow-all-upgrade.

Migrating to auto host endpoints

caution

Auto host endpoints have an allow-all profile attached which allows all traffic in the absence of network policy. This may result in unexpected behavior and data.

In order to migrate existing all-interfaces host endpoints to Calico-managed auto host endpoints:

1. Add any labels on existing all-interfaces host endpoints to their corresponding OpenShift nodes. Calico manages labels on automatic host endpoints by syncing labels from their nodes. Any labels on existing all-interfaces host endpoints should be added to their respective nodes. For example, if your existing all-interface host endpoint for node node1 has the label environment: dev, then you must add that same label to its node:

   oc label node node1 environment=dev

2. Enable auto host endpoints by following the enable automatic host endpoints how-to guide. Note that automatic host endpoints are created with a profile attached that allows all traffic in the absence of network policy.

   calicoctl patch kubecontrollersconfiguration default --patch ={"spec": {"controllers": {"node": {"hostEndpoint": {"autoCreate": "Enabled"}}}}}

3. Delete old all-interfaces host endpoints. You can distinguish host endpoints managed by Calico from others in several ways. First, automatic host endpoints have the label projectcalico.org/created-by: calico-kube-controllers. Secondly, automatic host endpoints’ name have the suffix -auto-hep.

   calicoctl delete hostendpoint <old_hostendpoint_name>