About Domino
Domino Data LabKnowledge BaseData Science BlogTraining

Google Kubernetes Engine (GKE) upgrade guide

This topic describes how to upgrade Kubernetes in your Google Kubernetes Engine (GKE) Domino deployment. GKE is hosted on Google Cloud Platform (GCP).

Important

Immediately after the Kubernetes upgrade, you must upgrade Domino to a compatible version of Kubernetes. Domino will not work as expected until this is completed.

For example, after upgrading Kubernetes to v1.22, you must upgrade to Domino v5.2 or later. Kubernetes v1.22 is not compatible with older versions of Domino. Similarly, after upgrading to Kubernetes 1.23 or 1.24, you must upgrade to Domino v5.3 or later.

Prerequisites

To upgrade Kubernetes in GKE, you must have a Unix or Linux terminal with the following:

Use environment variables to set the values of IDs, names, and labels. This simplifies the commands you’ll run while upgrading:

export CLUSTER_PROJECT=<The project name of the cluster>
export CLUSTER_NAME=<The name of the cluster>
export CLUSTER_REGION=<The region where the cluster is located>
Get cluster credentials

  1. If you aren’t already signed in with gcloud, run, gcloud init.

  2. To protect your current config file from being overwritten, run export KUBECONFIG=$(pwd)/kubeconfig.

  3. Get the remote kubeconfig:

    gcloud container clusters get-credentials $CLUSTER_NAME --region $CLUSTER_REGION --project $CLUSTER_PROJECT
Disable the disruption budgets

You must temporarily disable the PodDisruptionBudget (PDB) for certain services. To do this, back up the PDBs, delete the original PDBs, then update Kubernetes and restore the PDBs from the backup after the update is complete.

  1. Backup the PDBs:

    kubectl get pdb -n domino-platform -o yaml mongodb-arbiter > mongodb-arbiter.yml
kubectl get pdb -n domino-platform -o yaml mongodb-primary > mongodb-primary.yml
kubectl get pdb -n domino-platform -o yaml mongodb-secondary > mongodb-secondary.yml

  2. Delete the PDBs:

    kubectl delete pdb -n domino-platform mongodb-arbiter
kubectl delete pdb -n domino-platform mongodb-primary
kubectl delete pdb -n domino-platform mongodb-secondary
Determine upgrade version

  1. Get available GKE versions

    gcloud container get-server-config
  --region=${CLUSTER_REGION}
  --flatten="channels"
  --format="yaml(channels.channel,channels.validVersions)"

  2. Get current master version

gcloud container clusters describe ${CLUSTER_NAME} --region=${CLUSTER_REGION} --format="value(currentMasterVersion)"
Note
 You can only upgrade by one minor version at a time. For example, if you are on version Kubernetes version 1.22, you can upgrade to version 1.23.
Update the cluster

  1. Upgrade the control plane:

    gcloud container clusters upgrade $CLUSTER_NAME --project $CLUSTER_PROJECT --master --region $CLUSTER_REGION --cluster-version <version-number>
    Note
    		 The previous step might take several minutes to complete. Monitor the GCP console for the status of the cluster upgrade.

  2. Upgrade the node pools:

    gcloud container clusters upgrade $CLUSTER_NAME --project $CLUSTER_PROJECT --region $CLUSTER_REGION --node-pool=platform
gcloud container clusters upgrade $CLUSTER_NAME --project $CLUSTER_PROJECT --region $CLUSTER_REGION --node-pool=compute
gcloud container clusters upgrade $CLUSTER_NAME --project $CLUSTER_PROJECT --region $CLUSTER_REGION --node-pool=gpu
Restore the disruption budgets

Run these commands from the same folder as the backup:

kubectl apply -f mongodb-arbiter.yml
kubectl apply -f mongodb-primary.yml
kubectl apply -f mongodb-secondary.yml