How to upgrade Kubernetes Cluster - Step by Step Guide

Share on Social Media

Upgrade Kubernetes Cluster 1.34.0 to 1.35.0 like a CKA pro! Step-by-step guide with zero-downtime strategy. Don’t get left behind—master production-grade upgrades today at CentLinux. #Kubernetes #Orchestration #CentLinux

Kubernetes Upgrade from 1.34.0 to 1.35.0: A Step-by-Step Guide

As a Certified Kubernetes Administrator (CKA), one of the most critical tasks you’ll face is upgrading your Kubernetes cluster. In this comprehensive guide, I’ll walk you through upgrading a Kubernetes cluster from version 1.34.0 to 1.35.0 using kubeadm, with minimal downtime and proper node draining procedures.

How to upgrade Kubernetes Cluster - Step by Step Guide — How to upgrade Kubernetes Cluster – Step by Step Guide

Prerequisites

Before we begin, ensure you have:

SSH access to all nodes in your cluster
Root or sudo privileges on all nodes
A backup of your critical data and etcd (recommended)

Upgrade Strategy Overview

We’ll follow the recommended approach:

Upgrade the control plane node first
Drain and upgrade worker nodes one at a time
Maintain application availability by rescheduling pods

Let’s dive into the step-by-step process.

Note: You can purchase Kubernetes best books from CentLinux online store at discounted price.

Part 1: Upgrading the Control Plane Node

Step 1: Drain the Control Plane Node

First, we need to drain the control plane node to ensure no workloads are running during the upgrade:

kubectl drain controlplane --ignore-daemonsets

Why this step? Draining evicts all pods from the node except daemonsets. This ensures that your workloads are rescheduled to other nodes, preventing downtime. The --ignore-daemonsets flag is necessary because daemonsets are node-specific and cannot be moved.

Step 2: Update the Kubernetes Repository

The repository needs to reflect the new version. Open the apt source list file:

vi /etc/apt/sources.list.d/kubernetes.list

Update the repository URL to point to the version that supports Kubernetes 1.35.0. For Ubuntu/Debian systems, the entry might look like:

deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.35/deb/ /

Step 3: Update Package Lists and Verify Available Versions

Refresh your package cache and check available kubeadm versions:

sudo apt update
sudo apt-cache madison kubeadm

This command shows all available versions. Look for 1.35.0-1.1 or similar and copy the exact version string for later use.

Step 4: Install kubeadm 1.35.0

Unhold kubeadm (if it was held to prevent automatic updates), install the new version, then hold it again:

sudo apt-mark unhold kubeadm && \
sudo apt-get update && sudo apt-get install -y kubeadm='1.35.0-1.1' && \
sudo apt-mark hold kubeadm

The apt-mark hold prevents automatic updates that could break your cluster.

Step 5: Verify kubeadm Version

Confirm the installation was successful:

kubeadm version

You should see output indicating version 1.35.0.

Step 6: Plan the Upgrade

Before applying the upgrade, verify the upgrade plan:

sudo kubeadm upgrade plan

This command checks if your cluster is upgradable and displays a detailed plan of what will happen during the upgrade.

Step 7: Apply the Upgrade

Execute the control plane upgrade:

sudo kubeadm upgrade apply v1.35.0

Important: This command applies the upgrade to the control plane. It updates the etcd, API server, controller manager, and scheduler components.

Step 8: Upgrade kubelet and kubectl

Now upgrade the kubelet and kubectl packages:

sudo apt-mark unhold kubelet kubectl && \
sudo apt-get update && sudo apt-get install -y kubelet='1.35.0-1.1' kubectl='1.35.0-1.1' && \
sudo apt-mark hold kubelet kubectl

Step 9: Restart kubelet

Reload systemd and restart kubelet to apply the new version:

sudo systemctl daemon-reload
sudo systemctl restart kubelet

Step 10: Uncordon the Control Plane

Make the control plane node schedulable again:

kubectl uncordon controlplane

Why this step? Uncordoning allows pods to be scheduled on this node again after the upgrade is complete.

Part 2: Upgrading Worker Node (node01)

Step 11: Drain node01

Drain the worker node, ensuring gold-nginx pods are rescheduled elsewhere:

kubectl drain node01 --ignore-daemonsets

Step 12: SSH into node01

Connect to the worker node for the upgrade:

ssh node01

Steps 13-15: Update Repository and kubeadm

Perform the same repository update and kubeadm installation as on the control plane:

vi /etc/apt/sources.list.d/kubernetes.list
# Update repository to v1.35

sudo apt update
sudo apt-cache madison kubeadm

sudo apt-mark unhold kubeadm && \
sudo apt-get update && sudo apt-get install -y kubeadm='1.35.0-1.1' && \
sudo apt-mark hold kubeadm

Step 16: Verify and Upgrade

Check kubeadm version and upgrade the node:

kubeadm version
sudo kubeadm upgrade node

Note: The kubeadm upgrade node command is used for worker nodes, while kubeadm upgrade apply is used only on the control plane.

Steps 17-19: Upgrade kubelet and Restart

Upgrade kubelet and kubectl, then restart the service:

sudo apt-mark unhold kubelet kubectl && \
sudo apt-get update && sudo apt-get install -y kubelet='1.35.0-1.1' kubectl='1.35.0-1.1' && \
sudo apt-mark hold kubelet kubectl

sudo systemctl daemon-reload
sudo systemctl restart kubelet

Step 20: Exit from node01

Return to the control plane node:

exit

Step 21: Uncordon node01

Make the worker node schedulable again:

kubectl uncordon node01

Step 22: Verify the Cluster

Check the status of all nodes:

kubectl get nodes

You should see all nodes with the updated Kubernetes version. The STATUS column should show Ready for all nodes.

Verification and Validation

After completing the upgrade, verify that:

All nodes show the correct Kubernetes version (v1.35.0)
All pods are running and healthy
The gold-nginx deployment has been successfully rescheduled
Cluster functionality is working as expected

kubectl get nodes -o wide
kubectl get pods -A
kubectl get deployment gold-nginx

Important Considerations for Blog Readers

Downtime Minimization

By draining nodes one at a time, we ensure that the gold-nginx deployment is always running on at least one node. This significantly reduces application downtime.

Version Compatibility

Always check the official Kubernetes release notes for version-specific requirements and breaking changes before upgrading.

Backup Strategy

For production environments, consider:

Taking etcd snapshots before major upgrades
Creating backups of critical manifests
Testing the upgrade in a staging environment first

Rolling Back

If something goes wrong, Kubernetes supports downgrades, but only within a limited version range. Always test thoroughly.

Conclusion

You’ve successfully upgraded your Kubernetes cluster from 1.34.0 to 1.35.0 with minimal downtime. This systematic approach ensures high availability and follows CKA best practices for cluster upgrades.

The key takeaways:

Always upgrade control plane nodes first
Drain nodes before upgrading to reschedule workloads
One node at a time minimizes risk
Verify each step before proceeding

FAQs

1. What happens if I skip draining a node before upgrading?

Answer: Skipping the drain step can lead to application downtime and data loss. When you upgrade Kubernetes components without draining, running pods may be terminated abruptly, causing service interruptions. More critically, stateful applications with persistent volumes could become corrupted if processes are killed mid-operation.

The drain command (kubectl drain) performs three essential functions:

Evicts all pods (except daemonsets) gracefully
Marks the node as unschedulable (NoSchedule taint)
Allows pods to respect their terminationGracePeriodSeconds

Pro Tip from CentLinux: Always use --ignore-daemonsets flag during drain, as daemonsets are node-specific and cannot be relocated. For critical applications, consider using PodDisruptionBudgets (PDBs) to ensure a minimum number of replicas remain available during node maintenance.

2. Can I skip upgrading kubeadm on worker nodes?

Answer: Absolutely not. kubeadm is the core tool that manages cluster certificates, configuration, and node joining. While the control plane uses kubeadm upgrade apply, worker nodes require kubeadm upgrade node to:

Update the node’s certificate configuration
Sync the cluster configuration with the new version
Ensure the node can properly communicate with the upgraded control plane

If you skip this step, your worker nodes may fail to join the cluster after the control plane upgrade, or worse, cause version mismatches that lead to unpredictable behavior. Always follow the sequence: control plane → worker nodes.

3. How do I handle the upgrade if I have multiple worker nodes?

Answer: The same pattern applies to all worker nodes—upgrade them one at a time:

Drain the node (kubectl drain node0X)
SSH into the node
Upgrade kubeadm, then run kubeadm upgrade node
Upgrade kubelet/kubectl and restart
Exit and uncordon the node

Parallel upgrade vs. Sequential:

Sequential (Recommended): Safer, allows you to verify each node before moving to the next
Parallel (Not recommended): Faster but riskier—if something goes wrong, multiple nodes could be affected

Quick Tip for Large Clusters: Create a maintenance window and test the upgrade on a single worker node first. If successful, proceed with the remaining nodes one by one.

4. What should I do if the upgrade fails mid-way?

Answer: Don’t panic! Here’s a systematic troubleshooting approach:

Immediate Steps:

Check kubelet status:

sudo systemctl status kubelet

Check kubeadm version to confirm installation.

kubeadm version

Verify API server: check if the control plane is responsive.

kubectl get nodes

Check real-time logs.

journalctl -u kubelet -f

Common Failure Scenarios & Solutions:

Issue	Solution
Version mismatch error	Ensure you’re using exact version strings (e.g., `1.35.0-1.1`)
Network connectivity issues	Verify DNS and network connectivity between nodes
Certificate errors	Run `sudo kubeadm init phase upload-certs --upload-certs` to refresh
Control plane not ready	Wait 1-2 minutes for components to restart
APT repository not found	Double-check the source list URL for v1.35

Rollback Strategy:

Kubernetes supports downgrades within one minor version (1.35 → 1.34)
Reinstall previous version packages and restart kubelet
For critical failures, restore etcd from backup

CentLinux Recommendation: Always take an etcd snapshot before any upgrade:

sudo etcdctl snapshot save /var/lib/etcd/snapshot-$(date +%Y%m%d).db

5. Why does the `gold-nginx` deployment need to be rescheduled before upgrades?

Answer: The gold-nginx deployment represents your critical production workloads, and the rescheduling strategy ensures high availability during the upgrade process.

The Logic Behind Node Draining:

Node Status	Action	Impact on gold-nginx
Before draining control plane	Pods are running on controlplane node	Application is serving traffic
Control plane drained	Pods evicted and rescheduled to node01	Traffic shifts seamlessly
Control plane upgrading	Control plane offline for ~5-10 minutes	Application still running on node01
Control plane uncordoned	Node becomes schedulable again	Pods may return to control plane
node01 drained	Pods evicted and rescheduled to controlplane	Application continues uninterrupted

Why This Matters:

Zero downtime: Users never experience service interruption
Rolling updates: Each node gets upgraded while workloads run elsewhere
Risk mitigation: If one node fails during upgrade, applications are still available

Best Practice: For production environments, implement a replica count of at least 2 for gold-nginx to ensure even if one node fails, the deployment remains operational. Then use pod anti-affinity rules to spread replicas across different nodes.

This tutorial was brought to you by CentLinux – your trusted source for Linux and Kubernetes tutorials. Follow our blog for more hands-on guides and technical deep-dives.

How to upgrade Kubernetes Cluster – Step by Step Guide

Table of Contents