The Amazon EKS cluster upgrade utility
Project description
Amazon EKS Upgrade Utility
Amazon EKS cluster upgrade is a utility that automates the upgrade process for Amazon EKS clusters.
Process
The process for upgrading an Amazon EKS cluster using eksupgrade
consists of primarily of three parts:
- Perform pre-flight checks prior to upgrading the cluster
- Upgrade the cluster
- Evaluate the cluster after upgrade
Pre-Flight Checks
There are a number of version compatibility constraints, health checks, etc., before a cluster can successfully be upgraded. eksupgrade
performs the following pre-flight checks:
- Target Version Compatibility Check - Since any cluster in EKS is always allowed to upgrade to one version above and not beyond, a check for the target version is done as with each upgrade there are a lot of configuration changes and upgrading directly to a higher version can lead to breakdown of the services being provided by it.
- Customer Management Key - A cluster might have a CMK Key associated with it, so it is essential to verify if the same exists in user's account before carrying out the upgrade.
- Security Group - Every cluster has a security group associated with it to restrict and allow the flow of traffic across it, and therefore it has to be verified whether it exists in the user's VPC or not.
- Node group and worker node detail - EKS cluster supports multiple types of node groups. So for the purpose of upgrade, their kubelet version compatibility has to be checked to proceed with the upgrade step.
- Subnets - A minimum of 4-5 free IPs are required when doing a cluster upgrade to launch new nodes and node groups with the old ones to keep the services of the cluster running while the upgrade is going on. So check on the existence of the free IPs is performed.
- Cluster Roles - There are a lot of important cluster roles required during the upgrade related to addons, nodes, and other components of the cluster without which the cluster upgrade cannot be executed successfully.
- Pod Security Policy - EKS privileged role has to be checked to be present with the current pod security policy. (deprecated in Kubernetes v1.21, and removed from Kubernetes in v1.25)
- Cluster addons - The cluster addons like kube-proxy, VPC CNI and CoreDNS are essential for running various services across the cluster. The parameters available on these addons which are customized by the users on the target cluster have to be captured while upgrading so that they are to added back to maintain service availability.
- Pod Disruption Budget - The existence of PDB has to be checked in the cluster, and minimum and maximum available with it has to be taken into account while upgrading.
- Horizontal Pod and Cluster Autoscaler - As the other components are upgraded to the compatible image version, a check is performed to see if Cluster or Horizontal Pod Autoscaler are present. They are reviewed to upgrade to a compatible version with respect to the control plane.
Cluster Upgrade
- Control plane upgrade - This is handled entirely by AWS once the version upgrade has been requested.
- Identification of Managed and Self-managed node - The worker nodes are identified as EKS managed and Self-managed to perform upgrade.
- Managed Node group update - Updates managed node group to the specified version.
- Self-managed Node group update
- Launch new nodes with upgraded version and wait until they require ready status for next step.
- Mark existing nodes as unschedulable.
- If pod disruption budget (PDB) is present then check for force eviction flag (--force) which is given by user, only then evict the pods or continue with the flow.
Pre-Requisites
Before running eksupgrade
, you will need to have permission for both AWS and the Kubernetes cluster itself.
- Install
eksupgrade
locally:
pip install eksupgrade
- Ensure you have the necessary AWS permissions; an example policy of required permissions is listed below:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "iam",
"Effect": "Allow",
"Action": [
"iam:GetRole",
"sts:GetAccessKeyInfo",
"sts:GetCallerIdentity",
"sts:GetSessionToken"
],
"Resource": "*"
},
{
"Sid": "ec2",
"Effect": "Allow",
"Action": [
"autoscaling:CreateLaunchConfiguration",
"autoscaling:Describe*",
"autoscaling:SetDesiredCapacity",
"autoscaling:TerminateInstanceInAutoScalingGroup",
"autoscaling:UpdateAutoScalingGroup",
"ec2:Describe*",
"ssm:*"
],
"Resource": "*"
},
{
"Sid": "eks",
"Effect": "Allow",
"Action": [
"eks:Describe*",
"eks:List*",
"eks:UpdateAddon",
"eks:UpdateClusterVersion",
"eks:UpdateNodegroupVersion"
],
"Resource": "*"
}
]
}
- Update your local kubeconfig to authenticate to the cluster:
aws eks update-kubeconfig --name <CLUSTER-NAME> --region <REGION>
Usage
To view the arguments and options, run:
eksupgrade --help
usage: eksupgrade [-h] [--pass_vpc] [--max_retry MAX_RETRY] [--force]
[--preflight] [--disable-checks] [--parallel]
[--latest-addons] [--log-level LOG_LEVEL] [--version]
name version region
Amazon EKS cluster upgrade
positional arguments:
name Cluster Name
version new version which you want to update
region The AWS region where the cluster resides
optional arguments:
-h, --help show this help message and exit
--pass_vpc this --pass-vpc will skip the vpc cni upgrade
--max_retry MAX_RETRY
you can specify max retry or else by default it is 2
--force force pod eviction when you have pdb
--preflight Run pre-flight check without upgrade
--disable-checks Totally disable the pre-flight and post-flight
checks during upgrade scenarios
--parallel Upgrade all nodegroups in parallel
--latest-addons Upgrade addons to the latest eligible version
instead of default
--log-level LOG_LEVEL
The log level to be displayed in the console.
Default to: INFO
--version show program's version number and exit
example:
eksupgrade <name> <version> <region>
Force pod eviction when you have PDB (Pod Disruption Budget):
eksupgrade <name> <version> <region>n --force
Skip VPC CNI upgrade:
eksupgrade <name> <version> <region> --pass_vpc
Skip upgrade workflow:
eksupgrade <name> <version> <region> --preflight
Set log level to console (default to INFO):
eksupgrade <name> <version> <region> --log-level debug
Display the eksupgrade version:
eksupgrade --version
Security
See CONTRIBUTING for more information.
License
This library is licensed under the MIT-0 License. See the LICENSE file.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for eksupgrade-0.7.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cd1cdad71fdc9089897fd14a60be529b125554a62ad582e4eb66eaebcf6b2f32 |
|
MD5 | 2cec4088a76fae2eb3627531eae4ed97 |
|
BLAKE2b-256 | 4651c60505ac5f72bf57f292e0811d95ba3686ad0d24a08e91d69a9e4a3626ba |