Kops and cluster autoscaler

Rationale TL;DR

You deployed Kops and now you want your cluster to autoscale

Solution

Cluster changes

First, set the max and min sizes for your clusters.

kops edit instancegroups nodes

Set the maxSize and minSize values accordingly.

After that, you would need to edit to edit the cluster config and set the following policy:

kind: Cluster
...
spec:
  additionalPolicies:
    node: |
      [
        {
          "Effect": "Allow",
          "Action": [
            "autoscaling:DescribeAutoScalingGroups",
            "autoscaling:DescribeAutoScalingInstances",
            "autoscaling:DescribeLaunchConfigurations",
            "autoscaling:SetDesiredCapacity",
            "autoscaling:TerminateInstanceInAutoScalingGroup",
            "autoscaling:DescribeTags"
          ],
          "Resource": "*"
        }
      ]
...

And finally, we’d need to update the cluster. Because we want to use Terraform, we need to do this:

kops update cluster  --name $CLUSTER_NAME --state s3://$BUCKET_NAME —target terraform —out .

After this, we need to apply the Terraform plans. Use your CI/CD, I encourage you to do so. Otherwise, you’d need to apply it manually.

Finally, do a rolling update:

kops rolling-update cluster  --name $CLUSTER_NAME --state s3://$BUCKET_NAME --yes

Cluster autoscaling

Once we have our cluster with max size set and the nodes with the right policies, we need to download the Cluster Autoscaler manifest:

https://raw.githubusercontent.com/kubernetes/autoscaler/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml

We need to change some values:

• “YOUR CLUSTER NAME”
• Also set the right version when here k8s.gcr.io/cluster-autoscaler:XX.XX.XX

This should be enough. Scaling a deployment with plenty of pods should trigger a new node.