Kubernetes is a powerful tool for container orchestration. But does it guarantee optimal resource allocation and cost savings? Not necessarily.
While running more workloads on the same server instance might seem cost-effective, it’s challenging to track the costs generated by different projects or teams within Kubernetes. Moreover, understanding whether you’re actually saving money with your cluster can be tricky.
However, there’s a strategy that can help: autoscaling. The more efficiently your Kubernetes scaling mechanisms are configured, the less waste and cost you’ll incur in running your application.
In this guide, we’ll explore how to leverage Kubernetes autoscaling mechanisms to minimize your cloud expenses.
Understanding Autoscaling in Kubernetes: Horizontal vs. Vertical
Horizontal Autoscaling
Horizontal autoscaling enables you to establish rules for initiating or terminating instances tied to a resource when they exceed or fall below set thresholds.
However, it has its limitations:
- It may necessitate designing the application with a scale-out approach in mind to distribute workloads across multiple servers.
- It may not always keep pace with unexpected demand surges as instances take a few minutes to load.
Vertical Autoscaling
Vertical autoscaling operates based on rules that modify the amount of CPU or RAM allocated to an existing instance.
But it also has its drawbacks:
- You’re constrained by the maximum CPU and memory limits for a single instance.
- There are connectivity ceilings for every underlying physical host due to network-related limitations.
- Some of your resources may be idle at times, and you’ll continue to pay for them.
Delving into Kubernetes Autoscaling Techniques
1. Horizontal Pod Autoscaler (HPA)
As the demands of your application fluctuate, you may need to add or remove pod replicas. The Horizontal Pod Autoscaler (HPA) is designed to automate this process for you.
To function properly, the HPA controller needs a source of metrics. For instance, when scaling based on CPU usage, it uses metrics-server. If you want to use custom or external metrics for HPA scaling, you need to deploy a service implementing the custom.metrics.k8s.io or external.metrics.k8s.io API.
The HPA is an excellent tool for scaling stateless applications, but it can also be used to support scaling stateful sets. For workloads that see regular changes in demand, use the HPA in combination with cluster autoscaling to reduce the number of active nodes when the number of pods decreases.
2. Vertical Pod Autoscaler (VPA)
The Vertical Pod Autoscaler (VPA) is a Kubernetes autoscaling method that adjusts the CPU and memory resource requests of pod containers to better align the allocated cluster resource with actual usage.
The VPA only replaces the pods managed by a replication controller, so it requires the Kubernetes metrics-server to function. It’s a good practice to use the VPA and HPA simultaneously if the HPA configuration doesn’t use CPU or memory to identify scaling targets.
3. Cluster Autoscaler
The Cluster Autoscaler modifies the number of nodes in a cluster. It operates at the infrastructure level, so it requires permissions to add and delete infrastructures. It’s crucial to manage these necessary credentials securely, adhering to the principle of least privilege.
To manage the costs of running Kubernetes clusters on a cloud platform, it’s smart to dynamically scale the number of nodes to match the current cluster utilization. This is especially true for workloads designed to scale and meet the current demand.
Best Practices for Kubernetes Autoscaling
- Ensure that HPA and VPA policies don’t conflict: Review your binning and packing density settings when designing clusters for business- or purpose-class tier of service.
- Use instance weighted scores: Use instance weighted scores when selecting the instance sizes and types that are suitable for autoscaling.
- Cut costs with mixed instances: A mixed-instance strategy leads to excellent availability and performance at a reasonable cost.
Can Kubernetes Autoscaling Be Further Automated?
Here’s a vendor-neutral rewrite of the paragraph:
In order to maintain stable workload operation during peak load and minimize costs during lower demand, teams often require a balanced combination of all three Kubernetes autoscaling methods. There are several cloud cost management tools available that offer advanced automation mechanisms to make autoscaling even more efficient.
For instance, some tools offer smooth autoscaling, ensuring that the number of nodes in use matches the application’s requirements at all times, scaling them up and down automatically.
There are also tools that provide a headroom policy. This feature comes into play when a pod suddenly requests more CPU or memory than the resources available on any of the worker nodes. The autoscaler in these tools matches the demand by maintaining a buffer of spare capacity.
Additionally, some platforms offer a spot fallback feature. This ensures that workloads designated for spot instances have the capacity to run even if they’re temporarily not available.
In our comprehensive guide, we delve into the best cloud cost management tools available in the market. Some of these tools, like CAST AI are recognized for having one of the best autoscalers in the market.