Managing resources in Kubernetes clusters
We were writing about configuration management tools in some of the previous blog posts, so let’s continue with the hype train and mix some Kubernetes for the flavor. So, in a nutshell, we will write about Managing resources in Kubernetes.
This post is not about Kubernetes control plane configuration and management. It’s instead a short introduction to managing microservices or applications with Helm and adopting GitOps in managing them.
While most people associate Helm with application development, it can also be used in Ops. Stick around, and let’s explore the basics of what Helm is and some of its use cases.
What is Helm?
Helm is a tool that helps you manage Kubernetes applications. As simple as that.
Typically you would define any application or configuration within the Kubernetes cluster as YAML files published to Kubernetes API. Managing multiple versions of the same application with different configurable options can quickly become a chore. You would have to rewrite (search/replace) some critical components of those YAML definitions, which can become error-prone.
Helm is here to help you with templating and packaging those YAML files into charts that can be published into repositories, reused, and reconfigured based on options you wish to expose as configurable. If you wish, think of it like the Puppet module or Ansible role, but for your application in Kubernetes.
To get started with Helm, you will need to install it on your local machine by following these instructions.
What can Helm do for you?
As mentioned, Helm is a tool that offers multiple options.
Creating and helping you manage local charts
helm create myhelm-chart helm template myhelm-chart helm lint myhelm-chart
A chart is a collection of resources and dependencies that define your app in the Kubernetes cluster. Helm can help you create a base skeleton structure for your chart, as well as some predefined examples, linting your changes and rendering templates into YAML definitions that would be published to the cluster.
Package charts into archives
helm package myhelm-chart
This command will create a versioned tgz archive which you can later publish into the Helm repository. Think of it as a point-in-time snapshot of the current version of the Helm chart.
Interact with chart repositories
helm repo add repo-name repo-url helm repo list helm search repo repo-name
When you add remote repositories, you can search through them for packaged applications, version history, chart information, configurable values, etc.
(Un)Install and list applications in existing clusters
helm install release-name chart-name helm list helm uninstall release-name
You can install local charts or charts from repositories added in previous steps. The examples above will install charts in the default namespace defined in your active context with chart default options.
Manage application lifecycle that is installed with Helm
helm upgrade release-name chart-name helm history release-name helm rollback release-name revision
Helm keeps a revision history within the Kubernetes cluster for each installed application. You can easily find applications installed with Helm, upgrade them to new chart versions, get a history of releases and roll back broken deployments.
What does the chart look like
For the sake of brevity, we won’t get in-depth on chart structure, as it is very well defined in the Getting Started guide on the official pages.
Suffice to say that at its most basic, charts will need to contain a Chart.yaml file describing the chart, its version, maintainers, etc., values.yaml file will hold default values for configurable options used in chart templates. The templates/ directory within the chart will contain all the templated YAML files that will generate definitions of application resources and dependencies used in deploying to the cluster.
Template files are just regular Kubernetes YAML files enriched with a gotpl templating engine.
What can you do with Helm about Managing resources in Kubernetes?
The first and most obvious idea that comes to mind is installing apps!As mentioned before, Helm offers you configurable/template YAML files with some “programmable” logic within its templating language. You can then write configurable and reusable application deployment definitions that you can use in multiple projects/configurations scenarios by just modifying its values.
From a developer’s perspective, think of it as an app install script where you can modify your end-user domain, app names, etc.
There are also many pre-packaged open source web apps available on public Helm repositories like Bitnami or ArtifactHub.
So, how do we do managing resources in Kubernetes?
Since we mostly deal with operations, I’ll focus more on the Ops perspective. In most cases, when you install a Kubernetes cluster, you get just the bare bones. To expose and run your user-facing applications in a cluster, you will probably need at least some add-on software. The first thing that comes to mind is Ingress controllers like Haproxy, Nginx, etc., allowing you to route traffic to your application inside the cluster. Once you have Ingress covered, you will probably wish to know how the application and cluster perform. For this purpose, you will need monitoring add-ons like Prometheus, Grafana, Loki, etc. To have data persist your volatile cluster nature, you will wish to use PersistentVolumes, and that will require CSI drivers based on your Cloud provider of choice, for example, AWS-EBS or AWS-EFS.
All those add-ons are packaged as Helm charts by upstream vendors. You can reuse those charts across any of your Kubernetes clusters to have them bootstrapped and configured in no time.
Depending on the project, clusters will also have different requirements for add-on software. Nonetheless, having a unified way to deploy any of those add-ons and manage their lifecycle, with a possibility to roll them back in case of troubles quickly, is a great tool to have in your arsenal.
Packaging the packages for managing resources in Kubernetes
What if I require a collection of charts installed given clusters, each with different versions, values, etc.
One great tool to add to your collection is helmfile. To borrow from the project page.
Helmfile is a declarative spec for deploying helm charts. It lets you…
- Keep a directory of chart value files and maintain changes in version control.
- Apply CI/CD to configuration changes.
- Periodically sync to avoid skew in environments.
This tool comes in very handy when you wish to manage multiple system charts within the cluster via CI pipelines from GitLab. It enables you to have collections of ops software and its configuration defined via helmfile and executed via CI pipelines.
When connecting GitLab with Kubernetes clusters, you can define one project per cluster for management. This GitLab project is a perfect place for housing helmfile configuration. Having this in the repository gives you the history of software versions and their values.
All you need to do is define two simple stages in your project’s .gitlab-ci.yaml
diff: stage: build extends: .runner environment: name: production script: - helmfile --file $CI_PROJECT_DIR/helmfile.yml diff rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH apply: stage: deploy extends: .runner environment: name: production script: - kubectl get namespaces - helmfile --file $CI_PROJECT_DIR/helmfile.yml apply --suppress-secrets rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH when: manual
And it will give you a preview of changes and manual action to deploy if you are happy with the changes. You can still manage your custom and private Helm charts as separate projects and publish them in GitLab’s package registry.
Deploying and managing clusters in such a manner enables you to have reproducible clusters. Not only can you quickly rebuild your cluster from scratch if need be, but you can build a new one that is an exact copy. This ability on its own can be a great advantage for testing newer software versions or cluster upgrades before hitting your production clusters.
The previous statement assumes you have a control plane managed via GitOps, but more on that in future posts.
Conclusion about managing resources in Kubernetes
If you are new to deploying your application to Kubernetes clusters, I would highly recommend familiarizing yourself with Helm as soon as possible.
Or, if you wish to adopt GitOps for Kubernetes cluster management fully, Helm can be a great tool in helping you automate some of the add-on ops software deployments.
Suppose you are ops, supporting multiple dev teams on a single cluster. In that case, you can also use Helm to create charts published towards developers for automating application metrics collection and alerting to a different team.
All in all, Helm is an excellent tool for application deployments in Kubernetes. I hope we have given you some idea of why we think it should be a standard part of any DevOps toolkit. If you have any other great applications, be sure to post them in the comments section. And as always, stay tuned for updates in future posts.
