Microservices and containerisation are the hot topics of cloud computing. To put it simply, the microservices architecture allows independent deployment of a collection of loosely coupled scalable lightweight services, commonly used by software development organisations as a means of auxiliary income. Containers are the package in which microservices applications are deployed. They encapsulate all the essentials of running an application - binaries, libraries, configuration files and dependencies, essentially forming a fully portable computing environment.
Kubernetes (K8s) is undoubtedly the most popular open-source engine for automating deployment, scaling, and management of containerised applications. In this article, we discuss a popular feature offered by K8s for the deployment container applications - DaemonSet.
What is a DaemonSet?
Before we tackle DaemonSets, here is a brief introduction to some terminology used in K8s.
Cluster: A cluster consists of a set of nodes that run containerised applications. Note that a cluster likely runs multiple nodes in production environments to ensure fault tolerance and high availability.
Nodes: A node is a worker machine.
Pods: Every cluster contains at least one worker node. The worker nodes host Pods - the components of the application workload.
Kubernetes offers three methods to deploy Pods:
- Deployments, which is typically used for stateless applications
- StatefulSets, which is typically used for stateful applications
- DaemonSets, which used when one pod needs to run on all (or a subset of) nodes
What are Some Typical Applications of DaemonSet?
DaemonSets improve cluster performance by deploying Pods that perform general maintenance and service tasks to every node. Typically use cases for DaemonSets include:
- Running a cluster storage daemon on each node, such as:
- Running a logs collection daemon on every node, such as:
- Running a node monitoring daemon on every node, such as:
- Prometheus Node Exporter
- Datadog Agent
- Running infrastructure-level Pods for system operations.
How to Create a DaemonSet?
DaemonSets is configured using a YAML file.
kubectl apply -f https://k8s.io/examples/controllers/daemonset.yaml
As with other k8s configurations, the following parameters is required while creating a Daemonset:
A DaemonSet also needs a .spec section. Here, .spec.template, which species a pod template, is one of the required fields. Ensure that the Pod Template in a DaemonSet has a RestartPolicy set to default or Always only.
How to Deploy a Pod in a Subset of Nodes Rather Than all Nodes?
There are two ways to achieve this:
- Specifying .spec.template.spec.nodeSelector will allow you to create Pods only in those nodes that match with the Node Selector.
- Specifying .spec.template.spec.affinity will allow you to create Pods in only those nodes that match affinity.
How to Communicate with Daemon Pods?
Here are four commonly used design patterns to communicate with Daemon Pods:
- Push pattern: Here, the Pods send information to other services (like elasticSearch or a stats database). They do not receive any data.
- NodeIP and Known Port pattern: Here, pods are reachable via the node IPs using hostPort. Clients can use the known ports and node IPs to communicate with the pods.
- DNS Pattern: Here, you create a headless service first, and then discover DaemonSets using the endpoints resource.
- Service pattern: If it’s not necessary for you to reach a specific node, you can create a service to reach a Daemon on a random node.
How to Update a DaemonSet?
Changing node labels will prompt the DaemonSet to add Pods to newly matching nodes and delete Pods from newly not-matching nodes.
There are two strategies you can use to update a DaemonSet:
- OnDelete: Here, new DaemonSet pods will only be created when you manually delete old DaemonSet pods.
- RollingUpdate: This is the default strategy for Kubernetes 1.6 and above. After you update a DaemonSet template, old DaemonSet pods are destroyed, and new ones will be created automatically. To enable this feature set .spec.updateStrategy.type.
To delete a DaemonSet, run kubectl delete fluentd-es-demo. If you want to delete the DaemonSet without deleting the pods, add cascade=false with kubectl.
Important Things to Note Before you get Started
- You must specify a pod selector that matches the labels of the .spec.template.
- Do not create pods with names that match the selector. Doing so may create problems during updates or when you want to delete, as daemonSet will assume that it created the pods.
- You cannot mutate .spec.selector once a DaemonSet is created, as doing so will break things.
- It is important that you have a good understanding of taints and tolerations before configuring a DaemonSet. Before tolerations can be applied to a DaemonSet, appropriate taints must be set for nodes.
DaemonSets are a useful tool for performing many maintenance and monitoring applications. Now that you have a clearer understanding of the things you can do with this Kubernetes feature, you can get coding quickly. For making containerisation and microservices a breezy feature at your firm, experts from E2E Cloud are always here to help!