StatefulSets Explained: A Beginner’s Guide to Stateful Applications in Kubernetes

If you're diving into the world of Kubernetes, you've likely heard the terms "stateless" and "stateful" applications thrown around. While stateless applications, like web servers, don't need to remember anything about past requests, stateful applications, such as databases, need to keep track of data and maintain their identity. This is where StatefulSets come into play. In this post, we’ll explore what StatefulSets are and walk through a real-time example of deploying a MySQL database cluster.

What is a StatefulSet?

Think of a StatefulSet as a way to manage stateful applications in Kubernetes. Unlike Deployments, which are great for stateless apps, StatefulSets ensure each instance of your application has a unique, stable identity and persistent storage. This is crucial for applications like databases that need to maintain data consistency and reliability.

Key Features of StatefulSets

• Stable, Unique Pod Names: Each pod gets a consistent and unique name, such as mysql-0, mysql-1, mysql-2etc.

• Persistent Storage: Each pod is tied to a specific PersistentVolume, ensuring that data is preserved even if the pod is rescheduled.

• Ordered Operations: Pods are created, updated, and deleted in a specific order to maintain stability and data consistency.

Why Use StatefulSets Instead of Deployments?

While Deployments are fantastic for stateless applications, they don't quite cut it for stateful applications. Here’s why:

1. Stable Pod Names and Network Identities:

   • Deployments: Pods created by a Deployment have dynamic names (e.g., web-server-xxxxxx). These names change if pods are rescheduled, making it tough to maintain stable network identities.

   • StatefulSets: Pods have stable, unique names (e.g., mysql-0, mysql-1). These names stay the same even if the pods are rescheduled, ensuring stable network identities.

2. Persistent Storage:

   • Deployments: While you can use persistent storage with Deployments, the storage isn't inherently tied to a pod’s identity. If a pod is rescheduled, it might get a different volume, leading to potential data loss or inconsistency.

   • StatefulSets: Each pod in a StatefulSet is linked to a specific PersistentVolumeClaim (PVC), ensuring that the pod’s data is preserved even if it's rescheduled.

3. Ordered Operations:

   • Deployments: Pods are created, updated, and deleted randomly, which can disrupt stateful applications that need operations to happen in a specific order.

   • StatefulSets: Pods are created, updated, and deleted in a defined order. For instance, if you scale up a StatefulSet, Kubernetes will create mypod-3 only after mypod-0, mypod-1, and mypod-2 are running and ready.

By using StatefulSets, you ensure that your stateful applications, like databases, run smoothly with consistent data and stable identities.

Depolyments v/s StatefulSet

Real-Time Example: Deploying a MySQL Database Cluster

Let’s dive into a real-time example to see how StatefulSets work. We’ll deploy a MySQL database cluster with three replicas as per the architecture below, assuming you already have a Kubernetes cluster set up.

1. Define the StatefulSet Configuration

First, we create a YAML files that defines the StatefulSets,PV,PVC for our MySQL database cluster. This file specifies the number of replicas, storage requirements, and other configurations.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
spec:
  serviceName: "mysql"
  replicas: 3
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: mysql:5.7
        ports:
        - containerPort: 3306
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: "password"
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
  volumeClaimTemplates:
  - metadata:
      name: mysql-persistent-storage
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 100Mi

PersistentVolumeClaim (PVC):

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-persistent-storage
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 100Mi
  storageClassName: standard

PersistentVolume (PV):

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-mysql
spec:
  capacity:
    storage: 100Mi
  accessModes:
  - ReadWriteOnce
  storageClassName: standard
  hostPath:
    path: "/mnt/data"

2. Deploy the StatefulSet

Apply the configuration using the kubectl command:

kubectl apply -f mysql-statefulset.yaml
kubectl apply -f mysql-pv-statefulset.yaml
kubectl apply -f mysql-pvc-statefulset.yaml

Kubernetes will create three MySQL pods (mysql-0, mysql-1, mysql-2), each with its own persistent storage.The pods will be deployed one by one in an orderly manner.

3. Stable Pod Identities

Each pod in the StatefulSet has a stable and unique identity:

• mysql-0

• mysql-1

• mysql-2

  

These identities stay consistent even if the pods are rescheduled. This stability is crucial for stateful applications like databases, which need consistent network addresses and persistent storage.

4. Persistent Storage

Each pod gets a PersistentVolumeClaim (PVC) that provides stable storage. The PVCs are automatically created based on the volumeClaimTemplates defined in the StatefulSet YAML:

• mysql-0 -> PVC: mysql-persistent-storage-mysql-0

• mysql-1 -> PVC: mysql-persistent-storage-mysql-1

• mysql-2 -> PVC: mysql-persistent-storage-mysql-2

Even if a pod is deleted and recreated, it will be reattached to its corresponding PVC, ensuring data persistence.

5. Ordered Operations

StatefulSets create, update, and delete pods in a defined order. For instance, if you scale up the StatefulSet, Kubernetes will create mysql-3 only after mysql-0, mysql-1, and mysql-2 are running and ready.

Similarly, during updates, Kubernetes will update the pods one by one in order (e.g., mysql-0 first, then mysql-1, and so on), ensuring minimal disruption.

6. Scaling the StatefulSet

You can scale the StatefulSet by modifying the replicas field:

kubectl scale statefulset mysql --replicas=5

Kubernetes will create mysql-3 and mysql-4 in order, each with its own persistent storage and stable identity.

7. Handling Pod Failures

If a pod fails, Kubernetes will recreate it with the same identity and reattach it to its persistent storage. For example, if mysql-1 fails, Kubernetes will recreate mysql-1 with the same PVC (mysql-persistent-storage-mysql-1), ensuring data continuity.

8. Accessing the StatefulSet Pods

To access the pods in a StatefulSet, you typically use a headless service. The headless service doesn’t load balance but instead provides direct DNS entries for each pod:

apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  clusterIP: None
  selector:
    app: mysql
  ports:
  - port: 3306
    name: mysql

A headless service in Kubernetes is a service that does not have a cluster IP assigned to it. Instead of load balancing traffic across multiple pods, a headless service allows direct communication with each individual pod managed by a StatefulSet. This unique characteristic makes headless services particularly useful for stateful applications, such as databases, where each pod typically requires its own unique network identity and stable DNS resolution.

You can access each MySQL instance using its DNS name:

• mysql-0.mysql

• mysql-1.mysql

• mysql-2.mysql

Real-Time Usage Scenario

Imagine you need to set up a highly available MySQL database cluster for an application. With StatefulSets, you can ensure that each MySQL instance has a stable network identity and persistent storage. This setup is crucial for database applications that rely on data consistency and require minimal disruption during scaling or updates.

For instance:

• Replication: MySQL can be configured in a master-slave replication setup, where each pod (MySQL instance) needs a stable identity to participate in replication.

• Failover: In the event of a pod failure, the replica pod can quickly take over without losing data, as the persistent storage remains intact.

• Backup and Restore: Each pod’s data can be backed up and restored individually, thanks to the stable identities and persistent storage.

Conclusion

StatefulSets provide a robust mechanism for managing stateful applications in Kubernetes. By ensuring stable pod identities, persistent storage, and ordered operations, StatefulSets are ideal for deploying databases, distributed systems, and other stateful applications that require consistency and reliability.

With this real-time example, you can see how StatefulSets simplify the deployment and management of stateful applications, ensuring they run smoothly with data persistence and identity consistency. If you’re managing stateful applications in Kubernetes, StatefulSets are a powerful tool to have in your toolkit.