10 min read Kubernetes

Kubernetes Volumes

If you’re just getting started with Kubernetes, one of the concepts you need to understand is that of volumes and their different types. Volumes can be used for persistent data storage within your clusters and provide a secure, reliable way to store data as part of your application.

In this blog post, we will discuss the various types of volumes in Kubernetes, explain how they work with applications, explore the capabilities each type provides and look at when you might want to use them. Whether you’re an experienced DevOps engineer or new to Kubernetes infrastructure management, this guide should give you an informative overview of using volume storage in Kubernetes deployments.

What is a volume in Kubernetes?

A volume is an object that maintains a set of files in the cluster, allowing them to be shared between containers. A volume can provide persistent storage to a container without requiring changes to the application code. Volumes are flexible and can be created from various sources, such as local disks, networked file systems, cloud-based providers

What is the need for volumes?

Volumes are used to provide persistent and temporary storage to containerized applications running in a Pod. Here are some reasons why volumes are necessary in Kubernetes:

Persistence

Containers are ephemeral, which means that their data is lost when they are terminated or restarted. Volumes provide a way to persist data across container restarts and failures. This is particularly important for stateful applications that require access to persistent data.

Data Sharing

Volumes allow multiple containers in a Pod to share data. This can be useful for applications that consist of multiple containers that need to access the same data, such as a web server and a database.

Data Source Abstraction

Volumes abstract away the underlying storage technology used to store the data. This means that applications can access data stored in different types of storage systems, such as local storage, network storage, and cloud storage, without being aware of the specific implementation details of the storage technology.

Flexibility

Volumes allow for flexibility in the deployment and scaling of containerized applications. Applications can be deployed on different nodes in a cluster, and the storage can be provisioned dynamically based on the needs of the application.

What are the types of volumes in Kubernetes?

Kubernetes offers a wide variety of volumes, allowing users to store data in different ways and create an optimal setup for their applications. The most common types of volumes used in Kubernetes are:

EmptyDir

A Kubernetes emptyDir volume is the simplest type of volume that can be created in Kubernetes. It is created when a Pod is assigned to a node and exists as long as the Pod does. When the Pod terminates, the data vanishes and the volume is destroyed.

HostPath

This volume type allows a user to mount any directory from their host node into their container as a read-only or read-write volume. HostPath volumes give users the flexibility to access directories or files on the same machine where their application is running, which may be necessary for certain tasks such as debugging or testing purposes.

ConfigMap

ConfigMaps allow users to inject configuration data into Pods at runtime, through environment variables or mounted files. These ConfigMaps can have different formats such as JSON, YAML, and plain text, making them easy to use with both Docker containers and traditional applications such as Apache web servers.

Secret

Secrets are similar to ConfigMaps but with additional security features that make it suitable for securely storing sensitive information such as passwords or cryptographic keys in Kubernetes clusters. This volume type should always be encrypted using Secure Sockets Layer (SSL) encryption when stored on disk or transmitted over the network.

PersistentVolumeClaim (PVC)

PVCs provide users with an abstraction layer between Volumes and Pods to enable the dynamic provisioning of storage resources based on workload requirements at any given time. For instance, if a user's application requires more storage capacity during peak hours, then they can quickly scale up by creating additional PVCs without having to manually configure each volume individually every time they need extra storage space.

Container Storage Interface (CSI)

The Kubernetes Container Storage Interface (CSI) is a standard interface for integrating external storage systems with Kubernetes. It provides a standardized way for Kubernetes to communicate with external storage providers and enables storage vendors to develop drivers that can be used across different Kubernetes clusters.

Local

A local volume is a type of storage volume that allows a Pod to use a locally mounted directory on the node where it is running as a storage volume. They are particularly useful for stateful applications that require access to persistent data that is stored on the same node as the container. One important thing to note is that local volumes are node-specific, which means that they are only available on the node where they were created.

What is a volume mount?

A volumeMount in Kubernetes makes it feasible to attach a storage volume to a specific directory path in a container running inside a Pod.

In order to mount a volume in Kubernetes, you need to define the volume in the Pod's specification, and then mount it in the container's specification.

What is the difference between a Persistent volume and Persistent volume claim?

Persistent Volume Claim (PVC) and Persistent Volume (PV) are two different abstractions in Kubernetes. A PVC is a request for storage. It is created by a user and can be fulfilled by either a pre-created PV or dynamically provisioned storage. A PV, on the other hand, is an abstraction of cluster storage resources that provides a mechanism to create persistent storage.

A PVC defines the desired storage characteristics such as size and access modes, while the actual provisioning of the PV is done by a storage provider. When creating a PVC, you can specify whether you'd like it to be bound to an existing PV or if you'd like it to be dynamically provisioned. If you choose dynamic provisioning, Kubernetes will automatically create an appropriate PV for your PVC when needed.

Once the PVC is bound to a PV, it can be mounted into multiple pods running on different nodes without having to recreate it each time. This makes it much easier and more efficient for users to manage their persistent data because any pod that needs access to this data can mount the same volume instead of needing its own dedicated disk or physical resource.

In conclusion, Persistent Volume Claims (PVCs) are used to ask for a certain amount of storage with specific requirements such as size and access mode while Persistent Volumes (PVs) are the actual underlying resources used to satisfy these requests and provide persistent data storage in Kubernetes clusters.

What is Kubernetes volume vs Docker volume?

Kubernetes volumes and Docker volumes are both used for persistent data storage in a containerized environment. While they may seem similar at first glance, there are some key differences that set them apart.

Kubernetes volumes, or PersistentVolumes (PV), are managed by the orchestration platform itself. This allows Kubernetes to provide highly available storage clusters with the ability to scale automatically when needed. PV can be shared cluster-wide so that any application running within the same Kubernetes cluster can use it. It also provides support for snapshotting and cloning, which makes it ideal for backup and disaster recovery scenarios.

When using Kubernetes volumes, the underlying infrastructure is abstracted away from the user. This means that Kubernetes handles all the configuration details such as volume creation, access rights, etc. on behalf of the user and makes sure that data persists in the right location across multiple nodes running across a cluster of computers. This simplifies storage management significantly since users don’t have to worry about configuring every single node in a cluster to use a particular type of storage device or backend.

In contrast, Docker volumes are designed to be more lightweight and focused on providing ephemeral storage for containers. They have no replication or snapshot capabilities, meaning that if a single node fails then any data stored in the volume will be lost forever. However, this makes them ideal for storing temporarily data such as logs or temporary files that don't need to persist long-term.

Ultimately, both Kubernetes volumes and Docker Volumes have their respective strengths and weaknesses depending on your use case. For most applications requiring long-term data persistence and availability, Kubernetes PersistentVolumes (PV) will be the optimal choice whereas Docker Volumes may be better suited for lighter requirements such as temporary file stores or logs.

How to create an emptyDir volume in k8s?

You can create an emptyDir volume in Kubernetes by defining it in the Pod's YAML specification. Here is an example YAML definition of a Pod with an emptyDir volume:

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  containers:
    - name: example-container
      image: ubuntu:latest
      volumeMounts:
        - name: emptydir-volume
          mountPath: /data
  volumes:
    - name: emptydir-volume
      emptyDir: {}

In this example, we define a Pod named example-pod with an emptyDir volume named emptydir-volume. The emptyDir volume is created automatically when the Pod is created, and it is deleted when the Pod is terminated. The volume is mounted inside the container using the volumeMounts field in the container specification. In this example, we mount the volume at the /data path inside the container.

Once the emptyDir volume is mounted, the container can read and write data to the volume as if it was a local file system. The data in the emptyDir volume will be lost when the Pod is terminated, so it should not be used for data persistency.

How to create a Hostpath volume?

Here is an example YAML definition of a Pod with a HostPath volume:

apiVersion: v1
kind: Pod
metadata:
  name: hostpath-pod
spec:
  containers:
    - name: hostpath-container
      image: ubuntu:latest
      volumeMounts:
        - name: hostpath-volume
          mountPath: /data
  volumes:
    - name: hostpath-volume
      hostPath:
        path: /mnt/data

In this example, a volume named hostpath-volume is defined, that uses a HostPath to mount the /mnt/data directory on the node into the container at the /data path. The volume is then mounted in the container using the volumeMounts field in the container specification.

HostPath volumes can be used to share data between containers in the same Pod, but they should not be used for sharing data between Pods as they are not portable across nodes in the cluster.

How to create a ConfigMap?

Here is an example YAML definition of a ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: example-configmap
data:
  key1: value1
  key2: value2

In this example, we define a ConfigMap named example-configmap with two key-value pairs. The keys are key1 and key2, and their values are value1 and value2, respectively.

To apply the ConfigMap to your Kubernetes cluster, save the YAML definition to a file, example-configmap.yaml and run the kubectl apply command:

$ kubectl apply -f example-configmap.yaml

This will create the ConfigMap in your cluster. The ConfigMap can then be used the Pods by referencing it in the Pod's YAML specification. Here is an example Pod YAML specification that uses the ConfigMap:

apiVersion: v1
kind: Pod
metadata:
  name: configmap-pod
spec:
  containers:
    - name: configmap-container
      image: ubuntu:latest
      env:
        - name: KEY1
          valueFrom:
            configMapKeyRef:
              name: example-configmap
              key: key1
        - name: KEY2
          valueFrom:
            configMapKeyRef:
              name: example-configmap
              key: key2

In this example, we define a Pod named configmap-pod with a container named configmap-container. We also define two environment variables named KEY1 and KEY2. The values of these environment variables are referenced from the key1 and key2 keys in the example-configmap ConfigMap, respectively using the configMapKeyRef option.

How to create a Secret?

Here is an example YAML file for a Secret that contains a username and password:

apiVersion: v1
kind: Secret
metadata:
  name: example-secret
type: Opaque
data:
  username: uIBlcm5hbWU=
  password: xLQzc3dvcmQ=

The type: Opaque field indicates that the Secret contains arbitrary data, and the data section specifies the key-value pairs that make up the secret.

Save the YAML file and run the following command to create the Secret:

$ kubectl create -f example-secret.yaml

Create a YAML file that defines the Pod to use the Secret as a volume.

apiVersion: v1
kind: Pod
metadata:
  name: secret-pod
spec:
  containers:
  - name: secret-container
    image: ubuntu:latest
    volumeMounts:
    - name: example-secret-volume
      mountPath: /etc/my-app
      readOnly: true
  volumes:
  - name: example-secret-volume
    secret:
      secretName: example-secret

The volumeMounts section specifies that a volume named example-secret-volume should be mounted at the /etc/my-app directory in the container, and the readOnly: true field indicates that the container can only read the contents of the volume.

The volumes section specifies that the example-secret-volume volume should be created using the Secret named example-secret.

Types of Secrets

Opaque

This is the default type of secret, and it can be used to store any kind of data as an opaque, base64-encoded string. This can be used to store things like passwords, certificates, or keys.

Service Account

This type of secret is automatically created by Kubernetes when a service account is created. It contains a token that can be used to authenticate requests to the Kubernetes API.

TLS

This type of secret is used to store TLS certificates and keys for use in HTTPS or TLS-enabled applications.

Dockercfg

This type of secret is used to store credentials for private Docker registries. It contains a dockercfg file that specifies the credentials needed to access the registry.

Basic Auth

This type of secret is used to store credentials for basic authentication. It contains a username and password that can be used to authenticate requests.

SSH

This type of secret is used to store SSH keys for use in SSH-enabled applications.

How to create a Persistent volume claim?

Create a YAML file that defines the persistent volume claim's properties. Here is an example YAML file for a PVC that requests 1Gi of storage from a StorageClass called standard:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: example-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: standard

Save the YAML to a file example-pvc.yaml and run the following command to create the PVC:

$ kubectl create -f example-pvc.yaml

This command creates a PVC object in the Kubernetes API based on the properties specified in the YAML file. Kubernetes will attempt to find an existing PV that satisfies the storage class and size requirements of the PVC. If an appropriate PV is not found, Kubernetes will dynamically provision a new one.

You can check the status of the PVC by running the following command:

$ kubectl get pvc

This command will show you a list of all PVCs in the cluster, along with their status, size, and other properties.

You can use this PVC in a Pod by specifying the PVC's name in the Pod's YAML file under the volumes section.

apiVersion: v1
kind: Pod
metadata:
  name: pvc-pod
spec:
  containers:
  - name: pvc-container
    image: ubuntu:latest
    volumeMounts:
    - name: pvc-volume
      mountPath: /data
  volumes:
  - name: pvc-volume
    persistentVolumeClaim:
      claimName: example-pvc

In conclusion, Kubernetes volumes provide a powerful solution for data storage and transport. With their range of options, users can easily design systems that match their specific needs. They offer scalability, reliability and resiliency by leveraging persistent storage across multiple applications. Further, they feature built-in authentication and authorization, simplifying the process of protecting data. Best of all, Kubernetes' flexible deployment models make it simple to adapt and grow as you add or modify services over time. Kubernetes volumes let you focus on your application instead of worrying about data security and management - giving you the freedom to explore new ideas quickly.

Siva Kalyan Chintakana