Docker & Kubernetes : Persistent Volumes - Dynamic volume provisioning

bogotobogo.com site search:

Kubernetes PersistentVolume

The PersistentVolume subsystem provides an API for users and administrators that abstracts details of how storage is provided from how it is consumed.

To do this we introduce two new API resources: PersistentVolume (PV) and PersistentVolumeClaim (PVC).

There are two ways PVs may be provisioned: statically or dynamically.

When none of the static PVs the administrator created matches a user's PersistentVolumeClaim, the cluster may try to dynamically provision a volume specially for the PVC. This provisioning is based on StorageClasses: the PVC must request a storage class and the administrator must have created and configured that class in order for dynamic provisioning to occur.

Pictures from Dynamic Provisioning in Kubernetes | Kubernetes Concepts & Demos | Kubernetes Tutorial

To enable dynamic provisioning, a cluster administrator needs to pre-create one or more StorageClass objects for users. StorageClass objects define which provisioner should be used and what parameters should be passed to that provisioner when dynamic provisioning is invoked. The following manifest creates a storage class "slow" which provisions standard disk-like persistent disks.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: slow
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-standard

The name of a StorageClass object is significant, and is how users can request a particular class. Administrators set the name and other parameters of a class when first creating StorageClass objects, and the objects cannot be updated once they are created.

Note that Storage classes have a provisioner that determines what volume plugin is used for provisioning PVs.

The following manifest creates a storage class "fast" which provisions SSD-like persistent disks.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-ssd

Creating StorageClass

Here is a sample of configuration file (storage-class.yaml):

# storage-class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-ssd

Note that we're using GCEPersistentDisk (gce-pd). It should be aws-ebs if we want to use AWS EBS. Regarding the "type", we're using pd-ssd while the default is pd-standard type.

GKE:

Create the StorageClass:

$ gcloud config set compute/zone us-central1-a
Updated property [compute/zone].

$ gcloud container clusters get-credentials bogotobogo
Fetching cluster endpoint and auth data.
kubeconfig entry generated for bogotobogo.

$ kubectl create -f storage-class.yaml
storageclass.storage.k8s.io/fast created

$ kubectl get storageclass
NAME                 PROVISIONER            AGE
fast                 kubernetes.io/gce-pd   10m
standard (default)   kubernetes.io/gce-pd   20m

$ kubectl describe storageclass fast
Name:                  fast
IsDefaultClass:        No
Annotations:           <none>
Provisioner:           kubernetes.io/gce-pd
Parameters:            type=pd-ssd
AllowVolumeExpansion:  <unset>
MountOptions:          <none>
ReclaimPolicy:         Delete
VolumeBindingMode:     Immediate
Events:                <none>

We can see the StorageClass just created from the Console as well:

Creating Persistent Volume Claim (PVC) - Requesting storage

Pods access storage by using the claim as a volume. Claims must exist in the same namespace as the pod using the claim. The cluster finds the claim in the pod’s namespace and uses it to get the PersistentVolume backing the claim. The volume is then mounted to the host and into the pod.

Here is the PVC configuration file, pvc-1.yaml:

# pvc-1.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: myclaim-1
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 8Gi
  storageClassName: fast

Also, almost the same but with a bigger request, 16Gi, pvc-2.yaml:

# pvc-2.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: myclaim-2
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 16Gi
  storageClassName: fast

Let's create a PVC:

$ kubectl create -f pvc-1.yaml
persistentvolumeclaim/myclaim-1 created

$ kubectl get pvc
NAME        STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
myclaim-1   Bound     pvc-e655368e-5008-11e9-b000-42010a8001f1   8Gi        RWO            fast           1m

We can create another PVC:

$ kubectl create -f pvc-2.yaml
persistentvolumeclaim/myclaim-2 created

$ kubectl get pvc
NAME        STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
myclaim-1   Bound     pvc-e655368e-5008-11e9-b000-42010a8001f1   8Gi        RWO            fast           7m
myclaim-2   Bound     pvc-e0877dbc-5009-11e9-b000-42010a8001f1   16Gi       RWO            fast           6s

Referencing the claim from a Pod - Pod definition

As mentioned earlier, pods access storage by using the claim as a volume. Claims must exist in the same namespace as the pod using the claim. The cluster finds the claim in the pod's namespace and uses it to get the PersistentVolume backing the claim. The volume is then mounted to the host and into the pod.

Now we have our storage reserved. So, let's use it from our pods.

Here is our pod definition ():

# nginx-pod.yaml
kind: Pod
apiVersion: v1
metadata:
  name: pv-pod
spec:
  containers:
    - name: pv-test-container
      image: nginx
      volumeMounts:
      - mountPath: "/test-pd"
        name: my-test-volume
  volumes:
    - name: my-test-volume
      persistentVolumeClaim:
        claimName: myclaim-1

Let's create the pod:

$ kubectl create -f nginx-pod.yaml
pod/pv-pod created

$ kubectl get po -o wide
NAME      READY     STATUS    RESTARTS   AGE       IP          NODE                                        NOMINATED NODE
pv-pod    1/1       Running   0          35s       10.40.1.5   gke-bogotobogo-default-pool-abee3b7e-jjnt   <none>

We can see the disk space (8GB) is being used now:

Testing - Data persistence

Now we need to verify the data written on the volume not go away even after the pod is gone.

Let's go into the pod and write a file:

$ kubectl get pod
NAME      READY     STATUS    RESTARTS   AGE
pv-pod    1/1       Running   0          13m

$ kubectl exec -it pv-pod -- /bin/sh
# ls
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  
root  run  sbin  srv  sys  test-pd  tmp  usr  var

# cd /test-pd

# echo "I was here" > i_was_here.txt

# ls
i_was_here.txt  lost+found

# exit

$

Delete the pod and recreate a new pod with the same configuration:

$ kubectl delete -f nginx-pod.yaml
pod "pv-pod" deleted

$ kubectl get pod
No resources found.

$ kubectl create  -f nginx-pod.yaml
pod/pv-pod created

$ kubectl get po -o wide
NAME      READY     STATUS    RESTARTS   AGE       IP          NODE                                        NOMINATED NODE
pv-pod    1/1       Running   0          50s       10.40.1.6   gke-bogotobogo-default-pool-abee3b7e-jjnt   <none>

Because the same disk space (8GB) is being used now, we expect the file we wrote should be still there:

Let's check the earlier file is there:

$ kubectl exec pv-pod df /test-pd
Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/sdb         8191416 36856   8138176   1% /test-pd

$ kubectl exec pv-pod ls /test-pd
i_was_here.txt
lost+found

$ kubectl exec pv-pod cat /test-pd/i_was_here.txt
I was here

Yes, the file is there. Our PersistentVolumes are working great!

Cleanup

We need to delete the objects we created:

$ kubectl delete -f nginx-pod.yaml
pod "pv-pod" deleted

$ kubectl delete -f pvc-1.yaml
persistentvolumeclaim "myclaim-1" deleted

$ kubectl delete -f pvc-2.yaml
persistentvolumeclaim "myclaim-2" deleted

$ kubectl delete -f storage-class.yaml
storageclass.storage.k8s.io "fast" deleted