Back to Blog

What Happens to a PersistentVolume After It Is Provisioned, for the CKA, CKAD & KCSA Exams: Volume Expansion Internals (allowVolumeExpansion, Controller vs Node Resize, allocatedResourceStatuses, Shrinking a Failed Request), VolumeAttributesClass & modifyVolumeStatus, VolumeSnapshot / VolumeSnapshotContent / VolumeSnapshotClass, CSI Cloning & dataSource vs dataSourceRef, Volume Populators, Re-binding a Released PV with claimRef & volumeName, Migrating to ReadWriteOncePod, and the PV Deletion-Protection Finalizers

A practitioner's guide to the half of Kubernetes storage that study guides skip: everything you can do to a PersistentVolumeClaim after it is Bound. How expansion really works (StorageClass allowVolumeExpansion, the external-resizer's ControllerExpandVolume then the kubelet's NodeExpandVolume, why the Pod must be running for the filesystem step, the Resizing and FileSystemResizePending conditions, status.allocatedResources and allocatedResourceStatuses, and the stable-since-1.34 ability to lower a failed request), VolumeAttributesClass for changing IOPS and throughput in place (GA 1.34, locked on since 1.36, modifyVolumeStatus Pending / InProgress / Infeasible), the snapshot.storage.k8s.io CRDs and the snapshot controller versus the csi-snapshotter sidecar, restoring and cloning through dataSource, why dataSourceRef exists and the three documented differences, volume populators and the alpha cross-namespace source, the exact procedure for re-binding a Released Retain PV (clear claimRef, set volumeName, storageClassName ''), the documented ReadWriteOnce to ReadWriteOncePod migration, the pv-protection, pv-controller and external-provisioner finalizers with HonorPVReclaimPolicy, which in-tree plugins were removed and when, a diagnosis table, an 8-step lab, a scenario drill, mistakes and FAQ.

By Sailor Team , November 17, 2026

Every Kubernetes storage tutorial, including our own CKA PV, PVC and StorageClass guide and the CKAD persistent and ephemeral volumes guide, ends at the same moment: the PersistentVolumeClaim reports Bound, the Pod mounts it, the file is written and read back. That is the first half of a volume’s life. The exams, and real clusters, spend a lot of time in the second half.

The second half is where the claim is already Bound and somebody needs to change it. The database outgrew its disk and the claim must get bigger without a redeploy. The team wants more IOPS on the same volume. Someone needs a copy of production data in a staging namespace, or a point-in-time snapshot before a risky migration. A StatefulSet was deleted with Retain on its volumes and the data must be re-attached to a new claim. A node was decommissioned and the volume’s access mode needs to be tightened to ReadWriteOncePod. A PV with Delete policy was removed before its PVC and the cloud disk is still there, costing money.

Each of those operations has its own controller, its own status fields and its own failure modes, and almost none of them appear in the beginner material. This post covers them at the depth the CKA storage domain, the CKAD application-design domain and the KCSA storage-security questions actually test, using the Kubernetes 1.37 documentation and the storage API reference as the source. The security consequences of reclaim policies, finalizers and clones are owned by our KCSA storage security guide and are cross-referenced here rather than retaught.

The mental model: who acts on a Bound claim

Once a PVC is Bound, four different actors can still change it, and knowing which one is responsible tells you where to look when something stalls.

OperationTriggerControl-plane actorNode actorWhere status appears
ExpansionEdit spec.resources.requests.storage upwardexternal-resizer sidecar calls CSI ControllerExpandVolumekubelet calls CSI NodeExpandVolume and grows the filesystemPVC status.conditions, status.allocatedResources, status.allocatedResourceStatuses, status.capacity
Attribute changeEdit spec.volumeAttributesClassNameexternal-resizer sidecar calls CSI ControllerModifyVolumenonePVC status.currentVolumeAttributesClassName, status.modifyVolumeStatus
SnapshotCreate a VolumeSnapshotsnapshot controller creates VolumeSnapshotContent; csi-snapshotter sidecar calls CSI CreateSnapshotnoneVolumeSnapshot.status.readyToUse, boundVolumeSnapshotContentName, restoreSize
Restore or cloneCreate a new PVC with dataSourceexternal-provisioner calls CSI CreateVolume with a content sourcenoneNew PVC phase
PopulateCreate a new PVC with dataSourceRef to a custom kindthe matching volume populator controllernoneNew PVC phase, populator events
Re-bindDelete PVC, edit PV claimRef, create PVC with volumeNamePV controller in kube-controller-managernonePV phase Released → Available → Bound

The pattern that explains most exam scenarios: control-plane steps happen without a Pod; node steps need the volume mounted on a node, which means a running Pod. Expansion of a filesystem volume is the only operation in the table with a node step, and that is exactly why “my PVC resize is stuck” is the most common storage ticket.

Volume expansion, end to end

The prerequisites

Expansion is stable since Kubernetes 1.24 and enabled by default, but three things must all be true for a given claim:

  1. The claim’s StorageClass has allowVolumeExpansion: true. The default is false. A StorageClass is otherwise immutable in its parameters, but allowVolumeExpansion can be flipped on an existing class.
  2. The volume is a CSI volume whose driver implements the expansion capability. The documentation lists csi (including migrated in-tree types), plus the deprecated flexVolume and portworxVolume. If you provisioned a PV by hand with no CSI driver behind it (a hostPath or local PV on an exam cluster), there is nothing to expand: editing the PV’s capacity directly changes the number in the API and nothing else.
  3. You edit the PVC, not the PV, and only upward. Kubernetes does not support shrinking. Editing a PV’s capacity by hand is actively harmful: if the PV and PVC sizes then match, the controller believes no resize is needed and skips the storage operation.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-expandable
provisioner: ebs.csi.aws.com
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete

Then the whole user-facing operation is one patch:

kubectl patch pvc data-db-0 -p '{"spec":{"resources":{"requests":{"storage":"20Gi"}}}}'

The two-phase mechanics

What happens next is two separate phases carried out by two separate components, and the PVC status tells you which phase you are in.

Phase 1, controller expansion. The external-resizer sidecar running alongside the CSI controller plugin notices that spec.resources.requests.storage exceeds status.capacity.storage. It records the request in status.allocatedResources and sets status.allocatedResourceStatuses.storage to ControllerResizeInProgress, then calls the driver’s ControllerExpandVolume, which grows the backend disk. The PVC carries a condition of type Resizing while this runs. If the driver reports that no node-side work is needed (raw block volumes, or backends that resize the filesystem themselves), the capacity is updated and the operation is complete.

Phase 2, node expansion. For a filesystem volume, the backend disk is now bigger but the filesystem on it is not. The resizer sets allocatedResourceStatuses.storage to NodeResizePending and the PVC gets a condition of type FileSystemResizePending. The kubelet on the node where the volume is mounted calls NodeExpandVolume, sets NodeResizeInProgress, and grows the filesystem. Only XFS, ext3 and ext4 are supported. When it finishes, status.capacity.storage is updated to the new size and the status entry is cleared.

The documentation’s phrasing of the node step is the key exam fact: file system expansion is performed “when a new Pod using the PersistentVolumeClaim in ReadWrite mode is started”, or while a Pod is running if the filesystem supports online expansion, which the common ones do. Since 1.24 an in-use PVC is expanded without deleting or recreating the Pod. But the volume must be attached and mounted somewhere. A PVC with no Pod will finish phase 1, sit in FileSystemResizePending indefinitely, and report the old capacity until a Pod mounts it. That is not a failure; it is the controller waiting for a kubelet.

kubectl get pvc data-db-0 -o jsonpath='{.status.capacity.storage}{"\n"}{.status.allocatedResources}{"\n"}{.status.allocatedResourceStatuses}{"\n"}{.status.conditions[*].type}{"\n"}'

A typical mid-flight read:

10Gi
map[storage:20Gi]
map[storage:NodeResizePending]
FileSystemResizePending

The reading is unambiguous: the backend is already 20Gi, the filesystem is still 10Gi, and the next kubelet that mounts this volume will finish the job.

The status vocabulary

The allocatedResourceStatuses field is a map keyed by resource name (storage for the built-in case), and the API reference defines these values:

ValueMeaning
ControllerResizeInProgressThe resize controller has started resizing the volume in the control plane.
ControllerResizeFailedThe controller-side resize failed with a terminal error.
NodeResizePendingController resize finished; further resizing is needed on the node.
NodeResizeInProgressThe kubelet has started resizing the volume.
NodeResizeFailedThe kubelet resize failed with a terminal error. Transient errors do not set this.

When the map is empty, no resize is in progress. The companion field allocatedResources records the size that was requested and may be larger than the real capacity while the operation runs; the documentation notes that for storage quota, the larger of allocatedResources and spec.resources is charged, so a failed oversize request keeps consuming quota until it is lowered.

Recovering from a failed expansion

Until recently, requesting a size the backend could not provide (beyond the disk type’s maximum, beyond quota, beyond a free-tier limit on a cloud account) was a trap: the controller retried forever and you could not lower the request because shrinking is forbidden. The RecoverVolumeExpansionFailure feature gate changed that. It went alpha in 1.23, beta and enabled by default in 1.32, and stable and locked on in 1.34.

With it, recovery is one edit: lower spec.resources.requests.storage to a value that is still greater than status.capacity but smaller than the failed request. The resizer retries with the new target, and allocatedResources is lowered once no expansion is in progress and the actual capacity is at or below the new request. The documentation is careful to say the new value must exceed the current capacity; you still cannot go below what the volume actually is.

# Requested 500Gi, backend maxed at 256Gi, status shows ControllerResizeFailed.
kubectl patch pvc data-db-0 -p '{"spec":{"resources":{"requests":{"storage":"200Gi"}}}}'
kubectl get events --field-selector involvedObject.name=data-db-0 -w

The documentation also keeps the pre-1.34 manual procedure for clusters without the feature, and it is worth knowing because it is the same claimRef dance used for re-binding later in this post: set the PV reclaim policy to Retain, delete the PVC, remove claimRef from the PV, recreate the PVC with the smaller size and volumeName pointing at the PV, then restore the reclaim policy.

Expansion and StatefulSets

The one place expansion still surprises people is volumeClaimTemplates. The template in a StatefulSet is immutable, so you cannot bump the size there; you patch each PVC that the StatefulSet created (data-db-0, data-db-1, and so on) individually, and they expand independently while the Pods keep running. The template stays at the old size, which only matters for new replicas created later, and the documented workaround is to delete the StatefulSet with --cascade=orphan and recreate it with the larger template. The ordering and identity guarantees that make this safe are covered in our StatefulSets guide.

VolumeAttributesClass: changing a volume’s class without copying it

Expansion changes one number. Everything else about a provisioned volume (its IOPS, its throughput, its performance tier) used to be frozen at provisioning time because it came from StorageClass parameters, and storageClassName on a PVC is immutable. VolumeAttributesClass is the mechanism that makes those attributes mutable. It went alpha in 1.29, beta (off by default) in 1.31, stable in 1.34, and locked on from 1.36.

A VolumeAttributesClass is a cluster-scoped object in storage.k8s.io/v1 with two fields that matter: driverName, which must be a CSI driver that implements the ModifyVolume capability, and parameters, which are driver-specific and immutable once the class exists (at most 512 parameters, 256 KiB total, the same limits as a StorageClass).

apiVersion: storage.k8s.io/v1
kind: VolumeAttributesClass
metadata:
  name: silver
driverName: csi.storage.example.com
parameters:
  iops: "3000"
  throughput: "50Mi"
---
apiVersion: storage.k8s.io/v1
kind: VolumeAttributesClass
metadata:
  name: gold
driverName: csi.storage.example.com
parameters:
  iops: "10000"
  throughput: "250Mi"

A PVC opts in with spec.volumeAttributesClassName, which, unlike storageClassName, can be changed after creation. Setting it at creation time passes the parameters to CreateVolume; changing it later makes the external-resizer call ControllerModifyVolume on the existing volume. No data is copied and no Pod is restarted.

kubectl patch pvc data-db-0 -p '{"spec":{"volumeAttributesClassName":"gold"}}'
kubectl get pvc data-db-0 -o jsonpath='{.status.currentVolumeAttributesClassName} {.status.modifyVolumeStatus}{"\n"}'

The PVC status has two fields for this. status.currentVolumeAttributesClassName is the class actually applied (unset means none). status.modifyVolumeStatus exists only while a modification is being attempted and carries targetVolumeAttributesClassName plus a status with three documented values:

modifyVolumeStatus.statusMeaningWhat to do
PendingThe PVC cannot be modified because a requirement is unmet, most commonly that the named VolumeAttributesClass does not exist.Create the class, or fix the name. The controller proceeds when the class appears.
InProgressThe volume is being modified.Wait.
InfeasibleThe CSI driver rejected the request as invalid (for example, parameters the backend cannot apply to this volume type).Specify a valid class, or reset volumeAttributesClassName to its previous value (including nil) to cancel.

Two details carry exam weight. First, an empty string or nil volumeAttributesClassName means “no class”, and resetting to the previous value is the documented way to cancel a stuck Infeasible modification. Second, when you switch to a class that omits a parameter the old class set, whether the driver reverts that attribute to its default depends on the driver; the documentation tells you to read the driver’s documentation rather than assume.

The conceptual boundary to remember: StorageClass answers “which driver and how to provision”, VolumeAttributesClass answers “which mutable quality-of-service settings”. The first is immutable on a claim, the second is not.

Snapshots: three CRDs and two controllers

Volume snapshots have been stable since Kubernetes 1.20, but they are not core API objects. They are CustomResourceDefinitions in the snapshot.storage.k8s.io API group, and the CRDs and their controller are installed by the Kubernetes distribution or the cluster administrator, not by Kubernetes itself and not by individual CSI drivers. On a bare kubeadm cluster, kubectl get volumesnapshot returns “the server doesn’t have a resource type” until someone installs them. That alone is a plausible exam troubleshooting scenario.

The three objects mirror the PV/PVC/StorageClass trio exactly:

Snapshot objectScopeAnalogous toPurpose
VolumeSnapshotClassclusterStorageClassdriver, deletionPolicy (Delete or Retain), driver-specific parameters; a default can be marked with the snapshot.storage.kubernetes.io/is-default-class annotation
VolumeSnapshotnamespacedPersistentVolumeClaimA user’s request for a snapshot of a PVC (spec.source.persistentVolumeClaimName) or a binding to a pre-provisioned content (spec.source.volumeSnapshotContentName)
VolumeSnapshotContentclusterPersistentVolumeThe actual snapshot on the backend: spec.driver, spec.source.volumeHandle (dynamic) or spec.source.snapshotHandle (pre-provisioned), spec.volumeSnapshotRef, spec.deletionPolicy, spec.sourceVolumeMode

Two controllers divide the work. The snapshot controller (one per cluster, installed with the CRDs) watches VolumeSnapshot and VolumeSnapshotContent objects and handles binding and the creation and deletion of content objects. The csi-snapshotter sidecar (one per CSI driver deployment) watches VolumeSnapshotContent and makes the actual CreateSnapshot and DeleteSnapshot calls to the driver. If a snapshot never becomes ready, the question is which of the two is missing or failing.

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: csi-snapclass
driver: ebs.csi.aws.com
deletionPolicy: Delete
---
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: db-before-migration
  namespace: prod
spec:
  volumeSnapshotClassName: csi-snapclass
  source:
    persistentVolumeClaimName: data-db-0
kubectl -n prod get volumesnapshot db-before-migration \
  -o jsonpath='{.status.readyToUse} {.status.boundVolumeSnapshotContentName} {.status.restoreSize}{"\n"}'

The lifecycle facts that generate questions:

  • Binding is one-to-one, enforced by the snapshot controller, exactly like PV/PVC.
  • A PVC that is the source of an in-progress snapshot is protected: if you delete it, removal is deferred until the snapshot is readyToUse or aborted.
  • deletionPolicy lives on the content, copied from the class at creation. Delete removes the backend snapshot and the content object when the VolumeSnapshot is deleted; Retain keeps both, leaving a cluster-scoped object that references real data, which is the KCSA angle covered in the storage security guide.
  • Volume mode conversion is blocked by default. Restoring a Filesystem snapshot into a Block PVC (or vice versa) requires the administrator to annotate the VolumeSnapshotContent with snapshot.storage.kubernetes.io/allow-volume-mode-change: "true", because a conversion can expose raw filesystem bytes. For pre-provisioned contents the administrator must also set spec.sourceVolumeMode.
  • Pre-provisioned snapshots flow the other way: an administrator creates the VolumeSnapshotContent with a snapshotHandle from the backend and a volumeSnapshotRef naming the VolumeSnapshot that will claim it, then the user creates that VolumeSnapshot with source.volumeSnapshotContentName.

Restoring, cloning and populating: the dataSource family

Creating a new PVC with content already in it is one mechanism with three sources, all expressed through two fields on the PVC spec.

Restore from a snapshot

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-db-restored
  namespace: prod
spec:
  storageClassName: fast-expandable
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 20Gi
  dataSource:
    name: db-before-migration
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io

The external-provisioner passes the snapshot as the volume content source to CreateVolume, and the driver creates a new volume populated from it. The request must be at least the snapshot’s restoreSize, the snapshot must be in the same namespace, and the driver must support restore.

Clone from another PVC

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: clone-of-data-db-0
  namespace: prod
spec:
  storageClassName: fast-expandable
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 20Gi
  dataSource:
    kind: PersistentVolumeClaim
    name: data-db-0

Note the absence of apiGroup: a PersistentVolumeClaim is a core object, so the group is empty. The documented cloning rules:

  • CSI drivers only, and only drivers that implement cloning; dynamic provisioners only.
  • Source and destination must be in the same namespace.
  • The source PVC must be bound and available (not in use).
  • The destination may use the same StorageClass as the source or a different one, and may omit storageClassName to use the default.
  • Source and destination must have the same volumeMode: a block clone needs a block source.
  • The requested size must be equal to or larger than the source’s capacity.
  • The clone is a fully independent volume: you can use, snapshot, clone or delete it without reference to the source, and modifying or deleting the source does not affect the clone.

dataSource versus dataSourceRef

The dataSource field only ever accepted two kinds: VolumeSnapshot and PersistentVolumeClaim. To let any custom resource serve as a content source, Kubernetes added dataSourceRef, and the AnyVolumeDataSource feature gate that enables it went stable and locked on in 1.33. The API reference documents three differences, which are the exam content:

BehaviourdataSourcedataSourceRef
Allowed kindsOnly VolumeSnapshot and PersistentVolumeClaimAny object in a non-core API group, plus PersistentVolumeClaim
Disallowed valuesSilently droppedPreserved, and an error is returned
NamespaceSame namespace onlyAny namespace, via the namespace sub-field (alpha)

For backwards compatibility the API server keeps the two fields in sync: when dataSourceRef.namespace is not set and only one of the two fields is filled in, the other is copied from it. If both are set they must match. When dataSourceRef.namespace is set, dataSource is left empty and must stay empty.

The practical rule: use dataSource for snapshot restore and PVC clone in the same namespace (it is what the documentation examples show and what older provisioners expect), and use dataSourceRef whenever the source is a custom kind or lives in another namespace.

Volume populators

A volume populator is a controller that watches for PVCs whose dataSourceRef names its custom kind and fills the new volume from that source, for example by downloading an image, restoring a backup from object storage, or seeding a dataset. Kubernetes ships the API and a library for writing populators; the populators themselves are third-party. When a PVC’s dataSourceRef names a kind that no installed populator or provisioner handles, the documentation says binding will not succeed, so the PVC stays Pending with no provisioner picking it up.

spec:
  dataSourceRef:
    apiGroup: example.storage.k8s.io
    kind: ExampleDataSource
    name: dataset-2026-q3

Cross-namespace sources

The namespace sub-field of dataSourceRef is gated by CrossNamespaceVolumeDataSource, which has been alpha and off by default since 1.26 with no promotion so far. When enabled, a PVC in namespace staging can name a VolumeSnapshot in namespace prod as its source, subject to a Gateway API ReferenceGrant in the source namespace permitting the reference. Know that this exists, that it is alpha, and that a cluster with the gate off ignores the field, which means the everyday answer to “copy production data into staging” is still snapshot in prod, then restore in prod, or use a populator or backup tool.

Re-binding a Released PersistentVolume

This is the most-asked second-half scenario on the CKA, because it needs no CSI driver and works on any exam cluster: a PVC was deleted, the PV had persistentVolumeReclaimPolicy: Retain, the PV is now Released, and you must attach the data to a new claim.

The reason a Released PV does not simply bind to the next matching claim is spec.claimRef. Binding is bidirectional: the PVC records spec.volumeName and the PV records spec.claimRef with the claim’s name, namespace and uid. After the PVC is deleted that claimRef still points at an object that no longer exists, and the PV controller will not hand a volume holding someone’s data to an arbitrary new claim. You have to clear the reference deliberately.

kubectl get pv pv-db-0 -o jsonpath='{.status.phase} {.spec.claimRef.name} {.spec.claimRef.uid}{"\n"}'
# Released data-db-0 7c9d...

# Option A: remove the whole claimRef; the PV becomes Available to any matching claim.
kubectl patch pv pv-db-0 --type=json -p='[{"op":"remove","path":"/spec/claimRef"}]'

# Option B (safer): keep name and namespace, clear only the uid, so only a claim
# with that exact name in that namespace can bind.
kubectl patch pv pv-db-0 -p '{"spec":{"claimRef":{"uid":""}}}'

Then create the replacement claim and pin it to the volume:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-db-0
  namespace: prod
spec:
  storageClassName: ""
  volumeName: pv-db-0
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 10Gi

Three fields in that manifest are exam traps:

  • volumeName reserves the specific PV. The documentation notes that with volumeName set, most matching criteria are skipped, including node affinity, but the control plane still checks that storage class, access modes and requested size are compatible.
  • storageClassName: "" (explicit empty string) is required when the PV has no class. If you omit the field entirely, the DefaultStorageClass admission plugin assigns the cluster default and the class no longer matches, so the claim stays Pending. Retroactive default assignment, stable since 1.28, makes this bite even on claims created before a default existed; the only way to opt out is the explicit empty string.
  • The requested size must not exceed the PV’s capacity, and the access mode must be one the PV lists. A PVC asking for 20Gi cannot bind to a 10Gi PV even with volumeName set.

Option B above is the documented approach in the ReadWriteOncePod migration task and the one to prefer in production: by leaving claimRef.name and claimRef.namespace in place and only blanking the uid, you reserve the PV for a specific future claim and prevent a claim in another namespace from grabbing a volume full of data. The security reasoning is spelled out in the storage security guide.

Changing a volume’s access mode: the ReadWriteOncePod migration

Access modes on a PVC are immutable, and ReadWriteOncePod (stable since 1.29, CSI volumes only) cannot be combined with any other mode. So tightening an existing ReadWriteOnce volume to ReadWriteOncePod is the re-binding procedure with one extra patch. The documented sequence:

# 1. Protect the data.
kubectl patch pv cat-pictures-pv -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'

# 2. Stop the writer and release the claim.
kubectl scale --replicas=0 deployment cat-pictures-writer
kubectl delete pvc cat-pictures-pvc

# 3. Free the PV for a new claim with the same name and namespace.
kubectl patch pv cat-pictures-pv -p '{"spec":{"claimRef":{"uid":""}}}'

# 4. Change the mode on the PV (now unbound, so it is editable).
kubectl patch pv cat-pictures-pv -p '{"spec":{"accessModes":["ReadWriteOncePod"]}}'

# 5. Recreate the PVC with accessModes: [ReadWriteOncePod] and volumeName: cat-pictures-pv,
#    then scale the writer back up.
kubectl apply -f cat-pictures-pvc.yaml
kubectl scale --replicas=1 deployment cat-pictures-writer

# 6. Restore the original reclaim policy if it was Delete.
kubectl patch pv cat-pictures-pv -p '{"spec":{"persistentVolumeReclaimPolicy":"Delete"}}'

The documentation adds two constraints worth memorising: only ReadWriteOnce to ReadWriteOncePod migrations are supported, and you should not make other PVC changes such as a resize until the migration is complete. The sidecar minimums for ReadWriteOncePod are csi-provisioner 3.0.0, csi-attacher 3.3.0 and csi-resizer 1.3.0.

Deletion order and the finalizers

The last second-half topic is what happens when things are deleted in the wrong order, and it is governed by three finalizers on the PersistentVolume.

FinalizerAdded toPurpose
kubernetes.io/pv-protectionEvery PV that is bound to a PVCPrevents the PV object from being removed while a claim uses it; the PV sits in Terminating until the PVC goes away. (kubernetes.io/pvc-protection is the PVC-side twin that blocks PVC deletion while a Pod uses it.)
kubernetes.io/pv-controllerDynamically provisioned in-tree volumes (since 1.31); not added to statically provisioned in-tree volumesEnsures the backing storage is deleted before the PV object disappears when the policy is Delete
external-provisioner.volume.kubernetes.io/finalizerBoth dynamic and static CSI volumes (since 1.31); also replaces pv-controller on CSI-migrated volumesSame guarantee for CSI volumes

The last two exist because of a long-standing gap: if an administrator deleted a Delete-policy PV before its PVC, the PV object vanished immediately and the backing disk was orphaned, still billed and still holding data. The HonorPVReclaimPolicy feature gate closed that gap; it went beta on by default in 1.31 and stable in 1.33, and the gate itself has since been removed because the behaviour is now unconditional. With it, the reclaim policy is honoured regardless of deletion order: delete the PV first and it stays Terminating with the finalizer until the PVC is deleted, at which point the backend volume is removed and then the object.

kubectl get pv pv-db-0 -o jsonpath='{.metadata.finalizers}{"\n"}'
# ["kubernetes.io/pv-protection","external-provisioner.volume.kubernetes.io/finalizer"]

The exam-relevant consequence is the reverse: a PV stuck in Terminating is almost never a bug. It is either pv-protection waiting for a bound PVC to be deleted, or the provisioner finalizer waiting for the CSI driver to confirm backend deletion. Removing the finalizer by hand clears the object and leaves the real disk behind.

Two neighbouring facts round this out. The Recycle reclaim policy is deprecated, supported only by nfs and hostPath, and does nothing more than rm -rf the volume’s contents; dynamic provisioning replaced it. And the lastPhaseTransitionTime field on a PV, which records when it last changed phase, is the field a cleanup script would use to find volumes that have been Released for weeks.

In-tree plugins are gone: why every scenario above says CSI

Nearly every operation in this post requires a CSI driver, and the reason is that the in-tree volume plugins that used to ship inside Kubernetes have been removed. CSIMigration as a whole went stable in 1.25; existing PVs that reference an in-tree plugin keep working through translation to the equivalent CSI driver, but only if that driver is installed, which Kubernetes does not do for you.

In-tree pluginRemoved in
cinder (OpenStack)1.26
awsElasticBlockStore, azureDisk1.27
gcePersistentDisk1.28
azureFile, vsphereVolume1.30
portworxVolume1.36
flexVolumeDeprecated since 1.23, still present

The StorageClass documentation’s provisioner table now shows an internal provisioner only for a dwindling set, and nfs has never had one. For exam purposes: if a question shows provisioner: kubernetes.io/aws-ebs on a current cluster, the correct diagnosis is that nothing will provision it, and the fix is the ebs.csi.aws.com provisioner with the CSI driver installed.

Diagnosis table

SymptomWhere to lookLikely causeFix
PVC edit to larger size rejected by the API serverkubectl patch errorStorageClass lacks allowVolumeExpansion: truePatch the StorageClass, retry
PVC shows new request, capacity unchanged, condition FileSystemResizePending, status NodeResizePendingkubectl get pvc -o yamlNo Pod mounts the volume, so the kubelet cannot grow the filesystemStart a Pod that mounts the PVC (or wait for the existing one on online-capable filesystems)
Status ControllerResizeFailed, events mention backend limits or quotaPVC eventsRequested size beyond what the backend allowsLower the request to a value above current capacity (stable since 1.34)
kubectl edit pv capacity changed, nothing happenedPV and PVC sizes matchPV was edited instead of the PVCNever edit PV capacity by hand; edit the PVC
modifyVolumeStatus.status: PendingPVC statusNamed VolumeAttributesClass does not existCreate it or fix the name
modifyVolumeStatus.status: InfeasiblePVC status, eventsDriver rejected the parameters for this volumeChoose a valid class, or reset volumeAttributesClassName to the previous value to cancel
kubectl get volumesnapshot: resource type not foundAPI discoverySnapshot CRDs not installedInstall the CRDs and snapshot controller (distribution responsibility)
VolumeSnapshot never readyToUse, no VolumeSnapshotContent createdsnapshot controller logsSnapshot controller missing or no VolumeSnapshotClass (and no default)Install controller / create class
VolumeSnapshotContent exists, snapshot never readycsi-snapshotter sidecar logsDriver sidecar missing or driver lacks snapshot capabilityFix the driver deployment
Restore PVC Pending, event about volume modeprovisioner eventsFilesystem snapshot restored into Block PVC (or reverse)Annotate content with allow-volume-mode-change: "true" or match the mode
Clone PVC Pending, size errorprovisioner eventsClone request smaller than source capacityRequest at least the source size
Clone PVC Pending, namespace errorprovisioner eventsSource PVC in a different namespaceClone within the namespace; use snapshot + restore or a populator across namespaces
PVC with dataSourceRef custom kind stays Pending foreverPVC eventsNo populator installed for that kindInstall the populator or use a supported source
Released PV will not bind to a new PVCPV spec.claimRefStale claimRef from the deleted claimClear claimRef.uid (or the whole claimRef); create PVC with volumeName
New PVC with volumeName stays Pending, PV is AvailablePVC storageClassNameDefault class was injected because the field was omittedSet storageClassName: "" explicitly
New PVC with volumeName stays Pending, class matchesPVC size and access modesRequested size exceeds PV capacity, or access mode not offered by PVLower the request / match the mode
PV stuck Terminatingmetadata.finalizerspv-protection (PVC still bound) or provisioner finalizer waiting for backend deletionDelete the PVC first; wait for the driver; do not strip finalizers
provisioner: kubernetes.io/aws-ebs class provisions nothingStorageClassIn-tree plugin removed (1.27)Use the CSI provisioner name with the driver installed

Lab: the second half of a volume’s life in eight steps

This lab assumes a cluster with a CSI driver that supports expansion and snapshots (any of the major cloud drivers, or the CSI hostpath driver on a kind cluster) and the snapshot CRDs installed. Steps 6 to 8 work on any cluster, including a bare kubeadm exam environment.

Step 1: Create an expandable class and a claim, and mount it.

kubectl apply -f - <<'EOF'
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: lab-expandable
provisioner: hostpath.csi.k8s.io   # replace with your driver
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: lab-pvc
spec:
  storageClassName: lab-expandable
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: lab-writer
spec:
  containers:
  - name: app
    image: busybox:1.36
    command: ["sh","-c","echo hello > /data/hello && sleep 1d"]
    volumeMounts:
    - name: data
      mountPath: /data
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: lab-pvc
EOF
kubectl wait --for=condition=Ready pod/lab-writer --timeout=120s

Step 2: Expand while in use and watch both phases.

kubectl patch pvc lab-pvc -p '{"spec":{"resources":{"requests":{"storage":"2Gi"}}}}'
kubectl get pvc lab-pvc -o jsonpath='{.status.capacity.storage} {.status.allocatedResourceStatuses} {.status.conditions[*].type}{"\n"}' -w
kubectl exec lab-writer -- df -h /data

You should see the status pass through ControllerResizeInProgress, then NodeResizePending/FileSystemResizePending, then clear, with df reporting the larger filesystem without the Pod restarting.

Step 3: Expand a claim with no Pod and observe the stall.

kubectl delete pod lab-writer
kubectl patch pvc lab-pvc -p '{"spec":{"resources":{"requests":{"storage":"3Gi"}}}}'
sleep 20
kubectl get pvc lab-pvc -o jsonpath='{.status.capacity.storage} {.status.allocatedResourceStatuses}{"\n"}'
# 2Gi map[storage:NodeResizePending]  <- waits for a kubelet
kubectl apply -f lab-writer.yaml   # recreate the Pod; the resize completes

Step 4: Snapshot and restore.

kubectl apply -f - <<'EOF'
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: lab-snap
spec:
  source:
    persistentVolumeClaimName: lab-pvc
EOF
kubectl wait --for=jsonpath='{.status.readyToUse}'=true volumesnapshot/lab-snap --timeout=120s
kubectl get volumesnapshotcontent
kubectl apply -f - <<'EOF'
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: lab-restored
spec:
  storageClassName: lab-expandable
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 3Gi
  dataSource:
    name: lab-snap
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io
EOF

Mount lab-restored in a throwaway Pod and confirm /data/hello exists. Note that no default VolumeSnapshotClass means the snapshot never binds; create one if the wait times out.

Step 5: Clone directly from the PVC.

kubectl apply -f - <<'EOF'
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: lab-clone
spec:
  storageClassName: lab-expandable
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 3Gi
  dataSource:
    kind: PersistentVolumeClaim
    name: lab-pvc
EOF
kubectl get pvc lab-clone -o jsonpath='{.spec.dataSource}{"\n"}{.spec.dataSourceRef}{"\n"}'

Observe that the API server filled in dataSourceRef to mirror dataSource. Try the clone with storage: 1Gi and read the provisioner event that rejects it.

Step 6: Retain, delete, re-bind (works anywhere).

PV=$(kubectl get pvc lab-pvc -o jsonpath='{.spec.volumeName}')
kubectl patch pv "$PV" -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'
kubectl delete pod lab-writer; kubectl delete pvc lab-pvc
kubectl get pv "$PV" -o jsonpath='{.status.phase} {.spec.claimRef.uid}{"\n"}'   # Released <uid>
kubectl patch pv "$PV" -p '{"spec":{"claimRef":{"uid":""}}}'
kubectl get pv "$PV" -o jsonpath='{.status.phase}{"\n"}'                          # Available
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: lab-pvc
spec:
  storageClassName: lab-expandable
  volumeName: $PV
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 3Gi
EOF
kubectl get pvc lab-pvc   # Bound to the same PV, data intact

Repeat with storageClassName omitted on a cluster that has a default class and watch it stay Pending.

Step 7: Deletion order and finalizers.

kubectl get pv "$PV" -o jsonpath='{.metadata.finalizers}{"\n"}'
kubectl patch pv "$PV" -p '{"spec":{"persistentVolumeReclaimPolicy":"Delete"}}'
kubectl delete pv "$PV" --wait=false
kubectl get pv "$PV"            # Terminating, held by pv-protection
kubectl delete pvc lab-pvc      # now the PV and the backend volume go

Step 8: Change attributes (needs a driver with ModifyVolume).

kubectl apply -f - <<'EOF'
apiVersion: storage.k8s.io/v1
kind: VolumeAttributesClass
metadata:
  name: lab-gold
driverName: hostpath.csi.k8s.io   # replace with your driver and its parameters
parameters:
  iops: "10000"
EOF
kubectl patch pvc lab-restored -p '{"spec":{"volumeAttributesClassName":"lab-gold"}}'
kubectl get pvc lab-restored -o jsonpath='{.status.currentVolumeAttributesClassName} {.status.modifyVolumeStatus}{"\n"}'

Then patch to a class name that does not exist and read Pending; set it back to null and watch the status clear.

Scenario drill: cue to answer

Scenario cueAnswer
”PVC resize request rejected: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize”Set allowVolumeExpansion: true on the StorageClass
PVC shows 20Gi requested, 10Gi capacity, condition FileSystemResizePending, no Pod runningSchedule a Pod that mounts it; the kubelet performs the filesystem resize
Requested 1Ti, driver max 500Gi, resize failing in a loopLower the request above current capacity (1.34+ stable), or the Retain/delete/claimRef/volumeName procedure on older clusters
”Shrink the PVC from 100Gi to 50Gi”Not supported; create a smaller volume and copy data
Need more IOPS on an existing volume without downtime or copyingChange spec.volumeAttributesClassName to a class with higher parameters (1.34+)
modifyVolumeStatus.status: InfeasibleDriver rejected it; set a valid class or reset to the previous class name to cancel
Change storageClassName on an existing PVCImmutable; recreate the PVC (snapshot/clone to migrate data)
“Take a point-in-time copy before a schema migration”VolumeSnapshot with source.persistentVolumeClaimName
kubectl get volumesnapshot returns “no resource type”Snapshot CRDs not installed; the distribution/admin installs them
Snapshot content created, never readycsi-snapshotter sidecar or driver capability problem
Restore a Filesystem snapshot to a Block PVCNeeds snapshot.storage.kubernetes.io/allow-volume-mode-change: "true" on the content
”Copy the volume into a new PVC in the same namespace, same driver”dataSource: {kind: PersistentVolumeClaim, name: ...}, size ≥ source
Clone across namespacesNot with dataSource; dataSourceRef.namespace is alpha (CrossNamespaceVolumeDataSource)
PVC with dataSourceRef to a CRD kind stays PendingNo volume populator for that kind
dataSource with an unsupported kindSilently dropped; dataSourceRef would error instead
PV Released after PVC deletion, must reuse the dataClear claimRef (prefer only uid), create PVC with volumeName
PVC with volumeName Pending, PV Available, PV has no classPVC must set storageClassName: "" explicitly
PVC with volumeName Pending, PVC asks 20Gi, PV is 10GiRequest must not exceed PV capacity
Change RWO to RWOP on an existing volumeRetain, delete PVC, clear claimRef.uid, patch PV accessModes, recreate PVC with volumeName
PV stuck Terminatingpv-protection (PVC bound) or provisioner finalizer; delete PVC / wait; do not strip finalizers
Deleted a Delete-policy PV before its PVC; is the disk orphaned?Not since HonorPVReclaimPolicy (stable 1.33); the finalizer deletes backend storage after the PVC goes
provisioner: kubernetes.io/gce-pd on a 1.37 clusterRemoved in 1.28; use the CSI provisioner
StatefulSet volumes need to growPatch each PVC; the template is immutable (recreate with --cascade=orphan to update it)
Recycle reclaim policyDeprecated, rm -rf only, nfs/hostPath only

Common mistakes

  • Editing the PV’s capacity. It changes a number in the API and can prevent the real resize. Always edit the PVC.
  • Expecting a resize to complete with no Pod. Phase 2 needs the kubelet. The FileSystemResizePending condition is a wait, not an error.
  • Treating allowVolumeExpansion as a PVC field. It is a StorageClass field, and it can be added to an existing class.
  • Confusing storageClassName with volumeAttributesClassName. The first is immutable and selects the provisioner; the second is mutable and selects QoS parameters on a CSI driver with ModifyVolume.
  • Assuming snapshots are built in. They are CRDs plus a controller that the distribution installs; a cluster can have a CSI driver and still have no snapshot API.
  • Omitting storageClassName: "" on a static re-bind. The default class gets injected and the claim never matches the class-less PV.
  • Clearing the whole claimRef in a multi-tenant cluster. The volume becomes Available to any namespace. Clear only the uid to reserve it for the intended claim.
  • Cloning with a smaller request than the source. The request must be equal to or greater than the source capacity.
  • Stripping finalizers from a Terminating PV. You remove the object and leave the backend disk orphaned, which is the exact outcome the finalizer exists to prevent.
  • Writing kubernetes.io/aws-ebs or similar in a new StorageClass. Those plugins are gone; name the CSI driver.

Frequently Asked Questions

Why is my PVC resize stuck in FileSystemResizePending?

Because the backend volume has been expanded but the filesystem on it has not, and the filesystem step is done by the kubelet on the node where the volume is mounted. If no Pod mounts the PVC, nothing can perform that step. Start a Pod that uses the claim and the status clears within seconds.

Can I shrink a PersistentVolumeClaim?

No. Kubernetes does not support reducing a PVC below its current capacity. Since 1.34 you can lower a pending expansion request that failed, as long as the new value is still above the current capacity. To actually shrink, create a smaller volume and copy the data.

What is the difference between StorageClass and VolumeAttributesClass?

StorageClass chooses the provisioner and the provisioning-time parameters and is immutable on a claim. VolumeAttributesClass (stable in 1.34) describes mutable attributes such as IOPS and throughput, and a PVC’s volumeAttributesClassName can be changed after creation; the external-resizer applies it through the CSI ModifyVolume call with no data copy.

Are VolumeSnapshots part of the core Kubernetes API?

No. They are CustomResourceDefinitions in the snapshot.storage.k8s.io group (VolumeSnapshotClass, VolumeSnapshot, VolumeSnapshotContent) that the distribution or administrator installs along with the snapshot controller. Each CSI driver then runs a csi-snapshotter sidecar to perform the actual snapshot calls.

What is the difference between dataSource and dataSourceRef on a PVC?

dataSource accepts only VolumeSnapshot and PersistentVolumeClaim in the same namespace and silently drops anything else. dataSourceRef accepts any non-core kind (for volume populators) plus PVCs, returns an error for disallowed values, and has an alpha namespace field for cross-namespace sources. The API server keeps the two in sync when no namespace is specified.

How do I reuse a Released PersistentVolume?

Clear the stale spec.claimRef on the PV (preferably only its uid, so the PV remains reserved for a claim with the same name and namespace), then create a PVC that sets volumeName to the PV, requests no more than its capacity, uses a matching access mode, and sets storageClassName: "" if the PV has no class.

Can I change a PVC’s access mode?

Not directly; access modes are immutable on a PVC. The documented path for ReadWriteOnce to ReadWriteOncePod is to set the PV to Retain, delete the PVC, clear claimRef.uid, patch the PV’s accessModes, and recreate the PVC with volumeName. Only that direction of migration is supported.

Why is my PersistentVolume stuck in Terminating?

Either the kubernetes.io/pv-protection finalizer is holding it because a PVC is still bound, or the provisioner finalizer (external-provisioner.volume.kubernetes.io/finalizer for CSI, kubernetes.io/pv-controller for in-tree) is waiting for the backend storage to be deleted first, which is the HonorPVReclaimPolicy behaviour stable since 1.33. Delete the PVC or wait for the driver; removing the finalizer manually orphans the real disk.

Conclusion

A Bound PVC is not the end of a volume’s story. It can grow (edit the PVC, never the PV; the controller expands the disk and the kubelet expands the filesystem, which is why a Pod must be running), change its quality of service in place through a VolumeAttributesClass, be snapshotted through a trio of CRDs with two cooperating controllers, be restored or cloned through dataSource, be populated from a custom source through dataSourceRef, be re-bound after release by clearing claimRef and pinning volumeName with an explicit empty storageClassName, have its access mode tightened through the same procedure, and be protected from out-of-order deletion by three finalizers. Every one of those operations has a status field that tells you exactly where it is, and almost every “stuck” storage ticket is one of those fields correctly reporting that it is waiting for something.

For the CKA this is the material behind the storage domain’s troubleshooting scenarios; for the CKAD it is what the application-design questions about volume lifecycle assume; for the KCSA it is the mechanics beneath the data-lifecycle risks. If you are preparing for several of these exams at once, our KubeAstronaut certification path guide and the curriculum domain map across all five exams show where storage sits in each. For timed practice on scenarios like the ones in the drill above, where you have to read a PVC’s status fields and decide in under two minutes whether the fix is a Pod, a patch, a class or a claimRef, the KubeAstronaut mock exam bundle runs CKA, CKAD, CKS, KCNA and KCSA practice exams with per-domain breakdowns, so you can see whether storage is still where your time goes.

Limited Time Offer: Get 80% off all Mock Exam Bundles | Sale ends in 7 days. Start learning today.

Claim Now