The container's writable layer is a chalkboard
Everything you learned in Part A about disposable pods has a cost you have not yet met. A container's writable filesystem is ephemeral: when that container is replaced after a crash or a rollout, its writes disappear. A pod can also be deleted and replaced during a node upgrade. In either case, a file written only inside the container is not durable. For a stateless web app that is fine. For a database, upload folder, or ledger, it is a trap. An emptyDir volume lasts through a container restart within the same pod, but is deleted when that pod is removed; it is scratch space, not a persistent disk.
Persistent data belongs on storage mounted into the pod, with a lifecycle separate from that pod. A replacement can use the same bytes if the claim and backing storage still exist. The rest of this part names those objects and shows how the required sharing pattern guides the choice of backing storage.
PV, PVC, StorageClass — a request, a resource, and a factory
Three nouns carry the durable-storage model: declare what you need, provision it, and bind the two. A PersistentVolume (PV) represents a concrete piece of storage available to the cluster — backed here by an Azure disk or share. A PersistentVolumeClaim (PVC) is a separate Kubernetes object requesting storage: "I need ten gibibytes, writable from one node." A pod's specification references the claim, and Kubernetes binds the claim to a matching PV. The claim is the stable name your workload uses when a pod is replaced.
A StorageClass automates provisioning. It is a named recipe specifying the driver, storage tier, reclaim behavior, and when binding happens. When a PVC references it, the CSI driver can create a PV and its backing Azure resource. Some classes wait until a pod uses the claim so the disk can be placed in the right zone. The reclaim policy also matters: deleting a claim can delete its backing disk under a Delete policy. A PVC outlives a pod, not necessarily the PVC's own deletion. AKS installs several classes; inspect the one you use before storing important data.
The pod names what it needs. The class builds it.
The CSI driver — Container Storage Interface — is the plug-in that talks to Azure to create and attach storage. Azure Disk and Azure Files are the two choices to learn first. Azure Blob CSI is another option, but a mounted object store does not behave exactly like a normal disk. The StorageClass selects the driver and its configuration.
Follow one file through the three objects
Take one file, orders.log, that must survive a pod being replaced. Meet the three objects in the order you would touch them.
The claim, which you write. Ask for space; say nothing about Azure disks.
kind: PersistentVolumeClaim metadata: { name: orders-data } spec: accessModes: [ReadWriteOnce] storageClassName: managed-csi resources: { requests: { storage: 1Gi } }The pod later mounts orders-data by name. That name is all the workload ever knows.The class, which already exists. managed-csi is a StorageClass AKS ships with: a recipe that says "make an Azure managed disk with this driver". You did not write it; you named it. Run kubectl get storageclass to see it.
The volume, which appears. Once a pod uses the claim, the driver creates a disk and a PersistentVolume for it. Predict what kubectl get pv lists, then run it: one volume of 1Gi, status Bound, claim default/orders-data.
You wrote one object, named a second and watched the third appear. The storage lab does exactly this with a real pod.
Play it through
Three minutes. Watch an upload vanish when its pod is replaced, then attach a disk-backed claim and replace the pod again. The game stops when it needs your hands.
Do it for real. The lab for this class is on its own page: AKS storage that survives a pod. A PersistentVolumeClaim on an Azure Disk, a file written, the pod deleted, the file still there in its replacement. About twenty minutes.
Disk versus Files — one writer or many
The first question is whether the workload needs a shared filesystem across nodes. Azure Disk CSI attaches a managed disk to one node for read-write use. Its usual access mode is ReadWriteOnce (RWO): one node, not necessarily one pod.1 It is a sensible candidate for a single-writer workload, such as a database replica that owns its own files. If strict one-pod access matters, check whether the driver and class support ReadWriteOncePod; RWO alone is not that guarantee.
Azure Files CSI mounts an SMB or NFS file share and can provide ReadWriteMany (RWX), so pods on different nodes can use one shared directory. That suits shared content or uploads that multiple app instances must access. It does not make concurrent writes safe automatically; the application still needs to coordinate writes to the same file. Compare actual tier, capacity, transactions, throughput, and network requirements before claiming one option is cheaper or faster. Azure Blob CSI can mount object storage for suitable unstructured-data workloads, but object-storage behavior needs its own compatibility check. Disk and Files CSI drivers are normally enabled on AKS; Blob CSI needs enabling separately.
Start with this rule: one writer on one node: consider Disk and RWO; a filesystem shared across nodes: consider Files and RWX. Then test the workload's consistency, performance, recovery, and cost needs. Two pods on the same node can mount an RWO volume, so the access mode does not enforce one writer. A pod on another node cannot use that disk read-write while it is attached to the first. Likewise, RWX permits shared access; it does not make two database processes safe to point at the same data directory.
| Driver | What it is | Access mode | Reach for it when | Cost intuition |
|---|---|---|---|---|
| Azure Disk CSI | A managed disk, attached read-write to one node | ReadWriteOnce (RWO) | A database replica or other workload owning its own data | Compare tier, size, IOPS, and zone needs |
| Azure Files CSI | An SMB / NFS file share | ReadWriteMany (RWX) | Pods on different nodes sharing a directory | Compare capacity, operations, and throughput |
| Azure Blob CSI | Object storage mounted as a path | Driver and protocol dependent | Compatible unstructured-data workloads | Check API semantics and transfer costs |
The layering, drawn — pod to PVC to PV to Azure
The four nouns stack in a fixed order, and seeing the stack helps you debug it. A pod references a PVC; the PVC is bound to a PV; the PV is backed by Azure storage that a CSI driver may have provisioned from a StorageClass. If that pod is deleted and another pod mounts the same claim, it can read the same bytes while the claim and backing resource remain intact. Deleting the claim is a different operation: the StorageClass's reclaim policy decides what happens to the backing resource.
When a mount fails, check each layer instead of guessing: describe the pod for scheduling and attach events, the PVC for binding state, the PV for its class and zone, and the StorageClass for binding mode and reclaim policy. An access-mode mismatch or a disk already attached to a different node is one possible cause; credentials, quota, and driver errors are others. The event message tells you which path to investigate.
StatefulSet — when a pod needs a name and a disk of its own
A Deployment treats its pods as interchangeable. A StatefulSet is useful when replicas need stable, ordered identities and separate storage. It names pods db-0, db-1, and db-2. With a volumeClaimTemplates entry, each gets its own PVC. Delete db-1 and its replacement can mount the same claim. A StatefulSet supplies identity and storage wiring; it does not configure database replication, backups, or failover for you.
Most web services in this course are stateless and fit a Deployment. For a stateful workload, decide first whether a managed data service is the better operational choice. If you do run replicas in the cluster, a StatefulSet can supply stable names and per-replica claims, while the application's own design must handle data consistency and recovery.
The partner integration needs to remember a small ledger
The partner-integration gateway needs a reconciliation ledger: a record of wholesale orders it has handed off, so a restart can resume safely. Its reconciler is designed as one writer on one node, so an Azure Disk via a PVC is a reasonable candidate. But RWO itself does not prevent another pod on that node from mounting the disk. Keep the reconciler single-writer through deployment design, and make the handoff idempotent so a crash between writing the ledger and sending an order cannot double-send it. A file share is useful only if a real cross-node sharing need emerges; compare tiers and costs before claiming a price advantage.
And you say the other half out loud too, because a colleague floats it: the storefront's real database is not moving onto the cluster to sit next to this. That database stays where it is, on managed Azure data services outside AKS, exactly as the estate has run it. The ledger is a few megabytes the integration owns; the storefront database is a production system of record with its own backups, scaling, and blast radius. Putting it on the cluster "while we have Kubernetes" would grow the cluster past the size of the contract that justified it — the same discipline as every part before this one.2
The volume that will not mount
A pod sits in ContainerCreating. kubectl describe pod reports that its disk is already attached to another node; kubectl describe pvc shows the claim is Bound. Those events point to a node-level RWO conflict. Check whether the workload actually needs multiple nodes to mount one shared directory or whether each replica should have its own claim. Moving to a StatefulSet with per-replica claims solves the second design; Azure Files with RWX may solve the first, if the application safely supports shared writes.
Examination
Four drills, then two situations. The situations have no marking scheme — write your answer before you reveal the reasoning, or the exercise is worthless. Nothing is stored.
B. The new container does not inherit writes from the old container's writable layer. A assumes those writes persist; C invents an automatic copy that does not exist. D confuses pod management with storage: a Deployment creates replacements, but does not preserve files from their containers. A persistent volume is the mechanism to use when those files must survive.
B. Azure Files can provide one directory mounted from pods on different nodes. A single RWO disk cannot be attached read-write to all three nodes. A pod-local emptyDir would give each pod a different, temporary folder. Synchronizing separate disks could be built, but adds its own consistency and failure problems. A share solves the mount pattern; your application still needs safe concurrent-write behavior.
The surviving PVC, dynamic provisioning, and the StatefulSet with a claim template. RWO restricts the read-write mount to one node, so it cannot serve pods on different nodes at the same time. Files written only to a container's writable layer disappear when that container is replaced. A PVC is a separate object; keeping that claim lets a replacement pod use the same bound volume. The StatefulSet claim template creates a distinct claim for each replica.
# plan: run our reporting Postgres on AKS
1. Run it as a StatefulSet so each replica keeps
a stable identity and its own disk.
2. Back each replica with an Azure Disk PVC,
ReadWriteOnce, one disk per pod.
3. To let all three replicas share one data
directory, put it on a ReadWriteMany Files share.
4. Keep the storefront's production database off
the cluster, on managed Azure data services.
Line three. It contradicts everything the plan got right. Each database replica is a single writer that assumes it alone owns its files — which is precisely why lines one and two are correct, giving every replica its own ReadWriteOnce disk. Line three then tears that up by pointing all three replicas at one shared ReadWriteMany directory, which is how you get two processes writing the same data files at once and the kind of corruption no backup schedule fully undoes. Replicas share data through the database's own replication protocol over the network, never by mounting the same disk.
For this proposed multi-replica database, a StatefulSet with a separate claim per replica is a sensible design, so A is wrong. RWO permits each disk to be mounted read-write on one node, so B is wrong too, though the database still needs its own single-writer safeguards. Line four follows the case file: the storefront's existing system of record stays on its managed data service. Lines one and two do not make the shared-directory idea in line three safe.
Start with the access pattern. RWX permits a shared mount across nodes; it does not make concurrent writes safe. The gateway's ledger is designed for one reconciler, so cross-node sharing is not currently a requirement. That makes a disk-backed claim a candidate, subject to the workload's recovery and performance needs.
Name what the wider access actually changes. An Azure Files share can be mounted across nodes. A disk with RWO is read-write on one node at a time, but a second pod on that same node can still mount it; RWO is not a one-process lock. If this ledger must have one writer, enforce that through the workload design and, where supported, consider ReadWriteOncePod. Compare the relevant storage tiers and workload measurements before making a cost or speed claim.
Close with the decision. Use a disk-backed claim for this one-node reconciler if its measurements support it, and keep the application single-writer. If several pods on different nodes truly need the same directory, consider Azure Files with RWX and design their writes safely. Verify cost and performance for the selected tier rather than assuming a universal winner.
Separate the technical "can we" from the operational "should we." Yes, Kubernetes can run a database — a StatefulSet on ReadWriteOnce disks is exactly the tool, and this part taught it. So the objection is not that it is impossible. The objection is that running a production system of record on the cluster means you now own its durability, backups, failover, version upgrades, and disk performance by hand, on top of operating the cluster itself. Managed Azure data services already carry all of that for you, and moving the database onto AKS trades a solved problem for an unsolved one in the name of neatness.
Price the tidiness against the blast radius. "One place" sounds efficient until a routine node upgrade, a bad rollout, or a full disk touches the one system whose corruption ends the business. The storefront database is the record of every order and customer; its failure mode is not a restarted pod but a company-level incident. Keeping it off the cluster keeps that blast radius separate from the churn of a cluster you upgrade and redeploy constantly. The reconciliation ledger belongs on the cluster because the integration owns it and it is a few megabytes; the storefront database does not, because it is a different class of thing.
Close on the discipline the whole track has been teaching. The cluster should be exactly the size of the contract that justified it — the partner integration and nothing more. "We have Kubernetes, so put everything on it" is precisely the pressure that turns a small, boring, operable cluster into a sprawling one nobody can reason about. Say yes to the ledger, no to the database, and be able to explain that the line is drawn by ownership and blast radius, not by what happens to be nearby.
Five things worth carrying out of Part D
- A container's writable layer disappears when that container is replaced. An emptyDir lasts only as long as its pod. A PVC-backed volume can survive replacement of the pod.
- A PVC is the pod's request, a PV is the concrete storage, a StorageClass is the factory that dynamically provisions PVs. The pod mounts the claim, never the disk directly.
- Start with the access pattern: one node's writer can use Azure Disk and RWO; a directory shared across nodes can use Azure Files and RWX. RWO is not a one-pod lock.
- The stack is pod → PVC → PV → Azure storage. Replace the pod while keeping its claim and the bytes remain available; delete the claim and the reclaim policy matters.
- A StatefulSet supplies stable pod identities and, with volumeClaimTemplates, a separate PVC per replica. It does not provide database replication, backups, or failover.
- Kubernetes defines ReadWriteOnce as read-write access by one node, not one pod: two pods on that node may mount the same volume. ReadWriteOncePod is the stricter, CSI-supported mode when one pod at a time must mount it. Even then, application-level consistency and backup design remain necessary. ↩
- Running stateful workloads on Kubernetes at all remains a genuine debate, and the direction is worth more than any single verdict here: the platform can do it, managed data services usually do it better, and the right answer is workload-specific. Treat "databases on the cluster are always wrong" and "put everything on the cluster" as equally lazy. The discipline is to weigh ownership, blast radius, and who already carries the backups — as the second situation does — rather than to follow a slogan either way. ↩
Give a pod storage that survives a restart, and choose the right storage class.
Play the lesson above to stamp this class.