GKM_PVC_Exploration_Summary.txt

Requirement:
* Same logical data on all nodes
* Multiple nodes
* Underlying storage only supports RWO
* Pods should not need to know node-specific PVC names
* Operator should abstract this away
* Data pre-populated before workload starts
* Pod startup must be fast (no on-demand copy)

Does GKM Need to support:
* Pod (Pod on One Node)
* DaemonSet (Pod on Every Node)
* ReplicaSet (Fixed Number of Pods)
* Deployment (Deploys ReplicaSets - for migration of version)
* StatefulSet (Deploys ReplicaSets - but pods are fix)

Options:
========
X - Implies won't work
O - Implies works, but caveots
@ - Implies works as we want (haven't found yet)


X-Option: Pod spec references CSI Driver in Volume, and CSI driver determines the PVC Name based on the Node.

* A CSI driver does NOT mount a PVC into a Pod. Kubernetes does that. Your CSI Node plugin only mounts
  a volume to a path that kubelet gives you (target_path). Kubelet then bind-mounts that into the Pod.
* When CSI is called, VolumeId must be known, which means PVC is known.


X-Option: Use mutating admission webhook to modify the PVC in the Volume in the Pod Spec to the node specific PVC.

* In the CREATE, the pod has not been scheduled, so the Node is not known. Therefore the Mutating webhook
  can't know which PVC to update the Pod Spec too.
* In the MODIFY, the pod is already up and volumes have been mounted. Too late to update.


X-Option: Create a PV per Node, and one PVC. When the Pod is scheduled, Kubelet binds the PV to a PVC.

* Will work if there is only one workload Pod.
* Not sure what happens if the Pod dies and has to be rescheduled.
* Actually, won't work because we want the PVC pre-populated, which means the PVC is bound to the PV
  when the data is extracted into the PVC.


O-Option: Mutating Webhook, but need to use DaemonSet

* A DaemonSet knows the Node on CREATE, so a mutating webhook would work in this scenario.


O-Option: Operator Generates the Workload

* If operator generates the workload Pod, it can hardcode the PVC name into the Pod Spec.
  However, it needs to force the Pod to run a specific Node with a NodeSelector. Lose some of Kubernetes
  Pod Scheduling.
* Operator can also generate a workload DaemonSet. Then Node is known at start.
* Operator generating workload Pod Sepc is more common than I originally thought.


O-Option: Run the Workload in a StatefulSet.

* Need to investiate StatefulSets more, but my understanding is that pods are fixed to nodes.


Random Notes:
* A CSI driver can access the Kubernetes API, but:
  * Safe to call from Controller plugin
  * Can call from Node plugin, but be careful and not recommended
  * Cannot call from Sidecars (not yours to modify)
  Better Pattern is to:
  * Let a controller reconcile your CR
  * Store required data in:
    * PVC annotations
    * VolumeAttributes
    * Secret
  * CSI receives it via CreateVolumeRequest
* A PVC binds once, to exactly one PV for it's lifetime.