Workspace support for VolumeClaimTemplates

Expected Behavior

This issue started out as a general discussion on a mechanism for dynamically allocating storage for TaskRuns at runtime but has become focused on volume claim templates. So now this issue is for discussion of Volume Claim Templates to define PVCs in workspaces at runtime, similarly to the way StatefulSets and Argo handle storage (see comments in this PR for links to each of these).

See this issue about the artifact configmap for discussion of Workspaces supporting the config-artifact-pvc approach that we use with PipelineResources and to revisit the conversation around "Auto Workspaces".

Actual Behavior

Users have to explicitly define the PersistentVolumeClaim configuration every time they bind a workspace that spans multiple tasks.

[siamaksade]

That's a interesting idea. The default could be true which is a similar experience from other CI systems.

[bobcatfish]

Could use the configuration we use currently for pipelineresources (or similar): https://github.com/tektoncd/pipeline/blob/master/docs/install.md#how-are-resources-shared-between-tasks

[bobcatfish]

Discussed in beta working group and it doesn't feel like we need this for beta at least, so we want to wait until a user indicates they definitely need this functionality.

[rakhbari]

My 2-cents... Any feature that assumes the ability to automatically create a PVC (PVs, attaching volumes, or anything to do with underlying storage resources of any kind) is a no-can-do for us. Any resource like that must be pre-allocated and pre-created and its reference injected into a PipelineRun/TaskRun.

This is why Tekton pipelines simply wasn't usable for us here at eBay until the Workspaces was implemented. As such, I see no need for such feature as our underlying infrastructure would be unable to support it in the first place.

[vdemeester]

My 2-cents... Any feature that assumes the ability to automatically create a PVC (PVs, attaching volumes, or anything to do with underlying storage resources of any kind) is a no-can-do for us. Any resource like that must be pre-allocated and pre-created and its reference injected into a PipelineRun/TaskRun.

It could be a configuration switch (that is on or off by default)

In #2174 @tstmrk brought up the example of volumeClaimTemplates, which are a feature of Stateful Sets. They provide a similar kind of system to what we're discussing here I think, although they don't clean up the volumes after the Pod is destroyed. In our case we might want to do that when a Task or PIpeline have completed.

volume claim templates were mentioned again in #2218 as a possible implementation for this feature. Useful to know: they are used in Argo Workflows for a similar use case. Quoting @cameronbraid :

I would like to be able to have workspace PVC's creation and deletion fully managed by tekton using a volume claim template.

Currently this is supported in Argo Workflow and is a very useful feature (https://github.com/argoproj/argo/blob/master/examples/README.md)

  volumeClaimTemplates:                 # define volume, same syntax as k8s Pod spec
  - metadata:
      name: workdir                     # name of volume claim
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi                  # Gi => 1024 * 1024 * 1024

The PVC would be created when the pipeline is executed, and deleted afterwards

This would conceptually be comparable to an emptyDir volume, however with the ability to specify a storage class for the PVC and not relying on a folder on the host as emptyDir does.

The current drawback of the workspace PersistentVolumeClaim is that issues would arise if two pipelines run concurrently that use the same PVC