Feature/plan pvc support

[itamar-marom]

#336

With this PR, we can store terraform plans in Kubernetes PersistentVolumeClaim.
The given PVC should exist already and is not part of the controller's responsibility.

• Also supports all current options of storeReadablePlan

The runner will mount into a subPath that points to its own directory. The path looks like this:
/mnt/plan/terraform_controller/TERRAFORM_OBJECT_NAMESPACE/TERRAFORM_OBJECT_NAME/
This gives the security of a runner to mount only into its own object and also takes care the backend storage is organized and can serve other stuff then the terraform controller.

Example:

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: plan-config-claim-pv
  labels:
    type: local
spec:
  storageClassName: standard
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/tmp/plan-test"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: plan-config-claim-pvc
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: infra.contrib.fluxcd.io/v1alpha1
kind: Terraform
metadata:
  name: helloworld-plan-config-claim
spec:
  interval: 10s
  approvePlan: "auto"
  storeReadablePlan: "human"
  path: ./
  planConfig:
    storage:
      claimName: "plan-config-claim-pvc"
  sourceRef:
    kind: GitRepository
    name: helloworld

[itamar-marom]

Hi @yitsushi
Are you into reviewing this also? If so I will fix the conflicts.
Link to issues: #336 #536

[yitsushi]

You can, but I can't promise I'll have time to review in the near future as it requires changes we have to consider if we even want/can do.

1. It'll conflict with main when Feature/add tfvarspaths spec2 #1185 is merged, so you would have to resolve again.
2. It adds an extra resource to manage and we are already working on a solution to reduce that dependency or at least be able to reconcile is those resources are not available.

One of the main goals of the Workspace BLOB cache initiative is to prevent stuck resources when Source gets deleted before the Terraform resource. With this addition we would have to think about that extra dependency too, cache somewhere, manage what happens if it can't mount the resource yet still don't get stuck on a reconcile loop unable to even delete the resource without patching the finalizer and hacking the system hoping everything works out.

Any ideas, suggestions, opinions @chanwit ?