csi: add cephfs encryption support

[NymanRobin]

Ceph-CSI support for fscrypt encryption of cephfs.To achieve this commit add capability of mounting the required rook-ceph-csi-kms-config configmap into csi-cephfsplugin-provisioner and nodeplugin pods.

Further it modifies the modifies the ClusterRoles cephfs-csi-nodeplugin and cephfs-external-provisioner-runner to grant privileges required for reading encryption configuration and fetching encryption secrets from either kubernetes secrets or Key Management System (KMS).

These privileges are essential for the proper functioning of ceph-csi-cephfs with fscrypt encryption.

The following privileges have been added:

• secrets/get: Allows reading of secrets for encryption.
• configmaps/get: Grants access to configuration maps, this is used to read encryption configuration.
• serviceaccounts/get: Enables retrieval of service accounts for authentication to KMS and encryption secret retrieval from there.
• serviceaccounts/token/create: Allows creation of service account tokens, required for authenticating request to KMS when retrieving encryption secrets.

The commit also updated the csi documentation to include cephfs in the encryption section, with examples updated accordingly.

This changes fixes: #14035

[parth-gr]

Should we also document how to use CephFS + fscrypt?

[NymanRobin]

Yes, ideally, @parth-gr, but currently, I don't have an official ISO image that supports kernel 6.6 and works with Minikube for development purposes. However, if it's acceptable to leave the kernel-related details to the reader, I can proceed with documenting the process and create some deployment examples.

[parth-gr]

Yes, ideally, @parth-gr, but currently, I don't have an official ISO image that supports kernel 6.6 and works with Minikube for development purposes. However, if it's acceptable to leave the kernel-related details to the reader, I can proceed with documenting the process and create some deployment examples.

lets wait for there ideas @travisn @iamniting

[iamniting]

While talking to @Rakshith-R I got to know we also need the below permissions in both the clusterroles

  - apiGroups: [""]
    resources: ["serviceaccounts"]
    verbs: ["get"]
  - apiGroups: [""]
    resources: ["serviceaccounts/token"]
    verbs: ["create"]

[NymanRobin]

While talking to @Rakshith-R I got to know we also need the below permissions in both the clusterroles

  - apiGroups: [""]
    resources: ["serviceaccounts"]
    verbs: ["get"]
  - apiGroups: [""]
    resources: ["serviceaccounts/token"]
    verbs: ["create"]

Thanks for pointing it out, I had only used it with a secret directly and thus I missed this detail!
The PR is updated now! What does everyone think should this PR include some documentation for the cephfs + fscrypt feature or should we target that for another PR?

[travisn]

This has expanded to more than helm. How about the commit prefix manifest:, and also more description in the commit message for the privileges being added?

[iamniting]

The changes looks good to me, I leave the other decisions to Travis.

[NymanRobin]

Thanks @travisn for pointing out that commit message was not descriptive, sorry for that!

Also managed to close the PR once while updating it 😅
Now the commit message should be more descriptive of what privileges are added and what is the reason behind them

[parth-gr]

Should we also document how to use CephFS + fscrypt?

we should have the feature documented IMO, with this PR or a follow up never mind

[travisn]

LGTM, @Rakshith-R please also review

[Rakshith-R]

Thanks,
It looks good to me.

[Madhu-1]

If the target is only to update RBAC the changes looks good but for complete implementation, As per we are enabling a Key CSI_ENABLE_ENCRYPTION in the configmap https://rook.io/docs/rook/latest-release/Storage-Configuration/Ceph-CSI/ceph-csi-drivers/?h=encryption#enable-rbd-encryption-support, when this key is set we need to mount configmap to the cephfs as well like we do for rbd which is missing in this PR.

[NymanRobin]

I see your point @Madhu-1, I did not initially considered it in this PR as it is outside of the scope of the issue I submitted.

However, I am not opposed to extending the scope of this PR as it probably would make most sense to finish it all at once, depending on what others who have partaken in the conversation think? I should be able to extend this PR in reasonable time
I also see from the link that the rbd case just have a note for the kernel version so I think we can simply do the same for cephfs case

I found the rbd PR #9940 for reference

[Madhu-1]

I see your point @Madhu-1, I did not initially considered it in this PR as it is outside of the scope of the issue I submitted.

However, I am not opposed to extending the scope of this PR as it probably would make most sense to finish it all at once, depending on what others who have partaken in the conversation think? I should be able to extend this PR in reasonable time I also see from the link that the rbd case just have a note for the kernel version so I think we can simply do the same for cephfs case

I found the rbd PR #9940 for reference

@NymanRobin yes you are correct, Thanks for looking for the RBD implementation as well. For me the change is not that big we can accommodate in this PR and get it done instead of sending one more PR for implementation and document.

[NymanRobin]

I have updated the pull request to include the mounting of the ConfigMap and the relevant documentation.
I extended the existing documentation since this feature utilizes the same parameter and ConfigMap. @Madhu-1, please review these changes and let me know if you have any concerns or suggestions. It was not 100% clear to me if these both types of encryption should be behind the same configuration