Highlights
- Introduced a flexible, configurable, tiered data locality scheduling capability to optimize the affinity between application pods and data cache.
- Provided DataProcess for custom data operations, and add three trigger strategies for data operations for all the data operations – once, onEvent, Cron.
- Provided a Python SDK with dataflow capabilities for data scientists.
- Implemented A new runtime for sharing in-memory immutable data: VineyardRuntime is supported in Fluid.
- Major improvements have been made in:
- Improved security measures by defining stricter, minimum necessary cluster role permissions for Fluid components.
- Significantly optimized FUSE Recovery features to enhance their performance and reliability.
- Made significant updates to the project documentation and website.
We acknowledge all the contributors who put their effort in the version release. Credits to @zwwhdls, @xliuqq, @wangshli, @Hexilee, @weixiao-huang, @abowloflrf, @uniqueni, @TrafalgarZZZ, @dashanji, @cheyang, @fengshunli, @frankleaf, @zhang-x-z, @BillyChen1, @wushiming540, @RongGu and many others for their great contributions to Fluid v1.0.0 !
New Features
Data Operation Features
- Add cron for data migrate.
- Support cron dataload.
- Make toleration configurable in helm charts.
- Data process crd definition.
- Support dataflow.
- Data operations support resources.
- Support ssh password free pod for parallel data migrate.
Vineyard Runtime
- Add the Vineyard runtime CRD definitions.
- Add the helm chart for Vineyard runtime.
- Add the controller and RBAC yaml for Vineyard runtime.
- Implement Vineyard runtime engine and controller.
- Add a replicas field to the Vineyard Runtime and delete the svc suffix in the Vineyard helm chart.
Enhancements & Optimizations
CSI Plugin & FUSE Recovery
- Check path existence and lock in NodeUnpublishVolume.
- umountDuplicate larger than the threshold.
- Fix csi plugin concurrency issue on FuseRecovery and NodeUnpublishVolume.
- Enhancement: recover fuse according to multiple peer group options.
- Support symlink for NodePublish.
Bug Fix
- Bugfix: Support batch/v1beta1 cronjobs for compatibility before Kubernetes v1.21.
- Bugfix: fix csi plugin loop mount bug
- Bugfix: reconcile thinruntime failed when dataset is deleted
- Bugfix:fix thinruntime stuck bug when deleting it
- Bugfix:fix worker&fuse options & worker tiredstore
- Bugfix: del metadata sync && del duplicate metrics
- Bugfix: clean up orphaned thinruntime resources
- Bugfix: fix Alluxio master in HA mode start error
- Bugfix: pass AccessModes to thinruntime fuse container
- Bugfix: fix fatal error: concurrent map writes for runtime controllers
- Bugfix: fix incorrect conversion between integer types
Website & Documentation Update
Full Changelog: v0.9.0...v1.0.0
Contributors
RongGu, cheyang, and 14 other contributors