Releases: openebs/openebs
Releases · openebs/openebs
v0.6
Getting Started
Using kubectl
kubectl apply -f https://raw.githubusercontent.com/openebs/openebs/v0.6/k8s/openebs-operator.yaml
kubectl apply -f https://raw.githubusercontent.com/openebs/openebs/v0.6/k8s/openebs-storageclasses.yaml
Using Kubernetes Stable Helm charts
helm install --namespace openebs --name openebs -f https://openebs.github.io/charts/helm-values-0.6.0.yaml stable/openebs
kubectl apply -f https://raw.githubusercontent.com/openebs/openebs/v0.6/k8s/openebs-storageclasses.yaml
Using OpenEBS Helm Charts (will be deprecated in the coming releases)
helm repo add openebs-charts https://openebs.github.io/charts/
helm repo update
helm install openebs-charts/openebs
For more details refer to the documentation at: https://docs.openebs.io/
New Capabilities / Enhancements
- Integrate the Volume Snapshot capabilities with Kubernetes Snapshot controller
- Enhance maya-apiserver to use CAS Templates for orchestrating new Storage Engines
- Enhance mayactl to provide additional details about volumes such as - replica status and node details where replicas are running.
- Enhance maya-apiserver to schedule Replica Pods on specific nodes using nodeSelector
- Enhance provisioner and maya-apiserver to allow specifying cross AZ scheduling of Replica Pods.
- Support for deploying OpenEBS via Kubernetes stable Helm Charts
- openebs-operator.yaml is modified to run OpenEBS pods in its own namespace
openebs - Enhance e2e tests to simulate chaos at different layers such as - CPU, RAM, Disk, Network, and Node
Major Issues Fixed
- Fixed an issue where intermittent connectivity errors between controller and replica caused iSCSI initiator to mark the volume as read-only. openebs/gotgt#15
- Fixed an issue where intermittent connectivity errors were causing the controller to silently drop the replicas and mark the Volumes as read-only. The replicas dropped in this way were not getting re-added to the Controller. openebs/jiva#45
- Fixed an issue where volume would be marked as read-only if one of the three replicas returned an error to IO. openebs/jiva#56
- Fixed an issue where replica fails to register back with the controller if the attempt to register occurred before the controller cleared the replica's previous state. openebs/jiva#56
- Fixed an issue where a volume with a single replica would get stuck in the read-only state once the replica was restarted. openebs/jiva#45
Upgrade from older releases
Since 0.6 has made changes to the way controller and replica pods communicate with each other, the older volumes need to be upgraded with scheduled downtime for applications.
Limitations
- For OpenEBS volumes configured with more than 1 replica, at least more than half of the replicas should be online for the Volume to allow Read and Write. In the upcoming releases, with cStor data engine, Volumes can be allowed to Read/Write when there is at least one replica in the ready state.
- This release contains a preview support for cloning an OpenEBS Volume from a snapshot. This feature only supports single replica for a cloned volume, which is intended to be used for temporarily spinning up a new application pod for recovering lost data from the previous snapshot.
- While testing for different platforms, with a three-node/replica OpenEBS volume and shutting down one of the three nodes, there was an intermittent case where one of the 2 remaining replicas also had to be restarted.
- The OpenEBS target (controller) pod depends on the Kubernetes node tolerations to reschedule the pod in the event of node failure. For this feature to work, TaintNodesByCondition alpha feature must be enabled in Kubernetes. In a scenario where OpenEBS target (controller) is not rescheduled or is back to running within 120 seconds, the volume gets into a read-only state and a manual intervention is required to make the volume as read-write.
- The current version of OpenEBS volumes are not optimized for performance sensitive applications.
- For a more comprehensive list of open issues uncovered through e2e, please refer open issues.
Additional details are available on Project Tracker Wiki.
v0.5.4
Issues Fixed in v0.5.4
- Provision to specify filesystems other than ext4 (default) in the OpenEBS provisioner spec (#1454 )
- Support for xfs filesystem format for mongodb statefulset using OpenEBS Persistent Volume (#1446 )
Known Issues in v0.5.4
For a complete list of known issues, go to v0.5.4 known issues
- xfs formatted volumes are not remounted post snapshot reverts/forced restarts (bugs)
- Requires Kubernetes 1.7.5+
- Requires iSCSI initiator to be installed in the Kubernetes nodes or kubelet container
- Not recommended for mission critical workloads
- Not recommended for performance sensitive workloads. Ongoing efforts intended to improve performance
Enhancements
- OpenEBS is now available as a stable chart from Kubernetes (https://github.com/kubernetes/charts/tree/master/stable/openebs)
- Increased integration test & e2e coverage in the CI
Installation
Using kubectl
kubectl apply -f https://raw.githubusercontent.com/openebs/openebs/v0.5.4/k8s/openebs-operator.yaml
Using helm
helm repo add openebs-charts https://openebs.github.io/charts/
helm repo update
helm install openebs-charts/openebs
Alternately, refer : https://docs.openebs.io/docs/next/installation.html#install-openebs-using-helm-charts
v0.5.3
Issues Fixed in v0.5.3
- Fixed usage of StoragePool issue when rbac settings are applied 1189.
- Fixed hardcoded maya-apiserver-service name to a configurable value as it resulted in conflict with other services running on the same cluster 1227.
- Fixed an issue where the OpenEBS iSCSI volume showed progressive increase in the memory consumed by the controller pod 1298.
Known Issues in v0.5.3
For a complete list of known issues, go to v0.5.3 known issues.
- Requires Kubernetes 1.7.5+
- Requires iSCSI initiator to be installed in the Kubernetes nodes or kubelet container
- Not recommended for mission critical workloads
- Not recommended for performance sensitive workloads. Ongoing efforts intended to improve performance
Enhancement to Documentation
The OpenEBS documentation is now available at https://docs.openebs.io/. You can provide your feedback comments by clicking the Feedback button provided on every page.
Installation
Using kubectl
kubectl apply -f https://raw.githubusercontent.com/openebs/openebs/v0.5.3/k8s/openebs-operator.yaml
Using helm
helm repo add openebs-charts https://openebs.github.io/charts/
helm repo update
helm install openebs-charts/openebs
v0.5.2
This is a single-fix release on top of v0.5.1 to allow maya-apiserver and openebs-provisioner to work with Kubernetes non-SSL configuration.
Issue Fixed:
- #1184 : You can set the non-SSL Kubernetes endpoints to use by specifying the ENV variables OPENEBS_IO_KUBE_CONFIG and OPENEBS_IO_K8S_MASTER on maya-apiserver and openebs-provisioner.
To use the above ENV variables, the following image versions have to be used:
- openebs/m-apiserver:0.5.2: OpenEBS Maya API Server along with the latest maya cli.
- openebs/openebs-k8s-provisioner:0.5.2: Dynamic OpenEBS Volume Provisioner for Kubernetes.
v0.5.1
This is a incremental release on top of v0.5. This release fixes bugs and adds support for running OpenEBS on CentOS based Kubernetes Cluster including OpenShift 3.7+
Issues Fixed in v0.5.1
- Fix the inter-operability issues of connecting to OpenEBS Volume from CentOS iSCSI Initiator (#1087)
- Fix openebs-k8s-provisioner to be launched in non-default namespace (#1055)
- Update the documentation with steps to use OpenEBS on OpenShift Kubernetes Cluster (#1102) and Kubernetes on CentOS (#1104)
- Update helm charts to use OpenEBS 0.5.1 (#1100)
Known Limitations
- Requires Kubernetes 1.7.5+
- Requires iSCSI initiator to be installed in the Kubernetes nodes or kubelet container
- Not recommended for mission critical workloads
- Not recommended for performance sensitive workloads. Ongoing efforts intended to improve performance
Installation
Using kubectl
kubectl apply -f https://raw.githubusercontent.com/openebs/openebs/v0.5.1/k8s/openebs-operator.yaml
Using helm
helm repo add openebs-charts https://openebs.github.io/charts/
helm repo update
helm install openebs-charts/openebs
Images
- openebs/jiva:0.5.1 : Containerized Storage Controller
- openebs/m-apiserver:0.5.1 : OpenEBS Maya API Server along with the latest maya cli.
- openebs/openebs-k8s-provisioner:0.5.1 : Dynamic OpenEBS Volume Provisioner for Kubernetes.
- openebs/m-exporter:0.5.1 : OpenEBS Volume metrics exporter.
Setup OpenEBS Volume Monitoring
If you are running your own prometheus, please update it with the following job configuration:
- job_name: 'openebs-volumes'
scheme: http
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels: [__meta_kubernetes_pod_label_monitoring]
regex: volume_exporter_prometheus
action: keep
- source_labels: [__meta_kubernetes_pod_name]
action: replace
target_label: kubernetes_pod_name
- source_labels: [__meta_kubernetes_pod_label_vsm]
action: replace
target_label: openebs_pv
- source_labels: [__meta_kubernetes_pod_container_port_number]
action: drop
regex: '(.*)9501'
- source_labels: [__meta_kubernetes_pod_container_port_number]
action: drop
regex: '(.*)3260
If you don't have prometheus running, you can use the following yaml file to run prometheus and grafana.
kubectl apply -f https://raw.githubusercontent.com/openebs/openebs/v0.5.0/k8s/openebs-monitoring-pg.yaml
You can import the following grafana-dashboard file to view the OpenEBS Volume metrics.
v0.5.0
This release marks a significant milestone for OpenEBS. We are excited about the new capabilities - like policy based Volume Provisioning and Customizations that will finally provide DevOps teams, the missing tools to automate the StorageOperations. We are more excited about the contributions that poured in from 50+ new community members that have made this release possible.
Changelog
- Storage Policy Enforcement Framework that allows DevOps teams to deploy a customized storage. Some policies supported are:
- Storage Policy - for using a custom Storage Engine like Jiva,
- Storage Policy - for exposing volume metrics in Prometheus format using a side-car to volume controller
- Storage Policy - for defining Capacity
- Storage Policy - for defining the persistent storage location like /var/openebs (default) or a directory mounted on EBS or GPD etc.,
- Extend OpenEBS API Server to expose volume snapshot api
- Support for deploying OpenEBS via helm charts
- Sample Prometheus configuration for collecting OpenEBS Volume Metrics
- Sample Grafana OpenEBS Volume Dashboard - using the prometheus Metrics
- Sample Deployment YAMLs and corresponding Storage Classes for different types of applications (see Project Tracker Wiki for detailed list)
- Sample Deployment YAMLs for launching Kubernetes Dashboard for a preview of the changes done by OpenEBS Team to Kubernetes Dashboard (see Project Tracker Wiki for the PRs raised and merged)
- Sample Deployment YAMLs for Prometheus and Grafana - in case they are not already part of your deployment.
- Several Documentation and Code Re-factoring Changes for improving code quality
Additional Details are available on Project Tracker Wiki
Changes from earlier releases to v0.5.0
- Some of the ENV variables for customizing default options have changed (openebs/openebs #927)
- DEFAULT_CONTROLLER_IMAGE -> OPENEBS_IO_JIVA_CONTROLLER_IMAGE
- DEFAULT_REPLICA_IMAGE -> OPENEBS_IO_JIVA_REPLICA_IMAGE
- DEFAULT_REPLICA_COUNT -> OPENEBS_IO_JIVA_REPLICA_COUNT
Known Limitations
- Requires Kubernetes 1.7.5+
- Requires iscsi initiator to be installed in the kubernetes nodes or kubelet container
- Has been tested primarily with enabling openebs and its volumes (PVCs) in the default namespace
- Not recommended for mission critical workloads
- Not recommended for performance sensitive workloads. Ongoing efforts intended to improve performance
Installation
Using kubectl
kubectl apply -f https://raw.githubusercontent.com/openebs/openebs/v0.5.0/k8s/openebs-operator.yaml
Using helm
helm repo add openebs-charts https://openebs.github.io/charts/
helm repo update
helm install openebs-charts/openebs
Images
- openebs/jiva:0.5.0 : Containerized Storage Controller
- openebs/m-apiserver:0.5.0 : OpenEBS Maya API Server along with the latest maya cli.
- openebs/openebs-k8s-provisioner:0.5.0 : Dynamic OpenEBS Volume Provisioner for Kubernetes.
- openebs/m-exporter:0.5.0 : OpenEBS Volume metrics exporter.
Setup OpenEBS Volume Monitoring
If you are running your own prometheus, please update it with the following job configuration:
- job_name: 'openebs-volumes'
scheme: http
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels: [__meta_kubernetes_pod_label_monitoring]
regex: volume_exporter_prometheus
action: keep
- source_labels: [__meta_kubernetes_pod_name]
action: replace
target_label: kubernetes_pod_name
- source_labels: [__meta_kubernetes_pod_label_vsm]
action: replace
target_label: openebs_pv
- source_labels: [__meta_kubernetes_pod_container_port_number]
action: drop
regex: '(.*)9501'
- source_labels: [__meta_kubernetes_pod_container_port_number]
action: drop
regex: '(.*)3260
If you don't have prometheus running, you can use the following yaml file to run prometheus and grafana.
kubectl apply -f https://raw.githubusercontent.com/openebs/openebs/v0.5.0/k8s/openebs-monitoring-pg.yaml
You can import the following grafana-dashboard file to view the OpenEBS Volume metrics.
v0.4.0
Please try out the latest OpenEBS on your Kubernetes Cluster using the following quick start guide. https://docs.openebs.io/docs/runOpenEBSoperator.html
The following OpenEBS v0.4.0 containers are available from Docker Hub:
- openebs/jiva:0.4.0 : Storage Controller
- openebs/m-apiserver:0.4.0 : OpenEBS Maya API Server along with the latest maya cli.
- openebs/openebs-k8s-provisioner:0.4.0 : Dynamic OpenEBS Volume Provisioner for Kubernetes.
New v0.4.0 features
- Maya CLI Support for managing snapshots for OpenEBS Volumes
- Maya CLI Support for obtaining the capacity usage statistics from OpenEBS Volumes
- OpenEBS Volume - Dynamic Provisioner is merged into kubernetes-incubator/external-storage project.
- OpenEBS Maya API Server uses the Kubernetes scheduler logic to place OpenEBS Volume Replicas on different nodes.
- OpenEBS Maya API Server can be customized by providing ENV options through K8s YAML file for default replica count and jiva image to be used.
- OpenEBS User Documentation is made avilable at https://docs.openebs.io/
- OpenEBS now supports deployment on AWS, along with previously supported Google Cloud and On-premise setups
- OpenEBS Vagrant boxes are upgraded to support Kubernetes version 1.7.5
- OpenEBS can now be deployed within a minikube setup
Notable Issues Fixed in v0.4.0
- #166 (https://github.com/edorid): openebs-k8s-provisioner goes into crashloopbackoff, during the first volume creation
- #176 (https://github.com/maikotz): OpenEBS PV unreachable after one of the replica became unreachable.
CI Updates with v0.4.0
- Support for on-premise Jenkins CI for performing e2e tests
- iSCSI compliance tests are run as part of the CI
- CI can now be extended using a framework developer for running storage benchmark tests with vdbench or fio.
- CI has been extended to run Percona Benchmarking tests on Kubernetes.
Deprecated with v0.4.0
- The maya cli options (setup-omm, setup-osh, omm-status, osh-status) to setup and manage dedicated OpenEBS setup havebeen removed. Starting with v0.4.0, only hyperconverged with Kubernetes is supported.
Notes for Contributors
- OpenEBS user documentation is being moved into openebs/openebs/documentation
- OpenEBS developer documentation is being added to openebs/openebs/contribute
- The deployment and e2e functionality will continue to be located in openebs/k8s and openebs/e2e respectively.
- openebs/maya will act as a single repository for hosting differnt OpenEBS Storage Control plane (orchestration) components.
- New /metrics handlers are getting added to OpenEBS components to allow integration into tools like Prometheus.
- openebs/maya/cmd/maya-agent which will be deployed as a deamon-set running along-side kubelet is being developed. maya-agent will augument the kubelet with storage management functionality.
v0.3
It is simple to use OpenEBS. Try it!!
New in v0.3
- OpenEBS hyper-converged with Kubernetes Minion Nodes.
- Enable OpenEBS via the openebs-operator.yaml
- OpenEBS Volumes created using the Kuberentes Concepts - Services, Deployments, Pods and PVs
- Supports creation of OpenEBS volumes using Dynamic Provisioner (using the storage-incubator/provisionercontroller)
- Storage functionality is delivered as container images on DockerHub
- openebs/jiva:0.3-RC2
- Storage Orchestration/Management functionality is delivered as container images on DockerHub
- openebs/m-apiserver:0.3-RC3
- openebs/openebs-k8s-provisioner:0.3-RC2
- maya cli is packaged with m-apiserver.
- Storage Orchestration/Management functionality is also available as binaries, under the respective repositories.
- Ansible Playbooks are used (e2e/ansible) for automating the installation/configuration and also to run persistent workloads on K8s and OpenEBS.
v0.2
New in v0.2
Integrated into Kubernetes
- OpenEBS FlexVolume Driver
- Dynamically Provision OpenEBS Volumes
Easy to Install Vagrant Boxes
- Kubernetes 1.5.5 vagrant box
- OpenEBS 0.2 vagrant box
Maya API Server
Provides new AWS EBS-like API for provisioning Block Storage
- Hyper Converged with Nomad Scheduler
- Specify the VSM Network and VSM Storage via Configuration files
Storage Tests Framework
- openebs/tests-vdbench
- openebs/tests-fio
Changes since v0.1
Maya CLI
- New CLI options ( maya vsm-stop, maya vsm-stats )
VSM / Jiva
- Backend Containers auto registration with Frontend Containers
- Backup/Restore Data from Amazon S3
- Node Failure Resiliency Fixes