Flyte v0.15.1 milestone release

Flyte v0.15.1

Platform

• Updated helm chart to include missing ingress paths

Flyte v0.15.0 milestone release

Flyte v0.15.0

Platform

• Enum type support throughout the system
• Stabilization improvements for conditionals
• Support AWS Secrets Manager as a source for secrets injector
• Support Map tasks over Pod tasks
• Support max parallelism to limit how many nodes can be allowed to run in parallel.
• Add Athena flytekit plugin and examples
• Add BigQuery plugin

Flytekit

• Support Schema of Dataclasses
• Support node resource overrides

Please see the flytekit release for the full list and more details.

flytectl

• flytectl sandbox start to start a sandbox cluster locally.
• flytectl get workflow .... -o dot to visualize a workflow graph locally.
• Add Bash completion support

Update flyte docker images

v0.14.2

Update Flyte Components (#1206)

Add hack for cgroup v2 to sandbox entrypoints

Admin secrets (#1087)

* add stringdata

Signed-off-by: wild-endeavor <[email protected]>

* extra dash

Signed-off-by: wild-endeavor <[email protected]>

* wrong file

Signed-off-by: wild-endeavor <[email protected]>

* make helm

Signed-off-by: wild-endeavor <[email protected]>

Co-authored-by: Haytham Abuelfutuh <[email protected]>

Flyte v0.14.0 milestone release

Flyte v0.14.0

Platform

• Update to the Flyte Compiler, to supported nested branches and more. Better
  regression tests
• support for iam roles and k8s serviceaccounts
• customizable pod specs for pod tasks (add labels and annotations)
• copilot improvements
• Bug-fixes, visibility improvements
• support for non Cloud provider emailers - like Sendgrid
• performance improvement for dynamic workflows

Flyteconsole

• Bug fixes
• More updates coming soon

Flytekit

• Support for nested conditionals
• DoltTable and Dolt plugins and integration
• Better context management (ready for more work)
• Support for pre-built container plugins in flytekit. this makes it possible
  to library plugins and users do not need to build containers
• More control plane class features
• See full release notes
  Coming soon:
  • Great Expectations integration
  • More use case driven examples in flytesnacks

flytectl

• flytectl is ready for BETA. check it out - https://docs.flyte.org/projects/flytectl/en/latest/

Please see the flytekit release for the full list and more details.

Wait for deployments to be created using rollout before checking for availability

Fixes a race condition with checking availability status on flyte pods

Wait for deployments to be created using rollout before checking for availability

Signed-off-by: Prafulla Mahindrakar [email protected]

Flyte v0.13.0 milestone release

#979

Platform

• Oauth2 support with SingleSignOn and configuration examples for popular IDP's now available in Flyte.
  Please see the updated information and description of the feature, and the setup information
  Attention: If using Auth already - this is a BREAKING change. refer to the migration guide to update configuration to ensure Admin continues to work. (No migration needed if auth is not turned on.)

• Backend improvements to support dynamic workflow visualization (in future releases).
• Lot of features added to flytectl .
• Documentation site overhaul and redesign (more in progress)

Flytekit

The first two features should be considered beta and subject to change

• First cut of the control plane classes to replace the old Sdk... classes. These classes provide programmatic access to a Flyte backend.
• New paradigm for flytekit-only plugins
• SqlAlchemy/Dolt plugins.

Please see the flytekit release for the full list and more details.

Update sandbox deployment to ensure my-s3-bucket is created before registration

Add init container that will pre-create buckets in minio (#905)

* Add init container that will pre-create buckets in minio

Signed-off-by: Jeev B <[email protected]>

* move minio tag override to overlay

Signed-off-by: Jeev B <[email protected]>

Milestone 0.12.0

General

• Added CoPilot configuration to sandbox manifest
• Updated and streamlined documentation navigation and themes. Better content organization coming soon!

Performance

• Improved execution performance including:
  • Reduced cache lookups
  • Improved GetWorkflowExecution performance
  • Capped max number of nodes in each propeller round
  • Misc. propeller performance tweaks
• TaskTemplate offloading

Housekeeping

• Migrated Datacatalog protobuf definitions to flyteidl [thanks @tnsetting]
• Upgraded stow version used in flytestdlib
• Moved off lyft kubernetes forks and onto official kubernetes library dependencies
• Revamped pod tasks to use official kubernetes python client library for defining PodSpecs

Events

• Richer event metadata for task executions
• Better merging of custom info across task events

Bug fixes

• Resolved non-backwards protobuf role changes that prevented launching single task executions [thanks @kanterov]
• Better handling of large workflows and errors in flytepropeller

Flytekit (Python)

• Access to secrets
• Bug fixes around the 0.16 release.
  • Use original FlyteFile/FlyteDirectory
  • Fix serialization of pod specs in pod plugin [thanks @jeevb]
  • Accept auth role arg in single task execution
  • Fixed task resolver in map task
  • Requests and limits added to ContainerTask [thanks @migueltol22]

Update sandbox images to allow overriding api port

Allow K3s K8s Api Port to be overriden (#812)

Signed-off-by: Haytham Abuelfutuh <[email protected]>