-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Question][Standalone] How to add constraints
when creating tables with delta-standalone?
#2985
Comments
As I have responded in the other issue #2984 standalone is not going to get any new functionality. We want to add all these new functionality in the Delta Kernel project. Can you tell us a bit more about how you are using Standalone and why do you want to add constraints using standalone? Are you building a connector for an engine or are you just building your own application to directly operate on delta tables? |
Thanks for update @tdas. We are using delta-standalone currently to create delta tables in a non-spark way since we wanted to keep delta table creation times to minimal and delta-standalone offered best way since it just operates on writing metadata on parquet files. Let me catch up on delta kernel offerings and see if that could still be a fit for why we were using standalone in 1st place. |
@JaySardhara Constraints are tricky to support in Standalone. In Kernel, we took into this account as part of the design for writes into the Delta table. The current constraints can contain any Apache Spark supported expression and it is not easy to support all these expressions in a non-Spark engine. In Kernel, we are starting with a few handful of expressions and expanding the supported expression as we get to know the usage of constraints. Let us know what kind of expressions are you planning to use within constraints. See the design decision here. |
Question
Which Delta project/connector is this regarding?
Overview
How can we handle adding
constraints
when tables are created using delta-standalone?For example: In delta-spark we add constraints shown below.
spark.sql("Alter table delta.my_table7 add constraint age_less_than check (age < 100)")
Motivation
Delta Standalone can be used to create delta tables and adding constraints to tables is important aspect of it.
Further details
If this is not supported, can create a Feature request.
Willingness to contribute
The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?
The text was updated successfully, but these errors were encountered: