You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When running with Databricks (v14.0) on Azure and reading data from S3, it fails with the following error message:
transactions_transformation: FAILED - transformation failed to complete: spark submit job for transformation failed to run: (name: transactions_transformation variant:awesome_ride) the '532227170932654' job failed: java.lang.ClassCastException: shaded.databricks.org.apache.hadoop.fs.s3a.S3AStorageStatistics cannot be cast to org.apache.hadoop.fs.s3a.S3AStorageStatistics
But it works fine when I downgrade the cluster to v11.3.
Actual Behavior
The expected behavior is that we are able to run the job successfully end to end.
Steps To Reproduce
Create a Databricks Cluster with version 14.0 on Azure
Register Databricks with S3 provider
Run a transformation
What mode are you running Featureform in?
Hosted
What version of Python are you running?
3.9
Featureform Python Package Version
1.10.0
Featureform Helm Chart Version
No response
Kubernetes Version
No response
Relevant log output
transactions_transformation: FAILED - transformation failed to complete: spark submit job for transformation failed to run: (name: transactions_transformation variant:awesome_ride) the '532227170932654' job failed: java.lang.ClassCastException: shaded.databricks.org.apache.hadoop.fs.s3a.S3AStorageStatistics cannot be cast to org.apache.hadoop.fs.s3a.S3AStorageStatistics
The text was updated successfully, but these errors were encountered:
Expected Behavior
When running with Databricks (v14.0) on Azure and reading data from S3, it fails with the following error message:
But it works fine when I downgrade the cluster to
v11.3
.Actual Behavior
The expected behavior is that we are able to run the job successfully end to end.
Steps To Reproduce
What mode are you running Featureform in?
Hosted
What version of Python are you running?
3.9
Featureform Python Package Version
1.10.0
Featureform Helm Chart Version
No response
Kubernetes Version
No response
Relevant log output
The text was updated successfully, but these errors were encountered: