-
Notifications
You must be signed in to change notification settings - Fork 510
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ParquetTypeSortedBucketIO writes and reads don't work with CaseMapper #5017
Comments
Cause in in the If magnolify's As a 'dirty' workaround, users can define alias function for the key(s), respecting the storage casing, eg: case class Event(userName: String, event: String, timestamp: Int) {
def user_name: String = userName
}
implicit val parquetType: ParquetType[Event] = ParquetType[Event](SnakeCaseMapper) Clean fix would imply to give the bucket metadata a way to transform the key field into the proper chained calls |
key getter does not work when a CaseMapper is required to translate field names:
repro: https://github.com/spotify/scio/compare/parquet-type-caseformat?expand=1
The text was updated successfully, but these errors were encountered: