New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Semicolons used in descriptions break CSV import process - remediation required #3829
Comments
The CSV file format handles semicolons in strings by wrapping the entire string in double quotes. This is what currently happens according to your sample:
Looks like a problem with the import process in your spreadsheet application, not with Prowler. |
Hi @DawidKrysiak as @rieck-srlabs said, I think this issue is related with your process since the Prowler CSV is valid using the |
@jfagoagas thank you. Will try to fix the code. Let's close this issue for now. Thank you for your help! |
@DawidKrysiak I don't know what are you using but from this AWS documentation https://docs.aws.amazon.com/athena/latest/ug/csv-serde.html using the OpenCSVSerDe you can configure
to indicate separators, quoting and escaping characters. I hope this will help you. |
New feature motivation
Semicolons are used in sentences as per below example:
This is not a problem if JSON format is in use, but unfortunately, it also exists in the CSV report
This means, in the import process (to a spreadsheet of some kind, or - in my case - to AWS Athena) those additional semicolons are interpreted as field dividers, breaking formatting/data.
Solution Proposed
I don't have a universal solution that would suit everyone.
The first that comes to mind is, to encode semicolons to Unicode or HTML entity. But that might disrupt a lot of people's workflows.
Describe alternatives you've considered
Second approach: allow for a custom character to be defined as a field divider in CSV report. For example, I don't see
%
in CSV reports - if I could pass a flag to prowler to use%
rather than;
in CSV, I can modify my import functions / Athena schemas easily.Additional context
No response
The text was updated successfully, but these errors were encountered: