Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Semicolons used in descriptions break CSV import process - remediation required #3829

Closed
DawidKrysiak opened this issue Apr 22, 2024 · 4 comments · Fixed by #3830
Closed
Assignees
Labels
bug severity/medium Results in some unexpected or undesired behavior.

Comments

@DawidKrysiak
Copy link

New feature motivation

Semicolons are used in sentences as per below example:

"remediation": 
{ 
"desc": "Ensure that S3 buckets have Logging enabled. CloudTrail data events can be used in place of S3 bucket logging. If that is the case; this finding can be considered a false positive.",
...

This is not a problem if JSON format is in use, but unfortunately, it also exists in the CSV report

;;"Ensure that S3 buckets have Logging enabled. CloudTrail data events can be used in place of S3 bucket logging. If that is the case; this finding can be considered a false positive.";

This means, in the import process (to a spreadsheet of some kind, or - in my case - to AWS Athena) those additional semicolons are interpreted as field dividers, breaking formatting/data.

Solution Proposed

I don't have a universal solution that would suit everyone.
The first that comes to mind is, to encode semicolons to Unicode or HTML entity. But that might disrupt a lot of people's workflows.

Describe alternatives you've considered

Second approach: allow for a custom character to be defined as a field divider in CSV report. For example, I don't see % in CSV reports - if I could pass a flag to prowler to use % rather than ; in CSV, I can modify my import functions / Athena schemas easily.

Additional context

No response

@DawidKrysiak DawidKrysiak added feature-request New feature request for Prowler. status/needs-triage Issue pending triage labels Apr 22, 2024
@sergargar sergargar self-assigned this Apr 22, 2024
@sergargar sergargar added bug severity/medium Results in some unexpected or undesired behavior. and removed feature-request New feature request for Prowler. status/needs-triage Issue pending triage labels Apr 22, 2024
@sergargar sergargar linked a pull request Apr 22, 2024 that will close this issue
@rieck-srlabs
Copy link
Contributor

The CSV file format handles semicolons in strings by wrapping the entire string in double quotes. This is what currently happens according to your sample:

;;"Ensure that S3 buckets have Logging enabled. CloudTrail data events can be used in place of S3 bucket logging. If that is the case; this finding can be considered a false positive.";

Looks like a problem with the import process in your spreadsheet application, not with Prowler.

@jfagoagas
Copy link
Member

Hi @DawidKrysiak as @rieck-srlabs said, I think this issue is related with your process since the Prowler CSV is valid using the ; as delimiter.

@DawidKrysiak
Copy link
Author

@jfagoagas thank you. Will try to fix the code. Let's close this issue for now. Thank you for your help!

@jfagoagas
Copy link
Member

jfagoagas commented Apr 23, 2024

@DawidKrysiak I don't know what are you using but from this AWS documentation https://docs.aws.amazon.com/athena/latest/ug/csv-serde.html using the OpenCSVSerDe you can configure

WITH SERDEPROPERTIES ("separatorChar" = ",", "quoteChar" = "`", "escapeChar" = "\\" )

to indicate separators, quoting and escaping characters.

I hope this will help you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug severity/medium Results in some unexpected or undesired behavior.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants