[Bug]: Output issue

[Imiler]

Steps to Reproduce

I have found issue that has been close off before (#614) and it was mentioning using output command as following in order to remove coloring from cli output:
Prowler aws -M text html csv -D myS3bucket

This results to me as issue with "text" argument and just stops scan

Expected behavior

Output in CLI should be only black/white, no colors

Actual Result with Screenshots or Logs

2024-01-31T11:21:51.853406066Z prowler aws: error: argument -M/--output-modes: invalid choice: 'text' (choose from 'csv', 'json', 'json-asff', 'html', 'json-ocsf')

How did you install Prowler?

Cloning the repository from github.com (git clone)

Environment Resource

Kubernetes CronJob on Rancher.
This is working perfectly just output issue to cli

OS used

AL2

Prowler version

Prowler 3.11.3

Pip version

pip 23.3.2

Context

I need advice on proper command argument to remove green/red coloring from CLI output. This is needed for allow Loki/Grafana to pick up logs and display them in dashboard but color coding is making a mess of output in grafana.

[sergargar]

Hi @Imiler, thank you for reaching us out.
The --only-logs flag may solve your issue since this would print only Prowler logs by the stdout.
Let me know if that works for you.

[sergargar]

The text argument does not exist in Prowler.

[Imiler]

Thank you,

Unfortunately this is not what we needed. This completely removes anything showing in CLI. They way loki/grafana gets data is that it searched CLI output for results (FAIL, PASS) and from those keywords in query it creates dashboard.
This way we have nothing in logs at all, just CSV and HTML exports in file.

Any way to keep same output as it is with normal "Prowler aws" command but for it to be without color.?

This is also link to grafana dashboard, you can see in json on line 210 where it looks for specific word in logs, in this case "EC2". but since in normal output EC2 and results of findings are color coded this just gets out as mess.

[sergargar]

That's a good point @Imiler, we will include a flag to remove the colors in the cli output. Any naming that comes to your mind? Thanks!

[Imiler]

Hey, thank you for fast response and assistance.

For naming, suggestion i have is --no-color and maybe (-N if its not used already) as its pretty self explanatory.

[sergargar]

@Imiler out of curiosity, why aren't the output files (csv, json...) useful to grafana/loki?

[Imiler]

It could be useful but they require additional steps to make those files available for grafana to pick them up as in our case we store those to S3 bucket and from what i know Grafana cannot query s3 to get those files. We would need to ingest those to Database or make additional resources to provide those files to Grafana.

In this situation we have dashboard template: [https://grafana.com/grafana/dashboards/19722-aws-prowler-dashboard/] which fetches information from cli output.

But for some reason when running prowler and getting cli output in color and then when Grafana queries it gets messed up. this is example from dashboard: │ ec2 │ \u001B[31mFAIL (97)\u001B[0m │ \u001B[91m0\u001B[0m │ \u001B[31m18\u001B[0m │ \u001B[33m77\u001B[0m │ \u001B[34m2\u001B[0m │"

[toniblyx]

Why not to get it from the logs? The integration pattern you are following is probably not the best in performance and reliability. Our findings results source of truth are our output files and logs.

[Imiler]

I will try to do it that way as well but for now if possible to get just no color output to help with this case.

This dashboard template i am using seems to be focused on catching that first output but could be more underneath to sort out but im unable to even see what results i get from cli since its all messed up