Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(UI): New progress bar #3403

Open
wants to merge 58 commits into
base: v3
Choose a base branch
from

Conversation

Fennerr
Copy link
Contributor

@Fennerr Fennerr commented Feb 15, 2024

Context
Improved UI for prowler 4.0

Currently only supports the AWS provider.

Description
Switched from using alive_progress bar to using Rich to display the UI

I wanted to implement the new ui without having to update the service class's or checks code.

I was able to do it for the service classes, by modifying the threading_call to initialize the progress bar with X number of tasks (length of the iterator passed to the function), and increment the progress bar as each thread completes it's task.

There is also a progress_decorator to be used where threading_call is not appropriate - like for global services (see list_distributions in the CloudFront service)

Doing checks was not as straightforward, as the logic for the way checks are handled is coded into the execute function, so I couldnt do anything at the base class level. For the most part, checks are really quick, and dont need a progress bar. One place where it could be appropriate, where it does take a long time to execute, was the secrets check for lambda functions (other secrets check are also likely to take a decent amount of time and could benefit from a progress bar)

In these cases, inside the execute function, a call to self.start_task needs to be made with the display message and the number of tasks (self.start_task("Processing functions...", len(functions)) for checking secrets in lambda code). Then after completing an item in the list of items to check, a call to self.increment_task_progress() needs to be made. At the end of the check, just after returning the findings, I made a call to self.update_title_with_findings(findings) so that the UI shows X/Y failed the test. I like how it looks, but it requires some changes to be made to the checks which is what I was trying to avoid - but couldn't find a way around this.

I have also implemented an execution_manager. I thought I had previously pushed a branch with this change, but it doesnt seem to be the case. The execution_manager was made to help be more deliberate about which clients are intialized, then run all the checks that use that client, and then release the client for it to be garage collected.

It determines what clients are going to be imported by all the checks that are going to run (by using /check_to_client_mapper.py. It then starts running through the checks, and will handle initializing any clients needed by the check. It will then prioritize the next check based off of what clients are already loaded, and release the clients once they are no longer needed.

I have also implemented the --only-logs functionality - in which case the live display is not intialized.

This pull request is a WIP, but wanted to provide it so longer so it's easier to see the diff between main and what I have done.

I still need to complete changing the functions inside the service init's to make use of threading_call or progress_decorator, and any other feedback provided.

n4ch04 and others added 30 commits November 27, 2023 13:58
Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: r3drun3 <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: John Mastron <[email protected]>
Co-authored-by: John Mastron <[email protected]>
Co-authored-by: Sergio Garcia <[email protected]>
Co-authored-by: Sergio Garcia <[email protected]>
Co-authored-by: sergargar <[email protected]>
Co-authored-by: Pepe Fagoaga <[email protected]>
Co-authored-by: github-actions <[email protected]>
Co-authored-by: simone ragonesi <[email protected]>
Co-authored-by: Johnny Lu <[email protected]>
Co-authored-by: Vajrala Venkateswarlu <[email protected]>
Co-authored-by: Ignacio Dominguez <[email protected]>
removed function param - it is not needed
fixed blank space + removed the if statement in __init__ which I should have previously removed
Fennerr and others added 21 commits December 20, 2023 12:10
Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: r3drun3 <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: John Mastron <[email protected]>
Co-authored-by: John Mastron <[email protected]>
Co-authored-by: Sergio Garcia <[email protected]>
Co-authored-by: Sergio Garcia <[email protected]>
Co-authored-by: sergargar <[email protected]>
Co-authored-by: Pepe Fagoaga <[email protected]>
Co-authored-by: github-actions <[email protected]>
Co-authored-by: simone ragonesi <[email protected]>
Co-authored-by: Johnny Lu <[email protected]>
Co-authored-by: Vajrala Venkateswarlu <[email protected]>
Co-authored-by: Ignacio Dominguez <[email protected]>
@Fennerr Fennerr requested a review from a team as a code owner February 15, 2024 14:08
@sergargar sergargar changed the title Fennerr new progress bar feat(UI): New progress bar and ExecutionManager Feb 15, 2024
@sergargar sergargar changed the title feat(UI): New progress bar and ExecutionManager feat(UI): New progress bar Feb 15, 2024
@jfagoagas jfagoagas added no-merge Please, DO NOT MERGE this PR. testing-pending labels Feb 15, 2024
@sergargar sergargar added backport-v3 Pending to port to Prowler v3 branch and removed backport-v3 Pending to port to Prowler v3 branch labels Mar 14, 2024
@jfagoagas jfagoagas changed the base branch from master to v3 April 15, 2024 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no-merge Please, DO NOT MERGE this PR. testing-pending
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants