Add new parser - Rapplex #10202

AlperenY-cs · 2024-05-14T14:25:23Z

Rapplex - Web Application Security Scanner

For more information, Rapplex

dryrunsecurity · 2024-05-14T14:26:08Z

Hi there 👋, @DryRunSecurity here, below is a summary of our analysis and findings.

DryRun Security	Status	Findings
Configured Codepaths Analyzer	✅	0 findings
Sensitive Files Analyzer	✅	0 findings
AppSec Analyzer	✅	0 findings
Authn/Authz Analyzer	✅	0 findings
Secrets Analyzer	✅	0 findings

Note

🟢 Risk threshold not exceeded.

Change Summary (click to expand)

The following is a summary of changes in this pull request made by me, your security buddy 🤖. Note that this summary is auto-generated and not meant to be a definitive list of security issues but rather a helpful summary from a security perspective.

Summary:

The provided code changes cover various aspects of the DefectDojo application, including the addition of a new documentation page for the Rapplex web application security scanner integration, the implementation of a parser for Rapplex scan results, and the update of the application's settings.

The documentation change adds a new page that provides information and guidance on how to use the Rapplex scanner with the DefectDojo platform. This change is not directly related to the application's security, but it helps to improve the overall user experience and integration capabilities of the platform.

The implementation of the Rapplex parser is a significant security-focused change, as it allows the DefectDojo application to import and process findings from the Rapplex security scanner. The parser extracts relevant information, such as vulnerability details, severity levels, and CWE IDs, which can be used for security analysis and reporting within the DefectDojo platform.

The settings update includes several security-related changes, such as the configuration of deduplication algorithms and hashcode calculations for different types of scans, the addition of SAML authentication and remote user authentication options, and the management of file upload types. These changes demonstrate a focus on improving the security and reliability of the DefectDojo application.

Additionally, the provided security scan reports highlight the types of vulnerabilities that the Rapplex scanner can identify, including critical SQL Injection, high-severity Cross-Site Scripting (XSS), and various informational and low-severity issues. The inclusion of these sample reports, along with the unit tests for the Rapplex parser, further strengthens the overall security posture of the DefectDojo application.

Files Changed:

docs/content/en/integrations/parsers/file/rapplex.md: This file adds a new documentation page for the Rapplex web application security scanner integration.
dojo/tools/rapplex/parser.py: This file implements a parser for the Rapplex security scanner, which is responsible for importing and processing Rapplex JSON reports and converting them into Finding objects for the DefectDojo platform.
unittests/scans/rapplex/rapplex_one_vul.json, unittests/scans/rapplex/rapplex_many_vul.json, and unittests/scans/rapplex/rapplex_zero_vul.json: These files contain sample security scan reports from the Rapplex scanner, which are used for testing and demonstration purposes.
dojo/settings/settings.dist.py: This file updates the settings for the DefectDojo application, including the configuration of deduplication algorithms, SAML authentication, and file upload types.
unittests/tools/test_rapplex_parser.py: This file contains a unit test suite that verifies the functionality of the RapplexParser class, ensuring the parser's robustness and reliability.

Powered by DryRun Security

mtesauro · 2024-05-14T14:30:05Z

@AlperenY-cs I just kicked off the tests - they'll need to be green before we'll merge this PR - the Ruff linter has issues you can address now or when the rest of the tests have run.

AlperenY-cs · 2024-05-14T18:37:17Z

@AlperenY-cs I just kicked off the tests - they'll need to be green before we'll merge this PR - the Ruff linter has issues you can address now or when the rest of the tests have run.

Thanks. I solved the ruff linter's problems. #fyi

mtesauro

Approved

Maffooch · 2024-05-20T15:12:11Z

dojo/tools/rapplex/parser.py

+ for reference in issue_definition.get("References", []):
+ ref_title = reference.get("Title", "")
+ ref_link = reference.get("Link", "")
+ reference_texts.append(f"{ref_title}\n{ref_link}") # ref_title and ref_link combined to references section


Suggested change

reference_texts.append(f"{ref_title}\n{ref_link}") # ref_title and ref_link combined to references section

reference_texts.append(f"[{ref_title}]({ref_link})") # ref_title and ref_link combined to references section

Markdown links would be preferable imo, but feel free to disregard if you disagree

I agree with you, I made it suitable for markdown format.

Maffooch · 2024-05-20T15:15:32Z

dojo/tools/rapplex/parser.py

+ desc_sum = issue_definition.get("Sections", {}).get("Summary", "")
+
+ if (len(desc_rem) > 0):
+ desc_text = f"\n{desc_sum}\nRemediation:\n{desc_rem}" # summary and remediation combined to create description


DefectDojo offers a field called mitigation that would be better for the remediation section of the report

Remediation area assigned to mitigation.

Maffooch · 2024-05-20T15:20:01Z

dojo/tools/rapplex/parser.py

+ reference_texts = []
+ for reference in issue_definition.get("References", []):
+ ref_title = reference.get("Title", "")
+ ref_link = reference.get("Link", "")
+ reference_texts.append(f"{ref_title}\n{ref_link}") # ref_title and ref_link combined to references section


I noticed that the Definitions section is "above" the issues scope. It would be better to collect things like references, summary, remediation, etc. here rather than doing it over and over again in the issues loop. This is a small efficiency, but would be noticeable in larger reporter

I made this change using html2text.

Maffooch · 2024-05-20T15:20:43Z

dojo/tools/rapplex/parser.py

+ references=reference_array,
+ unique_id_from_tool=vuln_id,
+ )
+
+ finding.active = True


Suggested change

references=reference_array,

unique_id_from_tool=vuln_id,

)

finding.active = True

references=reference_array,

unique_id_from_tool=vuln_id,

active=True,

)

The active value is now defined in the finding object.

Maffooch · 2024-05-20T15:23:27Z

dojo/tools/rapplex/parser.py

+ req = issue.get("HttpRequest", "")
+ res = issue.get("HttpResponse", "")
+ vIndex = issue.get("vIndex", "")
+ vuln_id = f"{scan_id}_{vIndex}" # scanId and vIndex combined to create unique_id_from_tool


In most tools, the scan ID is used to identify a single instance of a scan. Is this the case here as well? If so, deduplication will never be able to identify a match from scan to scan, as the unique ID from tool is actually TOO unique

Yes, in Rapplex, the scan ID is determined specifically for each scan. unique_id_from_tool is not used now.

dojo/settings/settings.dist.py

…in parser.

dojo/tools/rapplex/parser.py

albay

Thanks for the code changes.

mtesauro · 2024-05-23T15:47:05Z

@albay Closing and re-opening as the tests seem stuck for some reason

Added Rapplex parser files

97a973e

github-actions bot added settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR docs unittests parser labels May 14, 2024

AlperenY-cs added 2 commits May 14, 2024 17:56

Ruff checks were made. Warnings fixed.

e3c488d

Ruff checks were made on unittest parser. Warnings fixed.

dd04163

Changed file loading process to use json.load instead of json.loads

bd754d6

mtesauro approved these changes May 16, 2024

View reviewed changes

devGregA approved these changes May 17, 2024

View reviewed changes

Maffooch requested changes May 20, 2024

View reviewed changes

Dedupe_algo changed to hash_code. Performance improvements and fixes …

e6ab56f

…in parser.

AlperenY-cs requested a review from Maffooch May 22, 2024 11:26

albay reviewed May 22, 2024

View reviewed changes

dojo/tools/rapplex/parser.py Outdated Show resolved Hide resolved

Corrections were made in accordance with DRY principles.

9d04ce8

AlperenY-cs requested a review from albay May 22, 2024 11:51

albay approved these changes May 22, 2024

View reviewed changes

html2text import fix

1889619

mtesauro closed this May 23, 2024

mtesauro reopened this May 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new parser - Rapplex #10202

Add new parser - Rapplex #10202

AlperenY-cs commented May 14, 2024

dryrunsecurity bot commented May 14, 2024 •

edited

mtesauro commented May 14, 2024

AlperenY-cs commented May 14, 2024

mtesauro left a comment

Maffooch May 20, 2024

AlperenY-cs May 22, 2024

Maffooch May 20, 2024

AlperenY-cs May 22, 2024

Maffooch May 20, 2024

AlperenY-cs May 22, 2024

Maffooch May 20, 2024

AlperenY-cs May 22, 2024

Maffooch May 20, 2024

AlperenY-cs May 22, 2024

albay left a comment

mtesauro commented May 23, 2024

	reference_texts.append(f"{ref_title}\n{ref_link}") # ref_title and ref_link combined to references section
	reference_texts.append(f"[{ref_title}]({ref_link})") # ref_title and ref_link combined to references section

Add new parser - Rapplex #10202

Are you sure you want to change the base?

Add new parser - Rapplex #10202

Conversation

AlperenY-cs commented May 14, 2024

dryrunsecurity bot commented May 14, 2024 • edited

mtesauro commented May 14, 2024

AlperenY-cs commented May 14, 2024

mtesauro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

albay left a comment

Choose a reason for hiding this comment

mtesauro commented May 23, 2024

dryrunsecurity bot commented May 14, 2024 •

edited