Trail of Bits Testing Handbook

The Trail of Bits Testing Handbook is a resource for developers and security professionals on configuring, optimizing, and automating many static and dynamic analysis tools we use at Trail of Bits.

Preview Testing Handbook: https://appsec.guide 🌐

Why is this needed? ✨

• 📃 The documentation for configuring and optimizing existing tools is often not developer-friendly, as it is often meant for security professionals. This is especially the case when it comes to fuzzing utilities. This can lead to frustration and poor adoption of security tools that should be straightforward to configure.
• ⚙️ Even if the tool is easy to configure locally, it can be difficult to configure them in CI/CD pipelines. Often, security tools are set up by following online documentation, but their configuration is rarely optimized. This can lead to a noisy tool that is more difficult to maintain than worth.
• 🧠 We aim to make it as easy as possible to set up security tools effectively. In doing so, we also hope to demystify static and dynamic analysis techniques such as fuzzing and taint analysis.

Chapters

✅ Released

Topic	Announcing Blog Post	Year
Semgrep	Announcing the Trail of Bits Testing Handbook	2023
CodeQL	Say hello to the next chapter of the Testing Handbook!	2023
Fuzzing	Master fuzzing with our new Testing Handbook chapter	2024

🎥 Webinars

Topic	Link
Introduction to Semgrep	https://www.youtube.com/watch?v=yKQlTbVlf0Q
Introduction to CodeQL: Examples, Tools and CI Integration	https://www.youtube.com/watch?v=rQRlnUQPXDw

🚧 Under construction

• Burp Suite Professional
• Formal verification and Tamarin
• Rust

How to contribute

If you would like to contribute to the Testing Handbook, here are some guidelines to help you get started:

1. Add a New Tool: If you want to cover a new tool in the Testing Handbook, propose a topic in GitHub Issues. Afterward, you can work on a new pull request.
2. Improve Existing Chapters: If you have an idea to make a specific chapter better, you can add a GitHub issue.
3. Pick Up Small Tasks: If you don't have much time but still want to contribute, you can pick up any small task from the GitHub issues list.
4. Report Issues: If you find a small technical issue or a typo, create a new GitHub issue and/or fix it in the new pull request.

Quick setup for convenient development

1. Install Hugo in your system

   brew install hugo

2. Clone the repo

   git clone --recurse-submodules https://github.com/trailofbits/testing-handbook.git

3. Create a new branch or select a branch you want to work on

   cd testing-handbook
   # then
   git checkout -b name-of-your-new-branch
   # or
   git checkout name-of-existing-branch
   

4. Run the Hugo server with drafts turned on (-D) from the project's root directory. Your browser will be automatically refreshed with changes whenever you save a file.

   hugo server -D

5. Add a new tool as "doc", and run the following from the project's root directory.

   hugo new docs/<name of tool>

   Note: This project uses the same hugo template as zkdocs. The template refers to each new page as a "doc," as opposed to a post. This is why you'd want to type hugo new docs/<name of tool> and not post/my-new-post.

6. Edit, add, and create pull requests to merge your changes into main.

7. ❗Keep in mind that when you merge your PR into main, the content goes live in https://appsec.guide. Our current policy forces at least one review before merging.

8. For updates to the home page, edit content/_index.md

Guidelines

• The format should be consistent between each "doc." When adding a new doc (i.e., when adding a new tool), follow the template in content/docs/template.md. Send a PR for this file with suggested changes as needed.

• Create a new branch with your changes, and create a PR to merge into main when you are done.

• The GitHub workflow in this repository verifies the correctness of Markdown files through three checks:

  1. Markdown Link Check: This step extracts links from Markdown files and verifies if they are valid and accessible. It uses the github-action-markdown-link-check action.
  2. Markdown Linter: This step ensures that Markdown files adhere to the desired style and formatting rules. It uses a custom configuration file (.github/workflows/.markdownlint.jsonc) and the markdownlint-cli2-action action. Use the markdownlint extension with Visual Studio Code for better user experience while working on the Testing Handbook.
  3. Spellcheck: This step checks the spelling in Markdown files (built on top of retext and remark). Uses the tbroadley/spellchecker-cli-action action.

• Familiarize yourself with the Hugo Book theme as it has a couple of nice features (buttons, etc.)

• Reach out in #testing-handbook Empire Hacking Slack if you have any questions.

Editing

Writing Guidelines

• The term "Testing Handbook" should be capitalized any time it appears on the website (whether in a header/subheader or running text), since it is the title of a document. But if you'd like to avoid the capitalization because it looks strange, you can substitute "Testing Handbook" for "this handbook" (since it's clear enough what the title of the handbook is).

Workflow: From Google Docs

1. Make your document viewable via a link share.
2. Create a Google account or use your private one (If you use this method, your document should be considered public but unpublished).
3. Install Docs to Markdown. This addon works better than the pandoc.
4. Open the document and make a copy.
5. Open the copy and run the Addon.
6. Export the markdown and apply fixes:
   • Search for occurrences of <code> or <strong> or any other html tags
   • Replace HTML tables with markdown ones (https://jmalarcon.github.io/markdowntables/)
   • If you split your document, fix internal links.
   • Add missing images.
   • Fix <, …, “, ’
   • Adjust markdown captions ## -> #
   • Verify missing formatting in PRO TIPs
   • . at the end of fig captions?
   • Note that index bundles do not use the "slug"

Custom environments

{{< customFigure "Caption" >}}
{{< /customFigure >}}

{{< resourceFigure "cov1.png" >}}
{{< /resourceFigure >}}

{{< hint info >}}
{{< /hint >}}