Centralized build configuration for LibAFL

[Evian-Zhang]

First of all, thank you for building such extraordinary Rust crates for fuzzing!

For background, researchers may want to modify QEMU code when proposing novel ideas for binary-only fuzzing, and this can be easily done by modifying build_qemu_support.sh file when using AFL++. However, when using LibAFL, one should look into libafl_qemu/libafl_qemu_build/src/build.rs for environment variables to change the url of QEMU code to their self-modified version (Moreover, it is more complicated for editor's language server to observe this change, which is a bad experience to use vscode). I found these ad-hoc things (not only for QEMU, but also other configurations for LLVM mode/frida mode, etc.) hard to follow, and LibAFL has so many configurations across the whole source code.

I am good at Rust and willing to help writing a centralized build configuration for LibAFL, but I have some questions about this after reading the source code of the build system in LibAFL.

Brief

In short, we may have a libafl.toml configuration file to centralize these environment variables which control the behavior of what and how to build LibAFL, its content may look like this

[qemu_mode]
qemu_url = "my_url"
qemu_version = "xxxx"
emulation_mode = "usermode"

[frida_mode]
# ....

[llvm_mode]
# ....

How to implement

We may create a new member crate libafl_conf, which provide parsing of such configuration files lazily (i.e. In one build script, the configuration file will be parsed only once). And in each position where one configuration value is needed, we access such configuration and get the corresponding fields.

Advantages

• Environment variables are not typed, and if the needed value is not string, parsing is done ad-hoc, which may increase the possibility of wrong-parsing. By centralizing the configuration, the parsing things can also be centralized, and each build scripts can access the typed value directly.
• There are many configurations which can be assigned via environment variables, others can be assigned via cargo features, and some can be assigned by both environment variables and cargo features. By centralizing configuration file, we don't need to bother about these things further more.
• For developers, add a new configuration is easier and for code reviewers, it is more intuitive to find a configuration added or changed.
• For users, the libafl_conf crate itself can also be used as a code-as-doc for building LibAFL. And we may also provide many configuration templates for difference purposes.

Drawbacks

Rerun

The cargo:rerun-if-changed=xxx things (docs here) should be re-considered.

To clarify, let's re-think the purpose for re-running:

• For developers

  • When developing a certain crate which need the build-script to re-run (for example, developing the QEMU code and need to debug the LibAFL and QEMU), re-run the build script every time
  • For crates we don't work on, never re-run the build script

• For users, never re-run the build script except for configuration changed

So current cargo:rerun-if-changed=xxx is not good enough for our purpose. I think we can do this in code-level rather than compiler level, i.e., we can add a rerun field in configuration to indicate whether this build script should be re-runned, and its default value is false. Note that this means the configuration file will be pared each time we run a build in each build script, and I believe this is not a heavy things so it may be acceptable.

Compatibility

Current projects depending on LibAFL may adopt new configuration files

Questions

So before I implementing such features, some questions need to be settled

• Whether we should keep current configuration methods for compatibility?

  If so, for environment-based configurations, it is easy to do so; For cargo-feature-based configurations, it need extra efforts since it is compiler-related

• Is the rerun thing mentioned above good?

• I am not familiar with LibAFL and I wonder if there are also some runtime configurations need to be unified like build configurations in libafl-run.toml?

• Other questions may be added in this discussion thread if I met.

Feel free to discuss this feature!:)

[domenukk]

Thanks for your interest! I think right now LibAFL's build.rs scripts make use regular environment variables at compiletime. Would it be a good first step to add an env variable for qemu_url for example? Something like LIBAFL_QEMU_REPO and LIBAFL_QEMU_REV env variables that get checked in build.rs?

[Evian-Zhang]

With a quick search of "env::var" in LibAFL's source code, it turns out there are 70 occurrences (some of them are checking env vars set by cargo internally) including a lot for QEMU and LLVM, and some of them can also be set with feature flags. However, env vars like QEMU_URL, QEMU_REV or CUSTOM_QEMU_DIR cannot be set via feature flags (at least I never saw similar conf in other crates).

To make this proposal more comprehensive, as an alternative to the centralized configuration file, this can also be done by env section in config.toml to make IDE-plugins recognize such configurations. However, this method lacks type info and cannot be hierarchically grouped, compared to a toml conf file.

I agree that cargo features are more rusty than a configuration file, so maybe we could only centralize those confs which couldn't be set via cargo features, and leave apart others. This could be another question we should discuss in this thread.

In terms of difficulty, I don't think it would be more difficult for contributors to use centralized configurations than env vars, since we could make this infrastructure independent and have a nice interface to use.

To make it work in practice, we could search this configuration file with a precedence rule: conf file defined in user's crate --> conf file defined in LibAFL's root (which is the default conf). And not all users of the library need the conf file, since most of the env vars should be optional and the default value should be hard-coded in the source code.

[Evian-Zhang]

To be more clarified about whether every users must have a config file, I think that, if we decide to replace the conf set by both env vars and cargo features with conf file, then answer is yes, since in most situations, currently users always specify cargo features. If we decide to only replace conf set by env vars, then users may not have a conf file since env vars are optional and can have many default values

[Evian-Zhang]

To complement my proposal, building Rust itself also requires a configuration file, as stated in rustc dev guide, and its content can be seen in config.example.toml, which is much like LibAFL's configurations set by env-vars and cargo-features.

[andreafioraldi]

Hi. I agree that env vars is not the best, how this config file will eventually affect the usage of libafl from crates.io? rustc is a tool not a library, it is not obvious that if something is good for it it is also good for libafl

[Evian-Zhang]

@andreafioraldi Thank you for your response.

how this config file will eventually affect the usage of libafl from crates.io?

As I said, most configurations in LibAFL have default values, so it is optional for a user to write a configuration file. And for those configurations that do not have default values, we can further discuss them one-by-one to determine whether it should be set by configuration file or in other ways.

rustc is a tool not a library, it is not obvious that if something is good for it it is also good for libafl

You are right, Rust is a tool while LibAFL is a library for users to build a binary. In order to give a more appropriate example, let's see tauri. This is a famous rust framework for users to build an application.

It has a tauri.conf.json in user's side, and the library will read it to setup some configurations. I think this can solve those conf-set-by-env-var problems.

For conf controlled by cargo features, tauri's code only do a check by comparing the features defined in configuration file and features set by Cargo.toml. I think this may due to We can't set cargo features via build.rs. As for LibAFL, this can be divided into two situations:

• features that control library code

  features like introspection does not have an optional dependency specified in Cargo.toml, those can be easily controlled by configuration file since we could use cargo:rust-cfg=KEY syntax to set those in build.rs.

• features that control library dependencies

  features like gzip will introduce optional dependencies like minis_oxide, those can't be setup by configuration file right now.

So we can also discuss whether we will include conf set by cargo features in configuration file in this discussion if we agree that we should maintain a configuration file to centralize configurations across LibAFL.