Cwerg aims to be a complete "from-scratch" compiler for a C-like language. The frontend is WIP. The multi-target backend is fairly far along and can be used independently from the frontend.
Documentation:
- Frontend Docs (misc. documentation)
- Cwerg Examples: Sexpr Syntax, Proposed Concrete Syntax
- Backend Docs (misc. documentation)
- Cwerg IR Examples (examples demonstrating backend API usage)
Most components are implemented twice (see rationale):
- spec/reference implementation: Python 3.9
- high performance implementation: C++17 (with limited STL usage)
Re-implementations in other languages are explicitly encouraged. A lot of code is table driven to facilitate that.
Cwerg de-emphasizes quality of the generated code (we hope to come within 50% of state of the art compilers) in favor of a small code base that can be understood by a single developer and very fast translation times.
Cwerg aims to be a much better C, avoiding some of its problems (pre-processor, array decay to pointers, etc.) while preserving its low level feel (no unexpected memory allocations or control flow changes).
Among the features added are:
- defer (scheduling code to run code at scope exit)
- tagged unions (sum types)
- optionally wrapped types (by-name type equivalence)
- modules
- simple hygienic macro system
- limited polymorphism
- slices (array views)
More details at Cwerg Frontend
The Cwerg backend is lightweight and suitable for new programming language implementations that want to avoid heavy dependencies like LLVM. It has no dependencies and can directly generate ELF executables for Arm32, Arm64 and X86-64 ISAs. Besides AOT compilation, (one-shot) JITing is also supported.
It currently consists of:
- RISC like Intermediate Representation (IR)
- Optimizer for the IR
- Elf Support Lib ((de-)compiler for ELF object files)
- A32 Support Lib ((dis-) assembler for ARM32 instructions)
- A64 Support Lib ((dis-) assembler for ARM64 instructions)
- X64 Support Lib ((dis-) assembler for X64-64 instructions)
- A32 Backend (code generator emitting ARM32 instructions)
- A64 Backend (code generator emitting ARM64 instructions)
- X64 Backend (code generator emitting X86-64 instructions)
- C Backend (code generator emitting C code)
- Standard Library (rudimentary library of mostly syscall wrappers)
- C Bindings (C bindings: wrappers around the C++ code)
If you are interested in using Cwerg as backend for your own compiler project, please check out Interfacing with Cwerg and/or reach out to the author.
- C Frontend (subset of C to IR compiler)
- WASM Frontend (WASM/WASI to IR transpiler)
The project tracks code size in LOC carefully. We use code size as a proxy for complexity and have the following self-imposed limits:
- frontend: 10kLOC
- IR optimizer: 10kLOC
- backend targets: 5kLOC per ISA
The limits are per implementation (e.g. C++ and Python) Code generated from tables is not counted but the tables (written in Python) are.
The goal for the c++ implementation of the backend is to translate the IR to an Elf executable at a speed of 500k IR instructions per sec using at most 4 cores on a 2020 era midrange desktop or high end laptop.
Whole program translation and parallel translation at the function level are explicit design goals for the C++ implementations.
Cwerg does not have a linker. Instead the responsibility of generating executables and resolving relocations rests with the assembler components of the various backends. An ELF library helps with the generation of ELF executables, which is the only object format currently supported.
To keep the project lightweight the feature set must be curtailed. Since the project is still evolving, the details are not entirely cast in stone but the following features are unlikely to be supported (contact us before starting any work on these):
- Instruction sets other than little endian (host and target) with 2's complement integers and IEEE floats.
- Variable number of function parameters (var-args). Basically only used for printf/scanf and responsible for a disproportionate amount of complexity in ABIs. (Note, this precludes a proper C frontend.)
- Full-blown dwarf debug info. The standard is over 300 pages long and unlikely to fit into the complexity budget. Line numbers will likely be supported.
- C++ exception/unwind tables. A lot of code and complexity that only benefits one language.
- Linking against code produced with other toolchains. There are currently no plans to emit linkable object code. And there is no ABI compatibility except for simple cases and syscalls.
- Shared libs/dynamic linking adds complexity and slows programs down (both because of slower code idioms and prevention of optimizations), not to mention the DLL hell problem. (see also: https://drewdevault.com/dynlib, https://www.kix.in/2008/06/19/an-alternative-to-shared-libraries/) (Lack of shared lib support likely precludes Windows as a target platform.)
- Sophisticated instruction scheduling which is less important for memory bound code and out-of-order CPUs.
- Sophisticated loop optimizations. Probably best left to the frontend.
- Variable sized stack frames (alloca). This requires a frame pointer, wasting a register, and makes it more difficult to reason about stack overflows.
- A large standard library with unicode and locale support. Those add a lot of complexity and are better left to dedicated libraries.
- Sophisticated DSLs like LLVM's Tablegen. DSLs increase cognitive load and require additional infrastructure like parsers which eat into our size targets. Instead we leverage the expressiveness of Python.
The IR optimizer currently does not use a full-blown Single Static Assigment (SSA) form. Instead it uses a modified use-def chain approach to get some of the benefits of SSA.
Ports for more regular architectures, e.g. RISC V, should be straight forward to implement (see porting hints).
Cwerg controls dependencies carefully to keep them at a bare minimum:
- pycparser used by the (optional) C frontend
