marslanalvi avatar

security-fuzzing

Fuzzing, sanitizers, property-based and cryptographic testing

version
1.0.0
GitHub stars
5

Install

/plugin install security-fuzzing@skills-stack

Run it inside Claude Code after adding the marketplace with /plugin marketplace add marslanalvi/SkillsStack.

Contents

Discovered from source

Skills 19

AddressSanitizer (ASan)

AddressSanitizer (ASan) is a widely adopted memory error detection tool used extensively during software testing, particularly fuzzing. It helps detect memory corruption bugs that might otherwise go unnoticed, such as buffer overflows, use-after-free errors, and other memory safety violations.

Overview

ASan is a standard practice in fuzzing due to its effectiveness in identifying memory vulnerabilities. It instruments code at compile time to track memory allocations and accesses, detecting illegal operations at runtime.

Key Concepts

ConceptDescription
InstrumentationASan adds runtime checks to memory operations during compilation
Shadow MemoryMaps 20TB of virtual memory to track allocation state
Performance CostApproximately 2-4x slowdown compared to non-instrumented code
Detection ScopeFinds buffer overflows, use-after-free, double-free, and memory leaks

When to Apply

Apply this technique when:

  • Fuzzing C/C++ code for memory safety vulnerabilities
  • Testing Rust code with unsafe blocks
  • Debugging crashes related to memory corruption
  • Running unit tests where memory errors are suspected

Skip this technique when:

  • Running production code (ASan can reduce security)
  • Platform is Windows or macOS (limited ASan support)
  • Performance overhead is unacceptable for your use case
  • Fuzzing pure safe languages without FFI (e.g., pure Go, pure Java)

Quick Reference

TaskCommand/Pattern
Enable ASan (Clang/GCC)-fsanitize=address
Enable verbosityASAN_OPTIONS=verbosity=1
Disable leak detectionASAN_OPTIONS=detect_leaks=0
Force abort on errorASAN_OPTIONS=abort_on_error=1
Multiple optionsASAN_OPTIONS=verbosity=1:abort_on_error=1

Step-by-Step

Step 1: Compile with ASan

Compile and link your code with the -fsanitize=address flag:

clang -fsanitize=address -g -o my_program my_program.c

The -g flag is recommended to get better stack traces when ASan detects errors.

Step 2: Configure ASan Options

Set the ASAN_OPTIONS environment variable to configure ASan behavior:

export ASAN_OPTIONS=verbosity=1:abort_on_error=1:detect_leaks=0

Step 3: Run Your Program

Execute the ASan-instrumented binary. When memory errors are detected, ASan will print detailed reports:

./my_program

Step 4: Adjust Fuzzer Memory Limits

ASan requires approximately 20TB of virtual memory. Disable fuzzer memory restrictions:

  • libFuzzer: -rss_limit_mb=0
  • AFL++: -m none

Common Patterns

Pattern: Basic ASan Integration

Use Case: Standard fuzzing setup with ASan

Before:

clang -o fuzz_target fuzz_target.c
./fuzz_target

After:

clang -fsanitize=address -g -o fuzz_target fuzz_target.c
ASAN_OPTIONS=verbosity=1:abort_on_error=1 ./fuzz_target

Pattern: ASan with Unit Tests

Use Case: Enable ASan for unit test suite

Before:

gcc -o test_suite test_suite.c -lcheck
./test_suite

After:

gcc -fsanitize=address -g -o test_suite test_suite.c -lcheck
ASAN_OPTIONS=detect_leaks=1 ./test_suite

Advanced Usage

Tips and Tricks

TipWhy It Helps
Use -g flagProvides detailed stack traces for debugging
Set verbosity=1Confirms ASan is enabled before program starts
Disable leaks during fuzzingLeak detection doesn't cause immediate crashes, clutters output
Enable abort_on_error=1Some fuzzers require abort() instead of _exit()

Understanding ASan Reports

When ASan detects a memory error, it prints a detailed report including:

  • Error type: Buffer overflow, use-after-free, etc.
  • Stack trace: Where the error occurred
  • Allocation/deallocation traces: Where memory was allocated/freed
  • Memory map: Shadow memory state around the error

Example ASan report:

==12345==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60300000eff4 at pc 0x00000048e6a3
READ of size 4 at 0x60300000eff4 thread T0
    #0 0x48e6a2 in main /path/to/file.c:42

Combining Sanitizers

ASan can be combined with other sanitizers for comprehensive detection:

clang -fsanitize=address,undefined -g -o fuzz_target fuzz_target.c

Platform-Specific Considerations

Linux: Full ASan support with best performance macOS: Limited support, some features may not work Windows: Experimental support, not recommended for production fuzzing

Anti-Patterns

Anti-PatternProblemCorrect Approach
Using ASan in productionCan make applications less secureUse ASan only for testing
Not disabling memory limitsFuzzer may kill process due to 20TB virtual memorySet -rss_limit_mb=0 or -m none
Ignoring leak reportsMemory leaks indicate resource management issuesReview leak reports at end of fuzzing campaign

Tool-Specific Guidance

libFuzzer

Compile with both fuzzer and address sanitizer:

clang++ -fsanitize=fuzzer,address -g harness.cc -o fuzz

Run with unlimited RSS:

./fuzz -rss_limit_mb=0

Integration tips:

  • Always combine -fsanitize=fuzzer with -fsanitize=address
  • Use -g for detailed stack traces in crash reports
  • Consider ASAN_OPTIONS=abort_on_error=1 for better crash handling

See: libFuzzer: AddressSanitizer

AFL++

Use the AFL_USE_ASAN environment variable:

AFL_USE_ASAN=1 afl-clang-fast++ -g harness.cc -o fuzz

Run with unlimited memory:

afl-fuzz -m none -i input_dir -o output_dir ./fuzz

Integration tips:

  • AFL_USE_ASAN=1 automatically adds proper compilation flags
  • Use -m none to disable AFL++'s memory limit
  • Consider AFL_MAP_SIZE for programs with large coverage maps

See: AFL++: AddressSanitizer

cargo-fuzz (Rust)

Use the --sanitizer=address flag:

cargo fuzz run fuzz_target --sanitizer=address

Or configure in fuzz/Cargo.toml:

[profile.release]
opt-level = 3
debug = true

Integration tips:

  • ASan is useful for fuzzing unsafe Rust code or FFI boundaries
  • Safe Rust code may not benefit as much (compiler already prevents many errors)
  • Focus on unsafe blocks, raw pointers, and C library bindings

See: cargo-fuzz: AddressSanitizer

honggfuzz

Compile with ASan and link with honggfuzz:

honggfuzz -i input_dir -o output_dir -- ./fuzz_target_asan

Compile the target:

hfuzz-clang -fsanitize=address -g target.c -o fuzz_target_asan

Integration tips:

  • honggfuzz works well with ASan out of the box
  • Use feedback-driven mode for better coverage with sanitizers
  • Monitor memory usage, as ASan increases memory footprint

Troubleshooting

IssueCauseSolution
Fuzzer kills process immediatelyMemory limit too low for ASan's 20TB virtual memoryUse -rss_limit_mb=0 (libFuzzer) or -m none (AFL++)
"ASan runtime not initialized"Wrong linking order or missing runtimeEnsure -fsanitize=address used in both compile and link
Leak reports clutter outputLeakSanitizer enabled by defaultSet ASAN_OPTIONS=detect_leaks=0
Poor performance (>4x slowdown)Debug mode or unoptimized buildCompile with -O2 or -O3 alongside -fsanitize=address
ASan not detecting obvious bugsBinary not instrumentedCheck with ASAN_OPTIONS=verbosity=1 that ASan prints startup info
False positivesInterceptor conflictsCheck ASan FAQ for known issues with specific libraries

Related Skills

Tools That Use This Technique

SkillHow It Applies
libfuzzerCompile with -fsanitize=fuzzer,address for integrated fuzzing with memory error detection
aflppUse AFL_USE_ASAN=1 environment variable during compilation
cargo-fuzzUse --sanitizer=address flag to enable ASan for Rust fuzz targets
honggfuzzCompile target with -fsanitize=address for ASan-instrumented fuzzing

Related Techniques

SkillRelationship
undefined-behavior-sanitizerOften used together with ASan for comprehensive bug detection (undefined behavior + memory errors)
fuzz-harness-writingHarnesses must be designed to handle ASan-detected crashes and avoid false positives
coverage-analysisCoverage-guided fuzzing helps trigger code paths where ASan can detect memory errors

Resources

Key External Resources

AddressSanitizer on Google Sanitizers Wiki

The official ASan documentation covers:

  • Algorithm and implementation details
  • Complete list of detected error types
  • Performance characteristics and overhead
  • Platform-specific behavior
  • Known limitations and incompatibilities

SanitizerCommonFlags

Common configuration flags shared across all sanitizers:

  • verbosity: Control diagnostic output level
  • log_path: Redirect sanitizer output to files
  • symbolize: Enable/disable symbol resolution in reports
  • external_symbolizer_path: Use custom symbolizer

AddressSanitizerFlags

ASan-specific configuration options:

  • detect_leaks: Control memory leak detection
  • abort_on_error: Call abort() vs _exit() on error
  • detect_stack_use_after_return: Detect stack use-after-return bugs
  • check_initialization_order: Find initialization order bugs

AddressSanitizer FAQ

Common pitfalls and solutions:

  • Linking order issues
  • Conflicts with other tools
  • Platform-specific problems
  • Performance tuning tips

Clang AddressSanitizer Documentation

Clang-specific guidance:

  • Compilation flags and options
  • Interaction with other Clang features
  • Supported platforms and architectures

GCC Instrumentation Options

GCC-specific ASan documentation:

  • GCC-specific flags and behavior
  • Differences from Clang implementation
  • Platform support in GCC

AddressSanitizer: A Fast Address Sanity Checker (USENIX Paper)

Original research paper with technical details:

  • Shadow memory algorithm
  • Virtual memory requirements (historically 16TB, now ~20TB)
  • Performance benchmarks
  • Design decisions and tradeoffs

AFL++

AFL++ is a fork of the original AFL fuzzer that offers better fuzzing performance and more advanced features while maintaining stability. A major benefit over libFuzzer is that AFL++ has stable support for running fuzzing campaigns on multiple cores, making it ideal for large-scale fuzzing efforts.

When to Use

FuzzerBest ForComplexity
AFL++Multi-core fuzzing, diverse mutations, mature projectsMedium
libFuzzerQuick setup, single-threaded, simple harnessesLow
LibAFLCustom fuzzers, research, advanced use casesHigh

Choose AFL++ when:

  • You need multi-core fuzzing to maximize throughput
  • Your project can be compiled with Clang or GCC
  • You want diverse mutation strategies and mature tooling
  • libFuzzer has plateaued and you need more coverage
  • You're fuzzing production codebases that benefit from parallel execution

Quick Start

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    // Call your code with fuzzer-provided data
    check_buf((char*)data, size);
    return 0;
}

Compile and run:

# Setup AFL++ wrapper script first (see Installation)
./afl++ docker afl-clang-fast++ -DNO_MAIN=1 -O2 -fsanitize=fuzzer harness.cc main.cc -o fuzz
mkdir seeds && echo "aaaa" > seeds/minimal_seed
./afl++ docker afl-fuzz -i seeds -o out -- ./fuzz

Installation

AFL++ has many dependencies including LLVM, Python, and Rust. We recommend using a current Debian or Ubuntu distribution for fuzzing with AFL++.

MethodWhen to UseSupported Compilers
Ubuntu/Debian reposRecent Ubuntu, basic features onlyUbuntu 23.10: Clang 14 & GCC 13
Debian 12: Clang 14 & GCC 12
Docker (from Docker Hub)Specific AFL++ version, Apple Silicon supportAs of 4.35c: Clang 19 & GCC 11
Docker (from source)Test unreleased features, apply patchesConfigurable in Dockerfile
From sourceAvoid Docker, need specific patchesAdjustable via LLVM_CONFIG env var

Ubuntu/Debian

Prior to installing afl++, check the clang version dependency of the packge with apt-cache show afl++, and install the matching lld version (e.g., lld-17).

apt install afl++ lld-17

Docker (from Docker Hub)

docker pull aflplusplus/aflplusplus:stable

Docker (from source)

git clone --depth 1 --branch stable https://github.com/AFLplusplus/AFLplusplus
cd AFLplusplus
docker build -t aflplusplus .

From source

Refer to the Dockerfile for Ubuntu version requirements and dependencies. Set LLVM_CONFIG to specify Clang version (e.g., llvm-config-18).

Wrapper Script Setup

Create a wrapper script to run AFL++ on host or Docker:

cat <<'EOF' > ./afl++
#!/bin/sh
AFL_VERSION="${AFL_VERSION:-"stable"}"
case "$1" in
   host)
        shift
        bash -c "$*"
        ;;
    docker)
        shift
        /usr/bin/env docker run -i \
            --privileged \
            -v ./:/src \
            --rm \
            --name "afl_fuzzing_$$" \
            "aflplusplus/aflplusplus:$AFL_VERSION" \
            bash -c "cd /src && bash -c \"$*\""
        ;;
    *)
        echo "Usage: $0 {host|docker}"
        exit 1
        ;;
esac
EOF
chmod +x ./afl++

The examples below use docker mode, apart from the system configuration commands that have to reach the host kernel. Swap in host to run any of them against an AFL++ installed on the machine itself. The wrapper joins everything after the mode argument into a single shell string, so quoting does not survive: an argument containing a space (-x "my dict.dict") arrives word-split. Rename such files without spaces, or edit the wrapper for that run.

The missing -t is deliberate. docker run -ti aborts with the input device is not a TTY whenever stdin is not a terminal, which covers CI jobs and anything an agent or script drives. afl-fuzz notices there is no terminal and prints plain status lines in place of the full-screen UI. A program that insists on a terminal, such as watch, has to run on the host side of the wrapper instead. $$ expands to the wrapper's PID, so parallel instances get distinct container names rather than colliding on a single afl_fuzzing. docker ps truncates the COMMAND column, so every row looks alike; use docker ps --no-trunc to tell the instances apart before stopping one.

Security Warning: The afl-system-config and afl-persistent-config scripts require root privileges and disable OS security features. Do not fuzz on production systems or your development environment. Use a dedicated VM instead.

System Configuration

Run after each reboot for up to 15% more executions per second:

./afl++ host afl-system-config

afl-system-config tunes the kernel it runs against, so run it on the machine that hosts the campaign. ./afl++ docker afl-system-config reaches the same settings through the wrapper's --privileged container, which is the only route when AFL++ is installed via Docker alone.

For maximum performance, disable kernel security mitigations (requires grub bootloader, not supported in Docker):

./afl++ host afl-persistent-config
update-grub
reboot
./afl++ host afl-system-config

Verify with cat /proc/cmdline - output should include mitigations=off.

Writing a Harness

Harness Structure

AFL++ supports libFuzzer-style harnesses:

#include <stdint.h>
#include <stddef.h>

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    // 1. Validate input size if needed
    if (size < MIN_SIZE || size > MAX_SIZE) return 0;

    // 2. Call target function with fuzz data
    target_function(data, size);

    // 3. Return 0 (non-zero reserved for future use)
    return 0;
}

Harness Rules

DoDon't
Reset global state between runsRely on state from previous runs
Handle edge cases gracefullyExit on invalid input
Keep harness deterministicUse random number generators
Free allocated memoryCreate memory leaks
Validate input sizesProcess unbounded input

See Also: For detailed harness writing techniques, patterns for handling complex inputs, and advanced strategies, see the fuzz-harness-writing technique skill.

Compilation

AFL++ offers multiple compilation modes with different trade-offs.

Compilation Mode Decision Tree

Choose your compilation mode:

  • LTO mode (afl-clang-lto): Best performance and instrumentation. Try this first.
  • LLVM mode (afl-clang-fast): Fall back if LTO fails to compile.
  • GCC plugin (afl-gcc-fast): For projects requiring GCC.

Basic Compilation (LLVM mode)

./afl++ docker afl-clang-fast++ -DNO_MAIN=1 -O2 -fsanitize=fuzzer harness.cc main.cc -o fuzz

GCC Compilation

./afl++ docker afl-g++-fast -DNO_MAIN=1 -O2 -fsanitize=fuzzer harness.cc main.cc -o fuzz

Important: GCC version must match the version used to compile the AFL++ GCC plugin.

With Sanitizers

./afl++ docker AFL_USE_ASAN=1 afl-clang-fast++ -DNO_MAIN=1 -O2 -fsanitize=fuzzer harness.cc main.cc -o fuzz

See Also: For detailed sanitizer configuration, common issues, and advanced flags, see the address-sanitizer and undefined-behavior-sanitizer technique skills.

Build Flags

Note that -g is not necessary, it is added by default by the AFL++ compilers.

FlagPurpose
-DNO_MAIN=1Skip main function when using libFuzzer harness
-O2Production optimization level (recommended for fuzzing)
-fsanitize=fuzzerEnable libFuzzer compatibility mode and adds the fuzzer runtime when linking executable
-fsanitize=fuzzer-no-linkInstrument without linking fuzzer runtime (for static libraries and object files)

Corpus Management

Creating Initial Corpus

AFL++ requires at least one non-empty seed file:

mkdir seeds
echo "aaaa" > seeds/minimal_seed

For real projects, gather representative inputs:

  • Download example files for the format you're fuzzing
  • Extract test cases from the project's test suite
  • Use minimal valid inputs for your file format

Corpus Minimization

After a campaign, minimize the corpus to keep only unique coverage:

./afl++ docker afl-cmin -i out/default/queue -o minimized_corpus -- ./fuzz

See Also: For corpus creation strategies, dictionaries, and seed selection, see the fuzzing-corpus technique skill.

Running Campaigns

Basic Run

./afl++ docker afl-fuzz -i seeds -o out -- ./fuzz

Setting Environment Variables

./afl++ docker AFL_FAST_CAL=1 afl-fuzz -i seeds -o out -- ./fuzz

Interpreting Output

AFL++ reports these statistics either way, but how you read them depends on the mode. The wrapper's docker run -i gives the container no TTY, so afl-fuzz drops the full-screen UI and writes plain status lines to the log instead — the fields below appear there, and in state/<instance>/fuzzer_stats. To get the interactive UI, run host mode in a terminal, or add -t to the wrapper for a run you are watching by hand.

OutputMeaning
execs/secExecution speed - higher is better
cycles doneNumber of queue passes completed
corpus countNumber of unique test cases in queue
saved crashesNumber of unique crashes found
stability% of stable edges (should be near 100%)

Output Directory Structure

out/default/
├── cmdline          # How was the SUT invoked?
├── crashes/         # Inputs that crash the SUT
│   └── id:000000,sig:06,src:000002,time:286,execs:13105,op:havoc,rep:4
├── hangs/           # Inputs that hang the SUT
├── queue/           # Test cases reproducing final fuzzer state
│   ├── id:000000,time:0,execs:0,orig:minimal_seed
│   └── id:000001,src:000000,time:0,execs:8,op:havoc,rep:6,+cov
├── fuzzer_stats     # Campaign statistics
└── plot_data        # Data for plotting

Analyzing Results

View live campaign statistics:

./afl++ docker afl-whatsup out

Create coverage plots. The aflplusplus image already ships gnuplot-nox; in host mode, install gnuplot first with apt install gnuplot.

./afl++ docker afl-plot out/default out_graph/

Re-executing Test Cases

Pass one of the filenames from out/default/crashes/:

./afl++ docker ./fuzz out/default/crashes/id:000000,sig:06,src:000002,time:286,execs:13105,op:havoc,rep:4

Fuzzer Options

OptionPurpose
-G 4000Maximum test input length (default: 1048576 bytes)
-t 1000Timeout in milliseconds for each test case (default: 1000ms)
-m 1000Memory limit in megabytes (default: 0 = unlimited)
-x ./dict.dictUse dictionary file to guide mutations

Environment Variables That Matter

AFL++ has many environment variables, but most are niche. These are the ones that matter in practice.

Always Set These

# Every campaign should use tmpfs — SSDs will thank you, and it's faster
AFL_TMPDIR=/dev/shm

AFL_TMPDIR is a free performance win with no downsides — not setting it wears out your SSD and slows fuzzing.

Slow Targets

# Speeds up calibration ~2.5x — use when targets are slow (e.g., >10 ms/exec)
AFL_FAST_CAL=1

AFL_FAST_CAL reduces calibration time with negligible precision loss. Recommended specifically for slow targets where calibration would otherwise take a long time.

Multi-Core Campaigns

# On the primary (-M) instance only — needed for afl-cmin, not for fuzzing itself
AFL_FINAL_SYNC=1

# On all instances — cache test cases in memory (default: 50 MB, good range: 50-250 MB)
AFL_TESTCACHE_SIZE=100

AFL_FINAL_SYNC tells the primary instance to do a final import from all secondaries when stopping. This does not affect the fuzzing process itself — it only matters when you later run afl-cmin for corpus minimization, ensuring the primary's queue has the full combined corpus. AFL_TESTCACHE_SIZE caches test cases in memory to reduce disk I/O; the default is 50 MB and values between 50-250 MB work well for most campaigns.

CI/Automated Fuzzing

# Fail fast if fuzzing isn't finding anything
AFL_EXIT_ON_TIME=3600  # 1 hour with no new paths = stop

# Or run until "done" (all queue entries processed)
AFL_EXIT_WHEN_DONE=1

# Headless environments
AFL_NO_UI=1

Unbounded fuzzing in CI wastes resources. Set time limits or use exit conditions.

Variables to Avoid

VariableWhy Skip It
AFL_NO_ARITHCan hurt coverage on binary formats, but may be useful for text-based targets
AFL_SHUFFLE_QUEUEOnly for exotic setups, usually harmful
AFL_DISABLE_TRIMTrimming is valuable, don't disable without reason

Multi-Core Fuzzing

AFL++ excels at multi-core fuzzing with two major advantages:

  1. More executions per second (scales linearly with physical cores)
  2. Asymmetrical fuzzing (e.g., one ASan job, rest without sanitizers)

Starting a Campaign

Start the primary fuzzer (in background):

./afl++ docker afl-fuzz -M primary -i seeds -o state -- ./fuzz 1>primary.log 2>primary.error </dev/null &

Start secondary fuzzers (as many as you have cores):

./afl++ docker afl-fuzz -S secondary01 -i seeds -o state -- ./fuzz 1>secondary01.log 2>secondary01.error </dev/null &
./afl++ docker afl-fuzz -S secondary02 -i seeds -o state -- ./fuzz 1>secondary02.log 2>secondary02.error </dev/null &

The </dev/null is required, not decorative. docker run -i keeps the client reading its own stdin, and a backgrounded process that reads the terminal is sent SIGTTIN, whose default action stops it — so without the redirect these jobs show up as Stopped in jobs and never fuzz.

Monitoring Multi-Core Campaigns

List all running jobs:

jobs

View live statistics. watch needs a terminal that docker run -i does not give it, so wrap the whole invocation instead of running watch inside the container. Every tick starts a container, which is why the interval is 5 seconds rather than 1:

watch -n5 --color ./afl++ docker afl-whatsup state/

Stopping All Fuzzers

kill $(jobs -p)

Coverage Analysis

AFL++ automatically tracks coverage through edge instrumentation. Coverage information is stored in fuzzer_stats and plot_data.

Measuring Coverage

Use afl-plot to visualize coverage over time:

./afl++ docker afl-plot out/default out_graph/

Improving Coverage

  • Use dictionaries for format-aware fuzzing
  • Run longer campaigns (cycles_wo_finds indicates plateau)
  • Try different mutation strategies with multi-core fuzzing
  • Analyze coverage gaps and add targeted seed inputs

See Also: For detailed coverage analysis techniques, identifying coverage gaps, and systematic coverage improvement, see the coverage-analysis technique skill.

CMPLOG

CMPLOG/RedQueen is the best path constraint solving mechanism available in any fuzzer. To enable it, the fuzz target needs to be instrumented for it. Before building the fuzzing target set the environment variable:

./afl++ docker AFL_LLVM_CMPLOG=1 make

No special action is needed for compiling and linking the harness.

To run a fuzzer instance with a CMPLOG instrumented fuzzing target, add -c0 to the command like arguments:

./afl++ docker afl-fuzz -c0 -S cmplog -i seeds -o state -- ./fuzz 1>cmplog.log 2>cmplog.error </dev/null &

Sanitizer Integration

Sanitizers are essential for finding memory corruption bugs that don't cause immediate crashes.

AddressSanitizer (ASan)

./afl++ docker AFL_USE_ASAN=1 afl-clang-fast++ -DNO_MAIN=1 -O2 -fsanitize=fuzzer harness.cc main.cc -o fuzz

Note: Memory limit (-m) is not supported with ASan due to 20TB virtual memory reservation.

UndefinedBehaviorSanitizer (UBSan)

./afl++ docker AFL_USE_UBSAN=1 afl-clang-fast++ -DNO_MAIN=1 -O2 -fsanitize=fuzzer,undefined harness.cc main.cc -o fuzz

Common Sanitizer Issues

IssueSolution
ASan slows fuzzingUse only 1 ASan job in multi-core setup
Stack exhaustionIncrease stack with ASAN_OPTIONS=stack_size=...
GCC version mismatchEnsure system GCC matches AFL++ plugin version

See Also: For comprehensive sanitizer configuration and troubleshooting, see the address-sanitizer technique skill.

Advanced Usage

Tips and Tricks

TipWhy It Helps
Use LLVMFuzzerTestOneInput harnesses where possibleIf a fuzzing campaign has at least 85% stability then this is the most efficient fuzzing style. If not then try standard input or file input fuzzing
Use dictionariesHelps fuzzer discover format-specific keywords and magic bytes
Set realistic timeoutsPrevents false positives from system load
Limit input sizeLarger inputs don't necessarily explore more space
Monitor stabilityLow stability indicates non-deterministic behavior

Standard Input Fuzzing

AFL++ can fuzz programs reading from stdin without a libFuzzer harness:

./afl++ docker afl-clang-fast++ -O2 main_stdin.c -o fuzz_stdin
./afl++ docker afl-fuzz -i seeds -o out -- ./fuzz_stdin

This is slower than persistent mode but requires no harness code.

File Input Fuzzing

For programs that read files, use @@ placeholder:

./afl++ docker afl-clang-fast++ -O2 main_file.c -o fuzz_file
./afl++ docker afl-fuzz -i seeds -o out -- ./fuzz_file @@

For better performance, use fmemopen to create file descriptors from memory.

Argument Fuzzing

Fuzz command-line arguments using argv-fuzz-inl.h:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#ifdef __AFL_COMPILER
#include "argv-fuzz-inl.h"
#endif

void check_buf(char *buf, size_t buf_len) {
    if(buf_len > 0 && buf[0] == 'a') {
        if(buf_len > 1 && buf[1] == 'b') {
            if(buf_len > 2 && buf[2] == 'c') {
                abort();
            }
        }
    }
}

int main(int argc, char *argv[]) {
#ifdef __AFL_COMPILER
    AFL_INIT_ARGV();
#endif

    if (argc < 2) {
        fprintf(stderr, "Usage: %s <input_string>\n", argv[0]);
        return 1;
    }

    char *input_buf = argv[1];
    size_t len = strlen(input_buf);
    check_buf(input_buf, len);
    return 0;
}

Download the header:

curl -O https://raw.githubusercontent.com/AFLplusplus/AFLplusplus/stable/utils/argv_fuzzing/argv-fuzz-inl.h

Compile and run:

./afl++ docker afl-clang-fast++ -O2 main_arg.c -o fuzz_arg
./afl++ docker afl-fuzz -i seeds -o out -- ./fuzz_arg

Performance Tuning

SettingImpact
CPU core countLinear scaling with physical cores
Persistent mode10-20x faster than fork server
-G input size limitSmaller = faster, but may miss bugs
ASan ratio1 ASan job per 4-8 non-ASan jobs

Troubleshooting

ProblemCauseSolution
Low exec/sec (<1k)Not using persistent modeCreate a LLVMFuzzerTestOneInput style harness
Low stability (<85%)Non-deterministic codeFuzz a program via stdin or file inputs, or create such a harness
GCC plugin errorGCC version mismatchEnsure system GCC matches AFL++ build and install gcc-$GCC_VERSION-plugin-dev
No crashes foundNeed sanitizersRecompile with AFL_USE_ASAN=1
Memory limit exceededASan uses 20TB virtualRemove -m flag when using ASan
Docker performance lossVirtualization overheadUse bare metal or VM for production fuzzing

Related Skills

Technique Skills

SkillUse Case
fuzz-harness-writingDetailed guidance on writing effective harnesses
address-sanitizerMemory error detection during fuzzing
undefined-behavior-sanitizerDetect undefined behavior bugs
fuzzing-corpusBuilding and managing seed corpora
fuzzing-dictionariesCreating dictionaries for format-aware fuzzing

Related Fuzzers

SkillWhen to Consider
libfuzzerQuick prototyping, single-threaded fuzzing is sufficient
libaflNeed custom mutators or research-grade features

Resources

Key External Resources

AFL++ GitHub Repository Official repository with comprehensive documentation, examples, and issue tracker.

Fuzzing in Depth Advanced documentation by the AFL++ team covering instrumentation modes, optimization techniques, and advanced use cases.

AFL++ Under The Hood Technical deep-dive into AFL++ internals, mutation strategies, and coverage tracking mechanisms.

AFL++: Combining Incremental Steps of Fuzzing Research Research paper describing AFL++ architecture and performance improvements over original AFL.

Video Resources

Atheris

Atheris is a coverage-guided Python fuzzer built on libFuzzer. It enables fuzzing of both pure Python code and Python C extensions with integrated AddressSanitizer support for detecting memory corruption issues.

When to Use

FuzzerBest ForComplexity
AtherisPython code and C extensionsLow-Medium
HypothesisProperty-based testingLow
python-aflAFL-style fuzzingMedium

Choose Atheris when:

  • Fuzzing pure Python code with coverage guidance
  • Testing Python C extensions for memory corruption
  • Integration with libFuzzer ecosystem is desired
  • AddressSanitizer support is needed

Quick Start

import sys
import atheris

@atheris.instrument_func
def TestOneInput(data: bytes):
    if len(data) == 4:
        if data[0] == 0x46:  # "F"
            if data[1] == 0x55:  # "U"
                if data[2] == 0x5A:  # "Z"
                    if data[3] == 0x5A:  # "Z"
                        raise RuntimeError("You caught me")

def main():
    atheris.Setup(sys.argv, TestOneInput)
    atheris.Fuzz()

if __name__ == "__main__":
    main()

Run:

uv run python fuzz.py

Installation

Atheris supports 32-bit and 64-bit Linux, and macOS. We recommend fuzzing on Linux because it's simpler to manage and often faster.

Prerequisites

Linux/macOS

uv init --bare   # once, if the harness directory is not yet a uv project
uv add atheris

Docker Environment (Recommended)

For a fully operational Linux environment with all dependencies configured:

# https://hub.docker.com/_/python
ARG PYTHON_VERSION=3.11

FROM python:$PYTHON_VERSION-slim-bookworm

RUN python --version

RUN apt update && apt install -y \
    ca-certificates \
    wget \
    && rm -rf /var/lib/apt/lists/*

# LLVM builds version 15-19 for Debian 12 (Bookworm)
# https://apt.llvm.org/bookworm/dists/
ARG LLVM_VERSION=19

RUN echo "deb http://apt.llvm.org/bookworm/ llvm-toolchain-bookworm-$LLVM_VERSION main" > /etc/apt/sources.list.d/llvm.list
RUN echo "deb-src http://apt.llvm.org/bookworm/ llvm-toolchain-bookworm-$LLVM_VERSION main" >> /etc/apt/sources.list.d/llvm.list
RUN wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key > /etc/apt/trusted.gpg.d/apt.llvm.org.asc

RUN apt update && apt install -y \
    build-essential \
    clang-$LLVM_VERSION \
    && rm -rf /var/lib/apt/lists/*

ENV APP_DIR "/app"
RUN mkdir $APP_DIR
WORKDIR $APP_DIR

ENV VIRTUAL_ENV "/opt/venv"
RUN python -m venv $VIRTUAL_ENV
ENV PATH "$VIRTUAL_ENV/bin:$PATH"

# https://github.com/google/atheris/blob/master/native_extension_fuzzing.md#step-1-compiling-your-extension
ENV CC="clang-$LLVM_VERSION"
ENV CFLAGS "-fsanitize=address,fuzzer-no-link"
ENV CXX="clang++-$LLVM_VERSION"
ENV CXXFLAGS "-fsanitize=address,fuzzer-no-link"
ENV LDSHARED="clang-$LLVM_VERSION -shared"
ENV LDSHAREDXX="clang++-$LLVM_VERSION -shared"
ENV ASAN_SYMBOLIZER_PATH="/usr/bin/llvm-symbolizer-$LLVM_VERSION"

# Allow Atheris to find fuzzer sanitizer shared libs
# https://github.com/google/atheris#building-from-source
RUN LIBFUZZER_LIB=$($CC -print-file-name=libclang_rt.fuzzer_no_main-$(uname -m).a) \
    python -m pip install --no-binary atheris atheris

# https://github.com/google/atheris/blob/master/native_extension_fuzzing.md#option-a-sanitizerlibfuzzer-preloads
ENV LD_PRELOAD "$VIRTUAL_ENV/lib/python3.11/site-packages/asan_with_fuzzer.so"

# 1. Skip memory allocation failures for now, they are common, and low impact (DoS)
# 2. https://github.com/google/atheris/blob/master/native_extension_fuzzing.md#leak-detection
ENV ASAN_OPTIONS "allocator_may_return_null=1,detect_leaks=0"

CMD ["/bin/bash"]

Build and run:

docker build -t atheris .
docker run -it atheris

Verification

python -c "import atheris; print(atheris.__version__)"

Writing a Harness

Harness Structure for Pure Python

import sys
import atheris

@atheris.instrument_func
def TestOneInput(data: bytes):
    """
    Fuzzing entry point. Called with random byte sequences.

    Args:
        data: Random bytes generated by the fuzzer
    """
    # Add input validation if needed
    if len(data) < 1:
        return

    # Call your target function
    try:
        your_target_function(data)
    except ValueError:
        # Expected exceptions should be caught
        pass
    # Let unexpected exceptions crash (that's what we're looking for!)

def main():
    atheris.Setup(sys.argv, TestOneInput)
    atheris.Fuzz()

if __name__ == "__main__":
    main()

Structured Input with FuzzedDataProvider

A target taking several typed arguments wastes most of the fuzzer's inputs if the harness slices data by hand, because every mutation shifts the byte offsets of everything after it. atheris.FuzzedDataProvider splits one bytes input into typed values instead:

fdp = atheris.FuzzedDataProvider(data)
name = fdp.ConsumeUnicodeNoSurrogates(fdp.ConsumeIntInRange(0, 64))
strict = fdp.ConsumeBool()

See structured-input.md for the full method reference, the fixed-draw- order rule, and what each method returns once the buffer runs dry.

Harness Rules

DoDon't
Use @atheris.instrument_func for coverageForget to instrument target code
Catch expected exceptionsCatch all exceptions indiscriminately
Use atheris.instrument_imports() for librariesImport modules after atheris.Setup()
Keep harness deterministicUse randomness or time-based behavior

See Also: For detailed harness writing techniques, patterns for handling complex inputs, and advanced strategies, see the fuzz-harness-writing technique skill.

Fuzzing Pure Python Code

For fuzzing broader parts of an application or library, use instrumentation functions:

import atheris
with atheris.instrument_imports():
    import your_module
    from another_module import target_function

def TestOneInput(data: bytes):
    target_function(data)

atheris.Setup(sys.argv, TestOneInput)
atheris.Fuzz()

Instrumentation Options:

  • atheris.instrument_func - Decorator for single function instrumentation
  • atheris.instrument_imports() - Context manager for instrumenting all imported modules
  • atheris.instrument_all() - Instrument all Python code system-wide

Fuzzing Python C Extensions

Python C extensions require compilation with specific flags for instrumentation and sanitizer support.

Environment Configuration

If using the provided Dockerfile, these are already configured. For local setup:

export CC="clang"
export CFLAGS="-fsanitize=address,fuzzer-no-link"
export CXX="clang++"
export CXXFLAGS="-fsanitize=address,fuzzer-no-link"
export LDSHARED="clang -shared"

Example: Fuzzing cbor2

Install the extension from source:

CBOR2_BUILD_C_EXTENSION=1 uv add --no-binary-package cbor2 'cbor2==5.6.4'

The --no-binary-package flag ensures the C extension is compiled locally with instrumentation rather than pulled as a prebuilt wheel. Persist that choice with no-binary-package = ["cbor2"] under [tool.uv] in pyproject.toml, or a later uv sync can silently swap in an uninstrumented wheel.

Create cbor2-fuzz.py:

import sys
import atheris

# _cbor2 ensures the C library is imported
from _cbor2 import loads

def TestOneInput(data: bytes):
    try:
        loads(data)
    except Exception:
        # We're searching for memory corruption, not Python exceptions
        pass

def main():
    atheris.Setup(sys.argv, TestOneInput)
    atheris.Fuzz()

if __name__ == "__main__":
    main()

Run:

uv run python cbor2-fuzz.py

Important: When running locally (not in Docker), you must set LD_PRELOAD manually.

Corpus Management

Creating Initial Corpus

mkdir corpus
# Add seed inputs
echo "test data" > corpus/seed1
echo '{"key": "value"}' > corpus/seed2

Run with corpus:

uv run python fuzz.py corpus/

Corpus Minimization

Atheris inherits corpus minimization from libFuzzer:

uv run python fuzz.py -merge=1 new_corpus/ old_corpus/

See Also: For corpus creation strategies, dictionaries, and seed selection, see the fuzzing-corpus technique skill.

Running Campaigns

Basic Run

uv run python fuzz.py

With Corpus Directory

uv run python fuzz.py corpus/

Common Options

# Run for 10 minutes
uv run python fuzz.py -max_total_time=600

# Limit input size
uv run python fuzz.py -max_len=1024

# Run with multiple workers
uv run python fuzz.py -workers=4 -jobs=4

Interpreting Output

OutputMeaning
NEW cov: XFound new coverage, corpus expanded
pulse cov: XPeriodic status update
exec/s: XExecutions per second (throughput)
corp: X/YbCorpus size: X inputs, Y bytes total
ERROR: libFuzzerCrash detected

Sanitizer Integration

AddressSanitizer (ASan)

AddressSanitizer is automatically integrated when using the provided Docker environment or when compiling with appropriate flags.

For local setup:

export CFLAGS="-fsanitize=address,fuzzer-no-link"
export CXXFLAGS="-fsanitize=address,fuzzer-no-link"

Configure ASan behavior:

export ASAN_OPTIONS="allocator_may_return_null=1,detect_leaks=0"

LD_PRELOAD Configuration

For native extension fuzzing:

export LD_PRELOAD="$(python -c 'import atheris; import os; print(os.path.join(os.path.dirname(atheris.__file__), "asan_with_fuzzer.so"))')"

See Also: For detailed sanitizer configuration, common issues, and advanced flags, see the address-sanitizer and undefined-behavior-sanitizer technique skills.

Common Sanitizer Issues

IssueSolution
LD_PRELOAD not setExport LD_PRELOAD to point to asan_with_fuzzer.so
Memory allocation failuresSet ASAN_OPTIONS=allocator_may_return_null=1
Leak detection noiseSet ASAN_OPTIONS=detect_leaks=0
Missing symbolizerSet ASAN_SYMBOLIZER_PATH to llvm-symbolizer

Advanced Usage

Tips and Tricks

TipWhy It Helps
Use atheris.instrument_imports() earlyEnsures all imports are instrumented for coverage
Start with small max_lenFaster initial fuzzing, gradually increase
Use dictionaries for structured formatsHelps fuzzer understand format tokens
Run multiple parallel instancesBetter coverage exploration

Custom Instrumentation

Fine-tune what gets instrumented:

import atheris

# Instrument only specific modules
with atheris.instrument_imports():
    import target_module
# Don't instrument test harness code

def TestOneInput(data: bytes):
    target_module.parse(data)

Performance Tuning

SettingImpact
-max_len=NSmaller values = faster execution
-workers=N -jobs=NParallel fuzzing for faster coverage
ASAN_OPTIONS=fast_unwind_on_malloc=0Better stack traces, slower execution

UndefinedBehaviorSanitizer (UBSan)

Add UBSan to catch additional bugs:

export CFLAGS="-fsanitize=address,undefined,fuzzer-no-link"
export CXXFLAGS="-fsanitize=address,undefined,fuzzer-no-link"

Note: Modify flags in Dockerfile if using containerized setup.

Real-World Examples

Two complete harnesses — a pure-Python parser and an HTTP response parser — are in examples.md.

Troubleshooting

ProblemCauseSolution
No coverage increasePoor seed corpus or target not instrumentedAdd better seeds, verify instrument_imports()
Slow executionASan overhead or large inputsReduce max_len, use ASAN_OPTIONS=fast_unwind_on_malloc=1
Import errorsModules imported before instrumentationMove imports inside instrument_imports() context
Segfault without ASan outputMissing LD_PRELOADSet LD_PRELOAD to asan_with_fuzzer.so path
Build failuresWrong compiler or missing flagsVerify CC, CFLAGS, and clang version

Related Skills

Technique Skills

SkillUse Case
fuzz-harness-writingDetailed guidance on writing effective harnesses
address-sanitizerMemory error detection during fuzzing
undefined-behavior-sanitizerCatching undefined behavior in C extensions
coverage-analysisMeasuring and improving code coverage
fuzzing-corpusBuilding and managing seed corpora

Related Fuzzers

SkillWhen to Consider
hypothesisProperty-based testing with type-aware generation
python-aflAFL-style fuzzing for Python when Atheris isn't available

Resources

Key External Resources

Atheris GitHub Repository Official repository with installation instructions, examples, and documentation for fuzzing both pure Python and native extensions.

Native Extension Fuzzing Guide Comprehensive guide covering compilation flags, LD_PRELOAD setup, sanitizer configuration, and troubleshooting for Python C extensions.

Continuously Fuzzing Python C Extensions Trail of Bits blog post covering CI/CD integration, ClusterFuzzLite setup, and real-world examples of fuzzing Python C extensions in continuous integration pipelines.

ClusterFuzzLite Python Integration Guide for integrating Atheris fuzzing into CI/CD pipelines using ClusterFuzzLite for automated continuous fuzzing.

Video Resources

Videos and tutorials are available in the main Atheris documentation and libFuzzer resources.

cargo-fuzz

cargo-fuzz is the de facto choice for fuzzing Rust projects when using Cargo. It uses libFuzzer as the backend and provides a convenient Cargo subcommand that automatically enables relevant compilation flags for your Rust project, including support for sanitizers like AddressSanitizer.

When to Use

cargo-fuzz is currently the primary and most mature fuzzing solution for Rust projects using Cargo.

FuzzerBest ForComplexity
cargo-fuzzCargo-based Rust projects, quick setupLow
AFL++Multi-core fuzzing, non-Cargo projectsMedium
LibAFLCustom fuzzers, research, advanced use casesHigh

Choose cargo-fuzz when:

  • Your project uses Cargo (required)
  • You want simple, quick setup with minimal configuration
  • You need integrated sanitizer support
  • You're fuzzing Rust code with or without unsafe blocks

Quick Start

#![no_main]

use libfuzzer_sys::fuzz_target;

fn harness(data: &[u8]) {
    your_project::check_buf(data);
}

fuzz_target!(|data: &[u8]| {
    harness(data);
});

Initialize and run:

cargo fuzz init
# Edit fuzz/fuzz_targets/fuzz_target_1.rs with your harness
cargo +nightly fuzz run fuzz_target_1

Installation

cargo-fuzz requires the nightly Rust toolchain because it uses features only available in nightly.

Prerequisites

  • Rust and Cargo installed via rustup
  • Nightly toolchain

Linux/macOS

# Install nightly toolchain
rustup install nightly

# Install cargo-fuzz
cargo install cargo-fuzz

Verification

cargo +nightly --version
cargo fuzz --version

Writing a Harness

Project Structure

cargo-fuzz works best when your code is structured as a library crate. If you have a binary project, split your main.rs into:

src/main.rs  # Entry point (main function)
src/lib.rs   # Code to fuzz (public functions)
Cargo.toml

Initialize fuzzing:

cargo fuzz init

This creates:

fuzz/
├── Cargo.toml
└── fuzz_targets/
    └── fuzz_target_1.rs

Harness Structure

#![no_main]

use libfuzzer_sys::fuzz_target;

fn harness(data: &[u8]) {
    // 1. Validate input size if needed
    if data.is_empty() {
        return;
    }

    // 2. Call target function with fuzz data
    your_project::target_function(data);
}

fuzz_target!(|data: &[u8]| {
    harness(data);
});

Harness Rules

DoDon't
Structure code as library crateKeep everything in main.rs
Use fuzz_target! macroWrite custom main function
Handle Result::Err gracefullyPanic on expected errors
Keep harness deterministicUse random number generators

See Also: For detailed harness writing techniques and structure-aware fuzzing with the arbitrary crate, see the fuzz-harness-writing technique skill.

Structure-Aware Fuzzing

cargo-fuzz integrates with the arbitrary crate for structure-aware fuzzing:

// In your library crate
use arbitrary::Arbitrary;

#[derive(Debug, Arbitrary)]
pub struct Name {
    data: String
}
// In your fuzz target
#![no_main]
use libfuzzer_sys::fuzz_target;

fuzz_target!(|data: your_project::Name| {
    data.check_buf();
});

Add to your library's Cargo.toml:

[dependencies]
arbitrary = { version = "1", features = ["derive"] }

Running Campaigns

Basic Run

cargo +nightly fuzz run fuzz_target_1

Without Sanitizers (Safe Rust)

If your project doesn't use unsafe Rust, disable sanitizers for 2x performance boost:

cargo +nightly fuzz run --sanitizer none fuzz_target_1

Check if your project uses unsafe code:

cargo install cargo-geiger
cargo geiger

Re-executing Test Cases

# Run a specific test case (e.g., a crash)
cargo +nightly fuzz run fuzz_target_1 fuzz/artifacts/fuzz_target_1/crash-<hash>

# Run all corpus entries without fuzzing
cargo +nightly fuzz run fuzz_target_1 fuzz/corpus/fuzz_target_1 -- -runs=0

Using Dictionaries

cargo +nightly fuzz run fuzz_target_1 -- -dict=./dict.dict

Interpreting Output

OutputMeaning
NEWNew coverage-increasing input discovered
pulsePeriodic status update
INITEDFuzzer initialized successfully
Crash with stack traceBug found, saved to fuzz/artifacts/

Corpus location: fuzz/corpus/fuzz_target_1/ Crashes location: fuzz/artifacts/fuzz_target_1/

Sanitizer Integration

AddressSanitizer (ASan)

ASan is enabled by default and detects memory errors:

cargo +nightly fuzz run fuzz_target_1

Disabling Sanitizers

For pure safe Rust (no unsafe blocks in your code or dependencies):

cargo +nightly fuzz run --sanitizer none fuzz_target_1

Performance impact: ASan adds ~2x overhead. Disable for safe Rust to improve fuzzing speed.

Checking for Unsafe Code

cargo install cargo-geiger
cargo geiger

See Also: For detailed sanitizer configuration, flags, and troubleshooting, see the address-sanitizer technique skill.

Coverage Analysis

cargo-fuzz integrates with Rust's coverage tools to analyze fuzzing effectiveness.

Prerequisites

rustup toolchain install nightly --component llvm-tools-preview
cargo install cargo-binutils
cargo install rustfilt

Generating Coverage Reports

# Generate coverage data from corpus
cargo +nightly fuzz coverage fuzz_target_1

Create coverage generation script:

cat <<'EOF' > ./generate_html
#!/bin/sh
if [ $# -lt 1 ]; then
    echo "Error: Name of fuzz target is required."
    echo "Usage: $0 fuzz_target [sources...]"
    exit 1
fi
FUZZ_TARGET="$1"
shift
SRC_FILTER="$@"
TARGET=$(rustc -vV | sed -n 's|host: ||p')
cargo +nightly cov -- show -Xdemangler=rustfilt \
  "target/$TARGET/coverage/$TARGET/release/$FUZZ_TARGET" \
  -instr-profile="fuzz/coverage/$FUZZ_TARGET/coverage.profdata"  \
  -show-line-counts-or-regions -show-instantiations  \
  -format=html -o fuzz_html/ $SRC_FILTER
EOF
chmod +x ./generate_html

Generate HTML report:

./generate_html fuzz_target_1 src/lib.rs

HTML report saved to: fuzz_html/

See Also: For detailed coverage analysis techniques and systematic coverage improvement, see the coverage-analysis technique skill.

Advanced Usage

Tips and Tricks

TipWhy It Helps
Start with a seed corpusDramatically speeds up initial coverage discovery
Use --sanitizer none for safe Rust2x performance improvement
Check coverage regularlyIdentifies gaps in harness or seed corpus
Use dictionaries for parsersHelps overcome magic value checks
Structure code as libraryRequired for cargo-fuzz integration

libFuzzer Options

Pass options to libFuzzer after --:

# See all options
cargo +nightly fuzz run fuzz_target_1 -- -help=1

# Set timeout per run
cargo +nightly fuzz run fuzz_target_1 -- -timeout=10

# Use dictionary
cargo +nightly fuzz run fuzz_target_1 -- -dict=dict.dict

# Limit maximum input size
cargo +nightly fuzz run fuzz_target_1 -- -max_len=1024

Multi-Core Fuzzing

# Experimental forking support (not recommended)
cargo +nightly fuzz run --jobs 1 fuzz_target_1

Note: The multi-core fuzzing feature is experimental and not recommended. For parallel fuzzing, consider running multiple instances manually or using AFL++.

Real-World Examples

Example: ogg Crate

The ogg crate parses Ogg media container files. Parsers are excellent fuzzing targets because they handle untrusted data.

# Clone and initialize
git clone https://github.com/RustAudio/ogg.git
cd ogg/
cargo fuzz init

Harness at fuzz/fuzz_targets/fuzz_target_1.rs:

#![no_main]

use ogg::{PacketReader, PacketWriter};
use ogg::writing::PacketWriteEndInfo;
use std::io::Cursor;
use libfuzzer_sys::fuzz_target;

fn harness(data: &[u8]) {
    let mut pck_rdr = PacketReader::new(Cursor::new(data.to_vec()));
    pck_rdr.delete_unread_packets();

    let output = Vec::new();
    let mut pck_wtr = PacketWriter::new(Cursor::new(output));

    if let Ok(_) = pck_rdr.read_packet() {
        if let Ok(r) = pck_rdr.read_packet() {
            match r {
                Some(pck) => {
                    let inf = if pck.last_in_stream() {
                        PacketWriteEndInfo::EndStream
                    } else if pck.last_in_page() {
                        PacketWriteEndInfo::EndPage
                    } else {
                        PacketWriteEndInfo::NormalPacket
                    };
                    let stream_serial = pck.stream_serial();
                    let absgp_page = pck.absgp_page();
                    let _ = pck_wtr.write_packet(
                        pck.data, stream_serial, inf, absgp_page
                    );
                }
                None => return,
            }
        }
    }
}

fuzz_target!(|data: &[u8]| {
    harness(data);
});

Seed the corpus:

mkdir fuzz/corpus/fuzz_target_1/
curl -o fuzz/corpus/fuzz_target_1/320x240.ogg \
  https://commons.wikimedia.org/wiki/File:320x240.ogg

Run:

cargo +nightly fuzz run fuzz_target_1

Analyze coverage:

cargo +nightly fuzz coverage fuzz_target_1
./generate_html fuzz_target_1 src/lib.rs

Troubleshooting

ProblemCauseSolution
"requires nightly" errorUsing stable toolchainUse cargo +nightly fuzz
Slow fuzzing performanceASan enabled for safe RustAdd --sanitizer none flag
"cannot find binary"No library crateMove code from main.rs to lib.rs
Sanitizer compilation issuesWrong nightly versionTry different nightly: rustup install nightly-2024-01-01
Low coverageMissing seed corpusAdd sample inputs to fuzz/corpus/fuzz_target_1/
Magic value not foundNo dictionaryCreate dictionary file with magic values

Related Skills

Technique Skills

SkillUse Case
fuzz-harness-writingStructure-aware fuzzing with arbitrary crate
address-sanitizerUnderstanding ASan output and configuration
coverage-analysisMeasuring and improving fuzzing effectiveness
fuzzing-corpusBuilding and managing seed corpora
fuzzing-dictionariesCreating dictionaries for format-aware fuzzing

Related Fuzzers

SkillWhen to Consider
libfuzzerFuzzing C/C++ code with similar workflow
aflppMulti-core fuzzing or non-Cargo Rust projects
libaflAdvanced fuzzing research or custom fuzzer development

Resources

Rust Fuzz Book - cargo-fuzz Official documentation for cargo-fuzz covering installation, usage, and advanced features.

arbitrary crate documentation Guide to structure-aware fuzzing with automatic derivation for Rust types.

cargo-fuzz GitHub Repository Source code, issue tracker, and examples for cargo-fuzz.

Constant-Time Analysis

Compile the code, inspect the emitted assembly or bytecode for variable-time instructions, then decide which of the flagged operations actually touch secrets. The compilation step is mechanical; the triage step is the work.

When to Use

  • Implementing or reviewing a signature, encryption, KEM, or key derivation routine
  • Code applies / or % to a value derived from a key, plaintext, nonce, or token
  • The user mentions "constant-time", "timing attack", "side-channel", or "KyberSlash"
  • Reviewing functions named sign, verify, encrypt, decrypt, derive_key

When NOT to Use

  • Measuring timing variance on a running binary — use the constant-time-testing skill from the testing-handbook-skills plugin, which covers dudect and statistical approaches and may not be installed. This skill inspects compiler output statically and never executes the code under test.
  • Non-cryptographic code, or crypto code where every input is public
  • High-level API usage where a vetted library owns the constant-time guarantees
  • Cache and other microarchitectural side channels — the assembly view cannot see them

Language Routing

Read the guide for the target language before interpreting any findings; each one lists that language's dangerous instructions and the idiomatic constant-time replacements.

GuideLanguages
references/compiled.mdC, C++, Go, Rust
references/swift.mdSwift
references/vm-compiled.mdJava, C#
references/kotlin.mdKotlin
references/php.mdPHP
references/javascript.mdJavaScript, TypeScript
references/python.mdPython
references/ruby.mdRuby

Running the Analyzer

The analyzer takes one file and detects the language from its extension. Always pass --warnings:

uv run {baseDir}/ct_analyzer/analyzer.py --warnings <source_file>

Without it the analyzer reports only error-severity findings, which means division, modulo and weak RNG. Four detector families are warning severity and stay silent: secret-dependent branches, early-exit comparison (memcmp, strcmp, .equals, ==), table lookups indexed by a secret, and variable-time encoding. Early-exit comparison of an authentication tag is the most common timing bug in real code — Lucky Thirteen was exactly that — so a default run is quiet about the finding you are most likely to have.

FlagEffect
--warningsAdd the four warning-severity families above. Pass it every time
--func <regex>Restrict output to function names matching the regex
--jsonMachine-readable output
--githubGitHub Actions annotations
--arch <target>Target architecture (x86_64, arm64, riscv64, ...) — native languages only
--opt-level <level>Optimization level (O0 through O3, Os, Oz) — native languages only
--compiler <name>Override compiler choice (gcc, clang, go, rustc, swiftc)

Narrow a large file to the routines that handle secrets with a regex, for example --func 'sign|verify'.

Run natively compiled code (C, C++, Go, Rust, Swift) at more than one --arch and --opt-level. Division timing and branch lowering are architecture- and optimization-dependent: x86_64 IDIV and arm64 SDIV differ, and a cmov at -O2 can become a branch at -O0. A single clean run proves one configuration safe, not the code.

How --arch crosses depends on the toolchain. clang crosses with --target and needs no second compiler, but any source that includes libc headers also needs that target's C library headers — libc6-dev-riscv64-cross and friends — or it fails with bits/libc-header-start.h file not found. Go cross-builds through GOARCH, though go tool objdump has no riscv64 disassembler. A GNU cross toolchain is a separate binary, so gcc needs it named explicitly — --compiler x86_64-linux-gnu-gcc, --compiler riscv64-linux-gnu-gcc — and nothing is substituted for you, so the report always names the binary that ran. rustc needs the target's standard library (rustup target add), and Swift on Linux targets only the host. Compare against the toolchain that builds your product, not whichever cross build a distribution packages.

Re-run the whole sweep on the fix, across compilers, targets and every level including Os and Oz. Any fix that works by handing the compiler a constant divisor to strength-reduce is a fix only where the compiler chooses to cooperate, and that choice varies more than it looks. Replacing key_coef / (2 * gamma2) with a #defined divisor still emits a real divide here:

ToolchainLevels that emit a division
gcc riscv64O0 through Oz — every level
gcc arm64, gcc x86_64Os, Oz
clang arm64O0, Oz

Strength reduction is an optimizer courtesy, not a language guarantee. Prefer an explicit multiply-shift, and verify it against the original expression over the full input range rather than on sampled values — an off-by-a-power-of-two reciprocal matches for millions of inputs before it diverges.

Java, Kotlin, and C# compile to JVM/CIL bytecode. The analyzer reads that bytecode, so --arch and --opt-level do not apply and the JIT may still introduce variable-time native code the analyzer cannot see.

Per-language coverage limits

Coverage is not uniform, and the gaps change what a clean report means:

LanguageWhat the report does not cover
GoOnly symbols from the analyzed file. go build links the runtime in, and its divisions — all on public data — would otherwise dominate the findings
JavaScript, TypeScriptBytecode findings are restricted to functions the file declares by name, because V8 dumps node's internals the same way it dumps yours. Anonymous callbacks fall to the source scan. For TypeScript, bytecode findings name the function but carry no line, since V8's positions index the transpiled output
Python, Ruby, PHPBytecode reflects the interpreter that ran, not a JIT'd or alternative runtime
RustAnalyzed as a library unless the file declares fn main; private functions with no caller may be optimized away before analysis
SwiftTargets the host platform on Linux; iOS and macOS triples need an Apple toolchain

Since findings and silence both depend on the configuration, say which compiler, architecture, and optimization level produced a result when reporting it.

To sweep a directory, loop in the shell — the analyzer is a deterministic script, one invocation per file:

for f in src/crypto/*.c; do uv run {baseDir}/ct_analyzer/analyzer.py --warnings --json "$f"; done

Prerequisites

LanguageRequirement
C, C++, Go, Rustgcc/clang, go, rustc in PATH
SwiftXcode or Swift toolchain (swiftc)
Java / KotlinJDK (javac, javap); Kotlin also needs kotlinc
C#.NET SDK plus ilspycmd (dotnet tool install -g ilspycmd)
PHPPHP with the VLD extension or OPcache
JavaScript / TypeScriptNode.js
PythonPython 3.x
RubyRuby with --dump=insns support

On a "toolchain not found" error, see references/vm-compiled.md for JVM and .NET installation, macOS keg-only PATH configuration, and troubleshooting.

Interpreting Results

PASSED — no error-severity finding for the configuration you ran. Warnings do not affect it, so Result: PASSED alongside Warnings: 6 is normal and is not a clean result. Read the warning list before concluding anything.

FAILED — dangerous instructions found, reported per function:

[ERROR] SDIV
  Function: decompose_vulnerable
  Reason: SDIV has early termination optimization; execution time depends on operand values

Triaging Findings

The analyzer has no data flow analysis. It flags every dangerous instruction regardless of whether a secret reaches it, so a FAILED report is a worklist, not a verdict. Reporting the raw output as a set of vulnerabilities is the primary failure mode of this skill.

For each flagged instruction, read the source and answer one question: does an operand depend on secret data? Trace from the instruction's function back to the caller's inputs, then classify:

// FALSE POSITIVE: operands are a buffer length, already public from the ciphertext size
int num_blocks = data_len / 16;

// TRUE POSITIVE: dividend is a private-key coefficient; IDIV/SDIV leaks its magnitude
int32_t q = secret_coef / GAMMA2;
QuestionIf yes
Is the operand a compile-time constant?Likely false positive
Is the operand a public parameter — length, count, index bound?Likely false positive
Is the operand derived from a key, plaintext, nonce, or token?True positive
Can an attacker influence the operand's value?True positive

State the verdict and the data flow that justifies it for every flagged item. A finding you cannot trace to a secret is not a finding; say so explicitly rather than dropping it silently.

{baseDir}/ct_analyzer/tests/triage_samples/ holds a known-answer case per language: each fixture pairs a true positive with a false positive that the analyzer reports identically, and expectations.json records which is which and why. triage_c.c is the shortest example — the analyzer flags the division in both ct_high_bits and ct_block_count, and correct triage confirms the first and clears the second.

Weak-RNG and encoding findings ask a different question. For Math.random, mt_rand, random.randint, System.Random and base64_encode, no operand is secret, so "does an operand depend on a secret?" does not resolve them. Ask instead what the result is used for: seeding a nonce or key is a true positive, jittering a retry delay is not. These are reported by a regex scan over the source rather than from bytecode, so they are attributed to <source> with a line number instead of to the enclosing function — except in PHP, where they carry the function.

Comparison and lookup findings have their own question, and their own fix. For an early-exit comparison, ask whether either side is secret: comparing an authentication tag, MAC, or password hash is a true positive, comparing a public protocol header is not. For a table lookup, ask whether the index is secret — the array's contents do not matter, only what selects the element. Both are exploitable as written, so a confirmed one needs the language's constant-time primitive rather than a rewrite of the loop:

LanguageConstant-time comparison
C, C++CRYPTO_memcmp (OpenSSL) or sodium_memcmp
Gocrypto/subtle.ConstantTimeCompare
Rustthe subtle crate's ConstantTimeEq
Java, KotlinMessageDigest.isEqual
C#CryptographicOperations.FixedTimeEquals
PHPhash_equals
Pythonhmac.compare_digest
RubyOpenSSL.secure_compare
JavaScript, TypeScriptcrypto.timingSafeEqual

A secret-indexed lookup has no drop-in replacement: it needs a bit-sliced or arithmetic formulation that touches every element, which is why AES S-box tables are the classic case. Encoding a secret through a table — base64_encode, bin2hex, chr/ord — is the same problem in a library, and paragonie/constant_time_encoding is the reference fix for PHP.

Limitations

  1. Static only — reads assembly and bytecode, never runtime behavior. Cache timing and other microarchitectural channels are invisible.
  2. No data flow analysis — see triage above.
  3. Configuration-specific — a different compiler, optimization level, architecture, or runtime version can emit different instructions from identical source.

Real-World Impact

  • KyberSlash (2023) — division instructions in ML-KEM implementations allowed key recovery
  • Lucky Thirteen (2013) — timing differences in CBC padding validation enabled plaintext recovery
  • RSA timing attacks — early implementations leaked private key bits through division timing

References

Constant-Time Testing

Timing attacks exploit variations in execution time to extract secret information from cryptographic implementations. Unlike cryptanalysis that targets theoretical weaknesses, timing attacks leverage implementation flaws - and they can affect any cryptographic code.

Background

Timing attacks were introduced by Kocher in 1996. Since then, researchers have demonstrated practical attacks on RSA (Schindler), OpenSSL (Brumley and Boneh), AES implementations, and even post-quantum algorithms like Kyber.

Key Concepts

ConceptDescription
Constant-timeCode path and memory accesses independent of secret data
Timing leakageObservable execution time differences correlated with secrets
Side channelInformation extracted from implementation rather than algorithm
MicroarchitectureCPU-level timing differences (cache, division, shifts)

Why This Matters

Timing vulnerabilities can:

  • Expose private keys - Extract secret exponents in RSA/ECDH
  • Enable remote attacks - Network-observable timing differences
  • Bypass cryptographic security - Undermine theoretical guarantees
  • Persist silently - Often undetected without specialized analysis

Two prerequisites enable exploitation:

  1. Access to oracle - Sufficient queries to the vulnerable implementation
  2. Timing dependency - Correlation between execution time and secret data

Common Constant-Time Violation Patterns

Four patterns account for most timing vulnerabilities:

// 1. Conditional jumps - most severe timing differences
if(secret == 1) { ... }
while(secret > 0) { ... }

// 2. Array access - cache-timing attacks
lookup_table[secret];

// 3. Integer division (processor dependent)
data = secret / m;

// 4. Shift operation (processor dependent)
data = a << secret;

Conditional jumps cause different code paths, leading to vast timing differences.

Array access dependent on secrets enables cache-timing attacks, as shown in AES cache-timing research.

Integer division and shift operations leak secrets on certain CPU architectures and compiler configurations.

When patterns cannot be avoided, employ masking techniques to remove correlation between timing and secrets.

Example: Modular Exponentiation Timing Attacks

Modular exponentiation (used in RSA and Diffie-Hellman) is susceptible to timing attacks. RSA decryption computes:

$$ct^{d} \mod{N}$$

where $d$ is the secret exponent. The exponentiation by squaring optimization reduces multiplications to $\log{d}$:

$$ \begin{align*} & \textbf{Input: } \text{base }y,\text{exponent } d={d_n,\cdots,d_0}_2,\text{modulus } N \ & r = 1 \ & \textbf{for } i=|n| \text{ downto } 0: \ & \quad\textbf{if } d_i == 1: \ & \quad\quad r = r * y \mod{N} \ & \quad y = y * y \mod{N} \ & \textbf{return }r \end{align*} $$

The code branches on exponent bit $d_i$, violating constant-time principles. When $d_i = 1$, an additional multiplication occurs, increasing execution time and leaking bit information.

Montgomery multiplication (commonly used for modular arithmetic) also leaks timing: when intermediate values exceed modulus $N$, an additional reduction step is required. An attacker constructs inputs $y$ and $y'$ such that:

$$ \begin{align*} y^2 < y^3 < N \ y'^2 < N \leq y'^3 \end{align*} $$

For $y$, both multiplications take time $t_1+t_1$. For $y'$, the second multiplication requires reduction, taking time $t_1+t_2$. This timing difference reveals whether $d_i$ is 0 or 1.

When to Use

Apply constant-time analysis when:

  • Auditing cryptographic implementations (primitives, protocols)
  • Code handles secret keys, passwords, or sensitive cryptographic material
  • Implementing crypto algorithms from scratch
  • Reviewing PRs that touch crypto code
  • Investigating potential timing vulnerabilities

Consider alternatives when:

  • Code does not process secret data
  • Public algorithms with no secret inputs
  • Non-cryptographic timing requirements (performance optimization)

Quick Reference

ScenarioRecommended ApproachSkill
Prove absence of leaksFormal verificationSideTrail, ct-verif, FaCT
Detect statistical timing differencesStatistical testingdudect
Track secret data flow at runtimeDynamic analysistimecop
Find cache-timing vulnerabilitiesSymbolic executionBinsec, pitchfork

Constant-Time Tooling Categories

The cryptographic community has developed four categories of timing analysis tools:

CategoryApproachProsCons
FormalMathematical proof on modelGuarantees absence of leaksComplexity, modeling assumptions
SymbolicSymbolic execution pathsConcrete counterexamplesTime-intensive path exploration
DynamicRuntime tracing with marked secretsGranular, flexibleLimited coverage to executed paths
StatisticalMeasure real execution timingPractical, simple setupNo root cause, noise sensitivity

1. Formal Tools

Formal verification mathematically proves timing properties on an abstraction (model) of code. Tools create a model from source/binary and verify it satisfies specified properties (e.g., variables annotated as secret).

Popular tools:

Strengths: Proof of absence, language-agnostic (LLVM bytecode) Weaknesses: Requires expertise, modeling assumptions may miss real-world issues

2. Symbolic Tools

Symbolic execution analyzes how paths and memory accesses depend on symbolic variables (secrets). Provides concrete counterexamples. Focus on cache-timing attacks.

Popular tools:

Strengths: Concrete counterexamples aid debugging Weaknesses: Path explosion leads to long execution times

3. Dynamic Tools

Dynamic analysis marks sensitive memory regions and traces execution to detect timing-dependent operations.

Popular tools:

Strengths: Granular control, targeted analysis Weaknesses: Coverage limited to executed paths

Detailed Guidance: See the timecop skill for setup and usage.

4. Statistical Tools

Execute code with various inputs, measure elapsed time, and detect inconsistencies. Tests actual implementation including compiler optimizations and architecture.

Popular tools:

Strengths: Simple setup, practical real-world results Weaknesses: No root cause info, noise obscures weak signals

Detailed Guidance: See the dudect skill for setup and usage.

Testing Workflow

Phase 1: Static Analysis        Phase 2: Statistical Testing
┌─────────────────┐            ┌─────────────────┐
│ Identify secret │      →     │ Detect timing   │
│ data flow       │            │ differences     │
│ Tool: ct-verif  │            │ Tool: dudect    │
└─────────────────┘            └─────────────────┘
         ↓                              ↓
Phase 4: Root Cause             Phase 3: Dynamic Tracing
┌─────────────────┐            ┌─────────────────┐
│ Pinpoint leak   │      ←     │ Track secret    │
│ location        │            │ propagation     │
│ Tool: Timecop   │            │ Tool: Timecop   │
└─────────────────┘            └─────────────────┘

Recommended approach:

  1. Start with dudect - Quick statistical check for timing differences
  2. If leaks found - Use Timecop to pinpoint root cause
  3. For high-assurance - Apply formal verification (ct-verif, SideTrail)
  4. Continuous monitoring - Integrate dudect into CI pipeline

Tools and Approaches

Dudect - Statistical Analysis

Dudect measures execution time for two input classes (fixed vs random) and uses Welch's t-test to detect statistically significant differences.

Detailed Guidance: See the dudect skill for complete setup, usage patterns, and CI integration.

Quick Start for Constant-Time Analysis
#define DUDECT_IMPLEMENTATION
#include "dudect.h"

uint8_t do_one_computation(uint8_t *data) {
    // Code to measure goes here
}

void prepare_inputs(dudect_config_t *c, uint8_t *input_data, uint8_t *classes) {
    for (size_t i = 0; i < c->number_measurements; i++) {
        classes[i] = randombit();
        uint8_t *input = input_data + (size_t)i * c->chunk_size;
        if (classes[i] == 0) {
            // Fixed input class
        } else {
            // Random input class
        }
    }
}

Key advantages:

  • Simple C header-only integration
  • Statistical rigor via Welch's t-test
  • Works with compiled binaries (real-world conditions)

Key limitations:

  • No root cause information when leak detected
  • Sensitive to measurement noise
  • Cannot guarantee absence of leaks (statistical confidence only)

Timecop - Dynamic Tracing

Timecop wraps Valgrind to detect runtime operations dependent on secret memory regions.

Detailed Guidance: See the timecop skill for installation, examples, and debugging.

Quick Start for Constant-Time Analysis
#include "valgrind/memcheck.h"

#define poison(addr, len) VALGRIND_MAKE_MEM_UNDEFINED(addr, len)
#define unpoison(addr, len) VALGRIND_MAKE_MEM_DEFINED(addr, len)

int main() {
    unsigned long long secret_key = 0x12345678;

    // Mark secret as poisoned
    poison(&secret_key, sizeof(secret_key));

    // Any branching or memory access dependent on secret_key
    // will be reported by Valgrind
    crypto_operation(secret_key);

    unpoison(&secret_key, sizeof(secret_key));
}

Run with Valgrind:

valgrind --leak-check=full --track-origins=yes ./binary

Key advantages:

  • Pinpoints exact line of timing leak
  • No code instrumentation required
  • Tracks secret propagation through execution

Key limitations:

  • Cannot detect microarchitecture timing differences
  • Coverage limited to executed paths
  • Performance overhead (runs on synthetic CPU)

Implementation Guide

Phase 1: Initial Assessment

Identify cryptographic code handling secrets:

  • Private keys, exponents, nonces
  • Password hashes, authentication tokens
  • Encryption/decryption operations

Quick statistical check:

  1. Write dudect harness for the crypto function
  2. Run for 5-10 minutes with timeout 600 ./ct_test
  3. Monitor t-value: high absolute values indicate leakage

Tools: dudect Expected time: 1-2 hours (harness writing + initial run)

Phase 2: Detailed Analysis

If dudect detects leakage:

Root cause investigation:

  1. Mark secret variables with Timecop poison()
  2. Run under Valgrind to identify exact line
  3. Review the four common violation patterns
  4. Check assembly output for conditional branches

Tools: Timecop, compiler output (objdump -d)

Phase 3: Remediation

Fix the timing leak:

  • Replace conditional branches with constant-time selection (bitwise operations)
  • Use constant-time comparison functions
  • Replace array lookups with constant-time alternatives or masking
  • Verify compiler doesn't optimize away constant-time code

Re-verify:

  1. Run dudect again for extended period (30+ minutes)
  2. Test across different compilers and optimization levels
  3. Test on different CPU architectures

Phase 4: Continuous Monitoring

Integrate into CI:

  • Add dudect tests to test suite
  • Run for fixed duration (5-10 minutes in CI)
  • Fail build if leakage detected

See the dudect skill for CI integration examples.

Common Vulnerabilities

VulnerabilityDescriptionDetectionSeverity
Secret-dependent branchif (secret_bit) { ... }dudect, TimecopCRITICAL
Secret-dependent array accesstable[secret_index]Timecop, BinsecHIGH
Variable-time divisionresult = x / secretTimecopMEDIUM
Variable-time shiftresult = x << secretTimecopMEDIUM
Montgomery reduction leakExtra reduction when intermediate > NdudectHIGH

Secret-Dependent Branch: Deep Dive

The vulnerability: Execution time differs based on whether branch is taken. Common in optimized modular exponentiation (square-and-multiply).

How to detect with dudect:

uint8_t do_one_computation(uint8_t *data) {
    uint64_t base = ((uint64_t*)data)[0];
    uint64_t exponent = ((uint64_t*)data)[1]; // Secret!
    return mod_exp(base, exponent, MODULUS);
}

void prepare_inputs(dudect_config_t *c, uint8_t *input_data, uint8_t *classes) {
    for (size_t i = 0; i < c->number_measurements; i++) {
        classes[i] = randombit();
        uint64_t *input = (uint64_t*)(input_data + i * c->chunk_size);
        input[0] = rand(); // Random base
        input[1] = (classes[i] == 0) ? FIXED_EXPONENT : rand(); // Fixed vs random
    }
}

How to detect with Timecop:

poison(&exponent, sizeof(exponent));
result = mod_exp(base, exponent, modulus);
unpoison(&exponent, sizeof(exponent));

Valgrind will report:

Conditional jump or move depends on uninitialised value(s)
  at 0x40115D: mod_exp (example.c:14)

Related skill: dudect, timecop

Case Studies

Case Study: OpenSSL RSA Timing Attack

Brumley and Boneh (2005) extracted RSA private keys from OpenSSL over a network. The vulnerability exploited Montgomery multiplication's variable-time reduction step.

Attack vector: Timing differences in modular exponentiation Detection approach: Statistical analysis (precursor to dudect) Impact: Remote key extraction

Tools used: Custom timing measurement Techniques applied: Statistical analysis, chosen-ciphertext queries

Case Study: KyberSlash

Post-quantum algorithm Kyber's reference implementation contained timing vulnerabilities in polynomial operations. Division operations leaked secret coefficients.

Attack vector: Secret-dependent division timing Detection approach: Dynamic analysis and statistical testing Impact: Secret key recovery in post-quantum cryptography

Tools used: Timing measurement tools Techniques applied: Differential timing analysis

Advanced Usage

Tips and Tricks

TipWhy It Helps
Pin dudect to isolated CPU core (taskset -c 2)Reduces OS noise, improves signal detection
Test multiple compilers (gcc, clang, MSVC)Optimizations may introduce or remove leaks
Run dudect for extended periods (hours)Increases statistical confidence
Minimize non-crypto code in harnessReduces noise that masks weak signals
Check assembly output (objdump -d)Verify compiler didn't introduce branches
Use -O3 -march=native in testingMatches production optimization levels

Common Mistakes

MistakeWhy It's WrongCorrect Approach
Only testing one input distributionMay miss leaks visible with other patternsTest fixed-vs-random, fixed-vs-fixed-different, etc.
Short dudect runs (< 1 minute)Insufficient measurements for weak signalsRun 5-10+ minutes, longer for high assurance
Ignoring compiler optimization levels-O0 may hide leaks present in -O3Test at production optimization level
Not testing on target architecturex86 vs ARM have different timing characteristicsTest on deployment platform
Marking too much as secret in TimecopFalse positives, unclear resultsMark only true secrets (keys, not public data)

Related Skills

Tool Skills

SkillPrimary Use in Constant-Time Analysis
dudectStatistical detection of timing differences via Welch's t-test
timecopDynamic tracing to pinpoint exact location of timing leaks

Technique Skills

SkillWhen to Apply
coverage-analysisEnsure test inputs exercise all code paths in crypto function
ci-integrationAutomate constant-time testing in continuous integration pipeline

Related Domain Skills

SkillRelationship
crypto-testingConstant-time analysis is essential component of cryptographic testing
fuzzingFuzzing crypto code may trigger timing-dependent paths

Skill Dependency Map

                    ┌─────────────────────────┐
                    │  constant-time-analysis │
                    │     (this skill)        │
                    └───────────┬─────────────┘
                                │
                ┌───────────────┴───────────────┐
                │                               │
                ▼                               ▼
    ┌───────────────────┐           ┌───────────────────┐
    │      dudect       │           │     timecop       │
    │  (statistical)    │           │    (dynamic)      │
    └────────┬──────────┘           └────────┬──────────┘
             │                               │
             └───────────────┬───────────────┘
                             │
                             ▼
              ┌──────────────────────────────┐
              │   Supporting Techniques      │
              │ coverage, CI integration     │
              └──────────────────────────────┘

Resources

Key External Resources

These results must be false: A usability evaluation of constant-time analysis tools Comprehensive usability study of constant-time analysis tools. Key findings: developers struggle with false positives, need better error messages, and benefit from tool integration. Evaluates FaCT, ct-verif, dudect, and Memsan across multiple cryptographic implementations. Recommends improved tooling UX and better documentation.

List of constant-time tools - CROCS Curated catalog of constant-time analysis tools with tutorials. Covers formal tools (ct-verif, FaCT), dynamic tools (Memsan, Timecop), symbolic tools (Binsec), and statistical tools (dudect). Includes practical tutorials for setup and usage.

Paul Kocher: Timing Attacks on Implementations of Diffie-Hellman, RSA, DSS, and Other Systems Original 1996 paper introducing timing attacks. Demonstrates attacks on modular exponentiation in RSA and Diffie-Hellman. Essential historical context for understanding timing vulnerabilities.

Remote Timing Attacks are Practical (Brumley & Boneh) Demonstrates practical remote timing attacks against OpenSSL. Shows network-level timing differences are sufficient to extract RSA keys. Proves timing attacks work in realistic network conditions.

Cache-timing attacks on AES Shows AES implementations using lookup tables are vulnerable to cache-timing attacks. Demonstrates practical attacks extracting AES keys via cache timing side channels.

KyberSlash: Division Timings Leak Secrets Recent discovery of timing vulnerabilities in Kyber (NIST post-quantum standard). Shows division operations leak secret coefficients. Highlights that constant-time issues persist even in modern post-quantum cryptography.

Video Resources

Coverage Analysis

Coverage analysis is essential for understanding which parts of your code are exercised during fuzzing. It helps identify fuzzing blockers like magic value checks and tracks the effectiveness of harness improvements over time.

Overview

Code coverage during fuzzing serves two critical purposes:

  1. Assessing harness effectiveness: Understand which parts of your application are actually executed by your fuzzing harnesses
  2. Tracking fuzzing progress: Monitor how coverage changes when updating harnesses, fuzzers, or the system under test (SUT)

Coverage is a proxy for fuzzer capability and performance. While coverage is not ideal for measuring fuzzer performance in absolute terms, it reliably indicates whether your harness works effectively in a given setup.

Key Concepts

ConceptDescription
Coverage instrumentationCompiler flags that track which code paths are executed
Corpus coverageCoverage achieved by running all test cases in a fuzzing corpus
Magic value checksHard-to-discover conditional checks that block fuzzer progress
Coverage-guided fuzzingFuzzing strategy that prioritizes inputs that discover new code paths
Coverage reportVisual or textual representation of executed vs. unexecuted code

When to Apply

Apply this technique when:

  • Starting a new fuzzing campaign to establish a baseline
  • Fuzzer appears to plateau without finding new paths
  • After harness modifications to verify improvements
  • When migrating between different fuzzers
  • Identifying areas requiring dictionary entries or seed inputs
  • Debugging why certain code paths aren't reached

Skip this technique when:

  • Fuzzing campaign is actively finding crashes
  • Coverage infrastructure isn't set up yet
  • Working with extremely large codebases where full coverage reports are impractical
  • Fuzzer's internal coverage metrics are sufficient for your needs

Quick Reference

TaskCommand/Pattern
LLVM coverage instrumentation (C/C++)-fprofile-instr-generate -fcoverage-mapping
GCC coverage instrumentation-ftest-coverage -fprofile-arcs
cargo-fuzz coverage (Rust)cargo +nightly fuzz coverage <target>
Generate LLVM profile datallvm-profdata merge -sparse file.profraw -o file.profdata
LLVM coverage reportllvm-cov report ./binary -instr-profile=file.profdata
LLVM HTML reportllvm-cov show ./binary -instr-profile=file.profdata -format=html -output-dir html/
gcovr HTML reportgcovr --html-details -o coverage.html

Ideal Coverage Workflow

The following workflow represents best practices for integrating coverage analysis into your fuzzing campaigns:

[Fuzzing Campaign]
       |
       v
[Generate Corpus]
       |
       v
[Coverage Analysis]
       |
       +---> Coverage Increased? --> Continue fuzzing with larger corpus
       |
       +---> Coverage Decreased? --> Fix harness or investigate SUT changes
       |
       +---> Coverage Plateaued? --> Add dictionary entries or seed inputs

Key principle: Use the corpus generated after each fuzzing campaign to calculate coverage, rather than real-time fuzzer statistics. This approach provides reproducible, comparable measurements across different fuzzing tools.

Step-by-Step

Step 1: Build with Coverage Instrumentation

Choose your instrumentation method based on toolchain:

LLVM/Clang (C/C++):

clang++ -fprofile-instr-generate -fcoverage-mapping \
  -O2 -DNO_MAIN \
  main.cc harness.cc execute-rt.cc -o fuzz_exec

GCC (C/C++):

g++ -ftest-coverage -fprofile-arcs \
  -O2 -DNO_MAIN \
  main.cc harness.cc execute-rt.cc -o fuzz_exec_gcov

Rust:

rustup toolchain install nightly --component llvm-tools-preview
cargo +nightly fuzz coverage fuzz_target_1

Step 2: Create Execution Runtime (C/C++ only)

For C/C++ projects, create a runtime that executes your corpus:

// execute-rt.cc
#include <stdio.h>
#include <stdlib.h>
#include <dirent.h>
#include <stdint.h>

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);

void load_file_and_test(const char *filename) {
    FILE *file = fopen(filename, "rb");
    if (file == NULL) {
        printf("Failed to open file: %s\n", filename);
        return;
    }

    fseek(file, 0, SEEK_END);
    long filesize = ftell(file);
    rewind(file);

    uint8_t *buffer = (uint8_t*) malloc(filesize);
    if (buffer == NULL) {
        printf("Failed to allocate memory for file: %s\n", filename);
        fclose(file);
        return;
    }

    long read_size = (long) fread(buffer, 1, filesize, file);
    if (read_size != filesize) {
        printf("Failed to read file: %s\n", filename);
        free(buffer);
        fclose(file);
        return;
    }

    LLVMFuzzerTestOneInput(buffer, filesize);

    free(buffer);
    fclose(file);
}

int main(int argc, char **argv) {
    if (argc != 2) {
        printf("Usage: %s <directory>\n", argv[0]);
        return 1;
    }

    DIR *dir = opendir(argv[1]);
    if (dir == NULL) {
        printf("Failed to open directory: %s\n", argv[1]);
        return 1;
    }

    struct dirent *entry;
    while ((entry = readdir(dir)) != NULL) {
        if (entry->d_type == DT_REG) {
            char filepath[1024];
            snprintf(filepath, sizeof(filepath), "%s/%s", argv[1], entry->d_name);
            load_file_and_test(filepath);
        }
    }

    closedir(dir);
    return 0;
}

Step 3: Execute on Corpus

LLVM (C/C++):

LLVM_PROFILE_FILE=fuzz.profraw ./fuzz_exec corpus/

GCC (C/C++):

./fuzz_exec_gcov corpus/

Rust: Coverage data is automatically generated when running cargo fuzz coverage.

Step 4: Process Coverage Data

LLVM:

# Merge raw profile data
llvm-profdata merge -sparse fuzz.profraw -o fuzz.profdata

# Generate text report
llvm-cov report ./fuzz_exec \
  -instr-profile=fuzz.profdata \
  -ignore-filename-regex='harness.cc|execute-rt.cc'

# Generate HTML report
llvm-cov show ./fuzz_exec \
  -instr-profile=fuzz.profdata \
  -ignore-filename-regex='harness.cc|execute-rt.cc' \
  -format=html -output-dir fuzz_html/

GCC with gcovr:

# Install gcovr (uv manages the isolated environment)
uv tool install gcovr
export PATH="$HOME/.local/bin:$PATH"   # uv's tool bin, if not already present

# Generate report
gcovr --gcov-executable "llvm-cov gcov" \
  --exclude harness.cc --exclude execute-rt.cc \
  --root . --html-details -o coverage.html

Rust:

# Install required tools
cargo install cargo-binutils rustfilt

# Create HTML generation script
cat <<'EOF' > ./generate_html
#!/bin/sh
if [ $# -lt 1 ]; then
    echo "Error: Name of fuzz target is required."
    echo "Usage: $0 fuzz_target [sources...]"
    exit 1
fi
FUZZ_TARGET="$1"
shift
SRC_FILTER="$@"
TARGET=$(rustc -vV | sed -n 's|host: ||p')
cargo +nightly cov -- show -Xdemangler=rustfilt \
  "target/$TARGET/coverage/$TARGET/release/$FUZZ_TARGET" \
  -instr-profile="fuzz/coverage/$FUZZ_TARGET/coverage.profdata" \
  -show-line-counts-or-regions -show-instantiations \
  -format=html -o fuzz_html/ $SRC_FILTER
EOF
chmod +x ./generate_html

# Generate HTML report
./generate_html fuzz_target_1 src/lib.rs

Step 5: Analyze Results

Review the coverage report to identify:

  • Uncovered code blocks: Areas that may need better seed inputs or dictionary entries
  • Magic value checks: Conditional statements with hardcoded values that block progress
  • Dead code: Functions that may not be reachable through your harness
  • Coverage changes: Compare against baseline to track improvements or regressions

Common Patterns

Pattern: Identifying Magic Values

Problem: Fuzzer cannot discover paths guarded by magic value checks.

Coverage reveals:

// Coverage shows this block is never executed
if (buf == 0x7F454C46) {  // ELF magic number
    // start parsing buf
}

Solution: Add magic values to dictionary file:

# magic.dict
"\x7F\x45\x4C\x46"

Pattern: Handling Crashing Inputs

Problem: Coverage generation fails when corpus contains crashing inputs.

Before:

./fuzz_exec corpus/  # Crashes on bad input, no coverage generated

After:

// Fork before executing to isolate crashes
int main(int argc, char **argv) {
    // ... directory opening code ...

    while ((entry = readdir(dir)) != NULL) {
        if (entry->d_type == DT_REG) {
            pid_t pid = fork();
            if (pid == 0) {
                // Child process - crash won't affect parent
                char filepath[1024];
                snprintf(filepath, sizeof(filepath), "%s/%s", argv[1], entry->d_name);
                load_file_and_test(filepath);
                exit(0);
            } else {
                // Parent waits for child
                waitpid(pid, NULL, 0);
            }
        }
    }
}

Pattern: CMake Integration

Use Case: Adding coverage builds to CMake projects.

project(FuzzingProject)
cmake_minimum_required(VERSION 3.0)

# Main binary
add_executable(program main.cc)

# Fuzzing binary
add_executable(fuzz main.cc harness.cc)
target_compile_definitions(fuzz PRIVATE NO_MAIN=1)
target_compile_options(fuzz PRIVATE -g -O2 -fsanitize=fuzzer)
target_link_libraries(fuzz -fsanitize=fuzzer)

# Coverage execution binary
add_executable(fuzz_exec main.cc harness.cc execute-rt.cc)
target_compile_definitions(fuzz_exec PRIVATE NO_MAIN)
target_compile_options(fuzz_exec PRIVATE -O2 -fprofile-instr-generate -fcoverage-mapping)
target_link_libraries(fuzz_exec -fprofile-instr-generate)

Build:

cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ .
cmake --build . --target fuzz_exec

Advanced Usage

Tips and Tricks

TipWhy It Helps
Use LLVM 18+ with -show-directory-coverageOrganizes large reports by directory structure instead of flat file list
Export to lcov format for better HTMLllvm-cov export -format=lcov + genhtml provides cleaner per-file reports
Compare coverage across campaignsStore .profdata files with timestamps to track progress over time
Filter harness code from reportsUse -ignore-filename-regex to focus on SUT coverage only
Automate coverage in CI/CDGenerate coverage reports automatically after scheduled fuzzing runs
Use gcovr 5.1+ for Clang 14+Older gcovr versions have compatibility issues with recent LLVM

Incremental Coverage Updates

GCC's gcov instrumentation incrementally updates .gcda files across multiple runs. This is useful for tracking coverage as you add test cases:

# First run
./fuzz_exec_gcov corpus_batch_1/
gcovr --html coverage_v1.html

# Second run (adds to existing coverage)
./fuzz_exec_gcov corpus_batch_2/
gcovr --html coverage_v2.html

# Start fresh
gcovr --delete  # Remove .gcda files
./fuzz_exec_gcov corpus/

Handling Large Codebases

For projects with hundreds of source files:

  1. Filter by prefix: Only generate reports for relevant directories

    llvm-cov show ./fuzz_exec -instr-profile=fuzz.profdata /path/to/src/
    
  2. Use directory coverage: Group by directory to reduce clutter (LLVM 18+)

    llvm-cov show -show-directory-coverage -format=html -output-dir html/
    
  3. Generate JSON for programmatic analysis:

    llvm-cov export -format=lcov > coverage.json
    

Differential Coverage

Compare coverage between two fuzzing campaigns:

# Campaign 1
LLVM_PROFILE_FILE=campaign1.profraw ./fuzz_exec corpus1/
llvm-profdata merge -sparse campaign1.profraw -o campaign1.profdata

# Campaign 2
LLVM_PROFILE_FILE=campaign2.profraw ./fuzz_exec corpus2/
llvm-profdata merge -sparse campaign2.profraw -o campaign2.profdata

# Compare
llvm-cov show ./fuzz_exec \
  -instr-profile=campaign2.profdata \
  -instr-profile=campaign1.profdata \
  -show-line-counts-or-regions

Anti-Patterns

Anti-PatternProblemCorrect Approach
Using fuzzer-reported coverage for comparisonsDifferent fuzzers calculate coverage differently, making cross-tool comparison meaninglessUse dedicated coverage tools (llvm-cov, gcovr) for reproducible measurements
Generating coverage with optimizations-O3 optimizations can eliminate code, making coverage misleadingUse -O2 or -O0 for coverage builds
Not filtering harness codeHarness coverage inflates numbers and obscures SUT coverageUse -ignore-filename-regex or --exclude to filter harness files
Mixing LLVM and GCC instrumentationIncompatible formats cause parsing failuresStick to one toolchain for coverage builds
Ignoring crashing inputsCrashes prevent coverage generation, hiding real coverage dataFix crashes first, or use process forking to isolate them
Not tracking coverage over timeOne-time coverage checks miss regressions and improvementsStore coverage data with timestamps and track trends

Tool-Specific Guidance

libFuzzer

libFuzzer uses LLVM's SanitizerCoverage by default for guiding fuzzing, but you need separate instrumentation for generating reports.

Build for coverage:

clang++ -fprofile-instr-generate -fcoverage-mapping \
  -O2 -DNO_MAIN \
  main.cc harness.cc execute-rt.cc -o fuzz_exec

Execute corpus and generate report:

LLVM_PROFILE_FILE=fuzz.profraw ./fuzz_exec corpus/
llvm-profdata merge -sparse fuzz.profraw -o fuzz.profdata
llvm-cov show ./fuzz_exec -instr-profile=fuzz.profdata -format=html -output-dir html/

Integration tips:

  • Don't use -fsanitize=fuzzer for coverage builds (it conflicts with profile instrumentation)
  • Reuse the same harness function (LLVMFuzzerTestOneInput) with a different main function
  • Use the -ignore-filename-regex flag to exclude harness code from coverage reports
  • Consider using llvm-cov's -show-instantiation flag for template-heavy C++ code

AFL++

AFL++ provides its own coverage feedback mechanism, but for detailed reports use standard LLVM/GCC tools.

Build for coverage with LLVM:

clang++ -fprofile-instr-generate -fcoverage-mapping \
  -O2 main.cc harness.cc execute-rt.cc -o fuzz_exec

Build for coverage with GCC:

AFL_USE_ASAN=0 afl-gcc -ftest-coverage -fprofile-arcs \
  main.cc harness.cc execute-rt.cc -o fuzz_exec_gcov

Execute and generate report:

# LLVM approach
LLVM_PROFILE_FILE=fuzz.profraw ./fuzz_exec afl_output/queue/
llvm-profdata merge -sparse fuzz.profraw -o fuzz.profdata
llvm-cov report ./fuzz_exec -instr-profile=fuzz.profdata

# GCC approach
./fuzz_exec_gcov afl_output/queue/
gcovr --html-details -o coverage.html

Integration tips:

  • Don't use AFL++'s instrumentation (afl-clang-fast) for coverage builds
  • Use standard compilers with coverage flags instead
  • AFL++'s queue/ directory contains your corpus
  • AFL++'s built-in coverage statistics are useful for real-time monitoring but not for detailed analysis

cargo-fuzz (Rust)

cargo-fuzz provides built-in coverage generation using LLVM tools.

Install prerequisites:

rustup toolchain install nightly --component llvm-tools-preview
cargo install cargo-binutils rustfilt

Generate coverage data:

cargo +nightly fuzz coverage fuzz_target_1

Create HTML report script:

cat <<'EOF' > ./generate_html
#!/bin/sh
FUZZ_TARGET="$1"
shift
SRC_FILTER="$@"
TARGET=$(rustc -vV | sed -n 's|host: ||p')
cargo +nightly cov -- show -Xdemangler=rustfilt \
  "target/$TARGET/coverage/$TARGET/release/$FUZZ_TARGET" \
  -instr-profile="fuzz/coverage/$FUZZ_TARGET/coverage.profdata" \
  -show-line-counts-or-regions -show-instantiations \
  -format=html -o fuzz_html/ $SRC_FILTER
EOF
chmod +x ./generate_html

Generate report:

./generate_html fuzz_target_1 src/lib.rs

Integration tips:

  • Always use the nightly toolchain for coverage
  • The -Xdemangler=rustfilt flag makes function names readable
  • Filter by source files (e.g., src/lib.rs) to focus on crate code
  • Use -show-line-counts-or-regions and -show-instantiations for better Rust-specific output
  • Corpus is located in fuzz/corpus/<target>/

honggfuzz

honggfuzz works with standard LLVM/GCC coverage instrumentation.

Build for coverage:

# Use standard compiler, not honggfuzz compiler
clang -fprofile-instr-generate -fcoverage-mapping \
  -O2 harness.c execute-rt.c -o fuzz_exec

Execute corpus:

LLVM_PROFILE_FILE=fuzz.profraw ./fuzz_exec honggfuzz_workspace/

Integration tips:

  • Don't use hfuzz-clang for coverage builds
  • honggfuzz corpus is typically in a workspace directory
  • Use the same LLVM workflow as libFuzzer

Troubleshooting

IssueCauseSolution
error: no profile data availableProfile wasn't generated or wrong pathVerify LLVM_PROFILE_FILE was set and .profraw file exists
Failed to load coverageMismatch between binary and profile dataRebuild binary with same flags used during execution
Coverage reports show 0%Wrong binary used for report generationUse the instrumented binary, not the fuzzing binary
no_working_dir_found error (gcovr).gcda files in unexpected locationAdd --gcov-ignore-errors=no_working_dir_found flag
Crashes prevent coverage generationCorpus contains crashing inputsFilter crashes or use forking approach to isolate failures
Coverage decreases after harness changeHarness now skips certain code pathsReview harness logic; may need to support more input formats
HTML report is flat file listUsing older LLVM versionUpgrade to LLVM 18+ and use -show-directory-coverage
incompatible instrumentationMixing LLVM and GCC coverageRebuild everything with same toolchain

Related Skills

Tools That Use This Technique

SkillHow It Applies
libfuzzerUses SanitizerCoverage for feedback; coverage analysis evaluates harness effectiveness
aflppUses edge coverage for feedback; detailed analysis requires separate instrumentation
cargo-fuzzBuilt-in cargo fuzz coverage command for Rust projects
honggfuzzUses edge coverage; analyze with standard LLVM/GCC tools

Related Techniques

SkillRelationship
fuzz-harness-writingCoverage reveals which code paths harness reaches; guides harness improvements
fuzzing-dictionariesCoverage identifies magic value checks that need dictionary entries
corpus-managementCoverage analysis helps curate corpora by identifying redundant test cases
sanitizersCoverage helps verify sanitizer-instrumented code is actually executed

Resources

Key External Resources

LLVM Source-Based Code Coverage Comprehensive guide to LLVM's profile instrumentation, including advanced features like branch coverage, region coverage, and integration with existing build systems. Covers compiler flags, runtime behavior, and profile data formats.

llvm-cov Command Guide Detailed CLI reference for llvm-cov commands including show, report, and export. Documents all filtering options, output formats, and integration with llvm-profdata.

gcovr Documentation Complete guide to gcovr tool for generating coverage reports from gcov data. Covers HTML themes, filtering options, multi-directory projects, and CI/CD integration patterns.

SanitizerCoverage Documentation Low-level documentation for LLVM's SanitizerCoverage instrumentation. Explains inline 8-bit counters, PC tables, and how fuzzers use coverage feedback for guidance.

On the Evaluation of Fuzzer Performance Research paper examining limitations of coverage as a fuzzing performance metric. Argues for more nuanced evaluation methods beyond simple code coverage percentages.

Video Resources

Not applicable - coverage analysis is primarily a tooling and workflow topic best learned through documentation and hands-on practice.

Fuzzing Dictionary

A fuzzing dictionary provides domain-specific tokens to guide the fuzzer toward interesting inputs. Instead of purely random mutations, the fuzzer incorporates known keywords, magic numbers, protocol commands, and format-specific strings that are more likely to reach deeper code paths in parsers, protocol handlers, and file format processors.

Overview

Dictionaries are text files containing quoted strings that represent meaningful tokens for your target. They help fuzzers bypass early validation checks and explore code paths that would be difficult to reach through blind mutation alone.

Key Concepts

ConceptDescription
Dictionary EntryA quoted string (e.g., "keyword") or key-value pair (e.g., kw="value")
Hex EscapesByte sequences like "\xF7\xF8" for non-printable characters
Token InjectionFuzzer inserts dictionary entries into generated inputs
Cross-Fuzzer FormatDictionary files work with libFuzzer, AFL++, and cargo-fuzz

When to Apply

Apply this technique when:

  • Fuzzing parsers (JSON, XML, config files)
  • Fuzzing protocol implementations (HTTP, DNS, custom protocols)
  • Fuzzing file format handlers (PNG, PDF, media codecs)
  • Coverage plateaus early without reaching deeper logic
  • Target code checks for specific keywords or magic values

Skip this technique when:

  • Fuzzing pure algorithms without format expectations
  • Target has no keyword-based parsing
  • Corpus already achieves high coverage

Quick Reference

TaskCommand/Pattern
Use with libFuzzer./fuzz -dict=./dictionary.dict ...
Use with AFL++afl-fuzz -x ./dictionary.dict ...
Use with cargo-fuzzcargo fuzz run fuzz_target -- -dict=./dictionary.dict
Extract from headergrep -o '".*"' header.h > header.dict
Generate from binarystrings ./binary | sed 's/^/"&/; s/$/&"/' > strings.dict

Step-by-Step

Step 1: Create Dictionary File

Create a text file with quoted strings on each line. Use comments (#) for documentation.

Example dictionary format:

# Lines starting with '#' and empty lines are ignored.

# Adds "blah" (w/o quotes) to the dictionary.
kw1="blah"
# Use \\ for backslash and \" for quotes.
kw2="\"ac\\dc\""
# Use \xAB for hex values
kw3="\xF7\xF8"
# the name of the keyword followed by '=' may be omitted:
"foo\x0Abar"

Step 2: Generate Dictionary Content

Choose a generation method based on what's available:

From LLM: Prompt ChatGPT or Claude with:

A dictionary can be used to guide the fuzzer. Write me a dictionary file for fuzzing a <PNG parser>. Each line should be a quoted string or key-value pair like kw="value". Include magic bytes, chunk types, and common header values. Use hex escapes like "\xF7\xF8" for binary values.

From header files:

grep -o '".*"' header.h > header.dict

From man pages (for CLI tools):

man curl | grep -oP '^\s*(--|-)\K\S+' | sed 's/[,.]$//' | sed 's/^/"&/; s/$/&"/' | sort -u > man.dict

From binary strings:

strings ./binary | sed 's/^/"&/; s/$/&"/' > strings.dict

Step 3: Pass Dictionary to Fuzzer

Use the appropriate flag for your fuzzer (see Quick Reference above).

Common Patterns

Pattern: Protocol Keywords

Use Case: Fuzzing HTTP or custom protocol handlers

Dictionary content:

# HTTP methods
"GET"
"POST"
"PUT"
"DELETE"
"HEAD"

# Headers
"Content-Type"
"Authorization"
"Host"

# Protocol markers
"HTTP/1.1"
"HTTP/2.0"

Pattern: Magic Bytes and File Format Headers

Use Case: Fuzzing image parsers, media decoders, archive handlers

Dictionary content:

# PNG magic bytes and chunks
png_magic="\x89PNG\r\n\x1a\n"
ihdr="IHDR"
plte="PLTE"
idat="IDAT"
iend="IEND"

# JPEG markers
jpeg_soi="\xFF\xD8"
jpeg_eoi="\xFF\xD9"

Pattern: Configuration File Keywords

Use Case: Fuzzing config file parsers (YAML, TOML, INI)

Dictionary content:

# Common config keywords
"true"
"false"
"null"
"version"
"enabled"
"disabled"

# Section headers
"[general]"
"[network]"
"[security]"

Advanced Usage

Tips and Tricks

TipWhy It Helps
Combine multiple generation methodsLLM-generated keywords + strings from binary covers broad surface
Include boundary values"0", "-1", "2147483647" trigger edge cases
Add format delimiters:, =, {, } help fuzzer construct valid structures
Keep dictionaries focused50-200 entries perform better than thousands
Test dictionary effectivenessRun with and without dict, compare coverage

Auto-Generated Dictionaries (AFL++)

When using afl-clang-lto compiler, AFL++ automatically extracts dictionary entries from string comparisons in the binary. This happens at compile time via the AUTODICTIONARY feature.

Enable auto-dictionary:

export AFL_LLVM_DICT2FILE=auto.dict
afl-clang-lto++ target.cc -o target
# Dictionary saved to auto.dict
afl-fuzz -x auto.dict -i in -o out -- ./target

Combining Multiple Dictionaries

Some fuzzers support multiple dictionary files:

# AFL++ with multiple dictionaries
afl-fuzz -x keywords.dict -x formats.dict -i in -o out -- ./target

Anti-Patterns

Anti-PatternProblemCorrect Approach
Including full sentencesFuzzer needs atomic tokens, not proseBreak into individual keywords
Duplicating entriesWastes mutation budgetUse sort -u to deduplicate
Over-sized dictionariesSlows fuzzer, dilutes useful tokensKeep focused: 50-200 most relevant entries
Missing hex escapesNon-printable bytes become mangledUse \xXX for binary values
No commentsHard to maintain and auditDocument sections with # comments

Tool-Specific Guidance

libFuzzer

clang++ -fsanitize=fuzzer,address harness.cc -o fuzz
./fuzz -dict=./dictionary.dict corpus/

Integration tips:

  • Dictionary tokens are inserted/replaced during mutations
  • Combine with -max_len to control input size
  • Use -print_final_stats=1 to see dictionary effectiveness metrics
  • Dictionary entries longer than -max_len are ignored

AFL++

afl-fuzz -x ./dictionary.dict -i input/ -o output/ -- ./target @@

Integration tips:

  • AFL++ supports multiple -x flags for multiple dictionaries
  • Use AFL_LLVM_DICT2FILE with afl-clang-lto for auto-generated dictionaries
  • Dictionary effectiveness shown in fuzzer stats UI
  • Tokens are used during deterministic and havoc stages

cargo-fuzz (Rust)

cargo fuzz run fuzz_target -- -dict=./dictionary.dict

Integration tips:

  • cargo-fuzz uses libFuzzer backend, so all libFuzzer dict flags work
  • Place dictionary file in fuzz/ directory alongside harness
  • Reference from harness directory: cargo fuzz run target -- -dict=../dictionary.dict

go-fuzz (Go)

go-fuzz does not have built-in dictionary support, but you can manually seed the corpus with dictionary entries:

# Convert dictionary to corpus files
grep -o '".*"' dict.txt | while read line; do
    echo -n "$line" | base64 > corpus/$(echo "$line" | md5sum | cut -d' ' -f1)
done

go-fuzz -bin=./target-fuzz.zip -workdir=.

Troubleshooting

IssueCauseSolution
Dictionary file not loadedWrong path or format errorCheck fuzzer output for dict parsing errors; verify file format
No coverage improvementDictionary tokens not relevantAnalyze target code for actual keywords; try different generation method
Syntax errors in dict fileUnescaped quotes or invalid escapesUse \\ for backslash, \" for quotes; validate with test run
Fuzzer ignores long entriesEntries exceed -max_lenKeep entries under max input length, or increase -max_len
Too many entries slow fuzzerDictionary too largePrune to 50-200 most relevant entries

Related Skills

Tools That Use This Technique

SkillHow It Applies
libfuzzerNative dictionary support via -dict= flag
aflppNative dictionary support via -x flag; auto-generation with AUTODICTIONARIES
cargo-fuzzUses libFuzzer backend, inherits -dict= support

Related Techniques

SkillRelationship
fuzzing-corpusDictionaries complement corpus: corpus provides structure, dictionary provides keywords
coverage-analysisUse coverage data to validate dictionary effectiveness
harness-writingHarness structure determines which dictionary tokens are useful

Resources

Key External Resources

AFL++ Dictionaries Pre-built dictionaries for common formats (HTML, XML, JSON, SQL, etc.). Good starting point for format-specific fuzzing.

libFuzzer Dictionary Documentation Official libFuzzer documentation on dictionary format and usage. Explains token insertion strategy and performance implications.

Additional Examples

OSS-Fuzz Dictionaries Real-world dictionaries from Google's continuous fuzzing service. Search project directories for *.dict files to see production examples.

Overcoming Fuzzing Obstacles

Codebases often contain anti-fuzzing patterns that prevent effective coverage. Checksums, global state (like time-seeded PRNGs), and validation checks can block the fuzzer from exploring deeper code paths. This technique shows how to patch your System Under Test (SUT) to bypass these obstacles during fuzzing while preserving production behavior.

Overview

Many real-world programs were not designed with fuzzing in mind. They may:

  • Verify checksums or cryptographic hashes before processing input
  • Rely on global state (e.g., system time, environment variables)
  • Use non-deterministic random number generators
  • Perform complex validation that makes it difficult for the fuzzer to generate valid inputs

These patterns make fuzzing difficult because:

  1. Checksums: The fuzzer must guess correct hash values (astronomically unlikely)
  2. Global state: Same input produces different behavior across runs (breaks determinism)
  3. Complex validation: The fuzzer spends effort hitting validation failures instead of exploring deeper code

The solution is conditional compilation: modify code behavior during fuzzing builds while keeping production code unchanged.

Key Concepts

ConceptDescription
SUT PatchingModifying System Under Test to be fuzzing-friendly
Conditional CompilationCode that behaves differently based on compile-time flags
Fuzzing Build ModeSpecial build configuration that enables fuzzing-specific patches
False PositivesCrashes found during fuzzing that cannot occur in production
DeterminismSame input always produces same behavior (critical for fuzzing)

When to Apply

Apply this technique when:

  • The fuzzer gets stuck at checksum or hash verification
  • Coverage reports show large blocks of unreachable code behind validation
  • Code uses time-based seeds or other non-deterministic global state
  • Complex validation makes it nearly impossible to generate valid inputs
  • You see the fuzzer repeatedly hitting the same validation failures

Skip this technique when:

  • The obstacle can be overcome with a good seed corpus or dictionary
  • The validation is simple enough for the fuzzer to learn (e.g., magic bytes)
  • You're doing grammar-based or structure-aware fuzzing that handles validation
  • Skipping the check would introduce too many false positives
  • The code is already fuzzing-friendly

Quick Reference

TaskC/C++Rust
Check if fuzzing build#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTIONcfg!(fuzzing)
Skip check during fuzzing#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION return -1; #endifif !cfg!(fuzzing) { return Err(...) }
Common obstaclesChecksums, PRNGs, time-based logicChecksums, PRNGs, time-based logic
Supported fuzzerslibFuzzer, AFL++, LibAFL, honggfuzzcargo-fuzz, libFuzzer

Step-by-Step

Step 1: Identify the Obstacle

Run the fuzzer and analyze coverage to find code that's unreachable. Common patterns:

  1. Look for checksum/hash verification before deeper processing
  2. Check for calls to rand(), time(), or srand() with system seeds
  3. Find validation functions that reject most inputs
  4. Identify global state initialization that differs across runs

Tools to help:

  • Coverage reports (see coverage-analysis technique)
  • Profiling with -fprofile-instr-generate
  • Manual code inspection of entry points

Step 2: Add Conditional Compilation

Modify the obstacle to bypass it during fuzzing builds.

C/C++ Example:

// Before: Hard obstacle
if (checksum != expected_hash) {
    return -1;  // Fuzzer never gets past here
}

// After: Conditional bypass
if (checksum != expected_hash) {
#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
    return -1;  // Only enforced in production
#endif
}
// Fuzzer can now explore code beyond this check

Rust Example:

// Before: Hard obstacle
if checksum != expected_hash {
    return Err(MyError::Hash);  // Fuzzer never gets past here
}

// After: Conditional bypass
if checksum != expected_hash {
    if !cfg!(fuzzing) {
        return Err(MyError::Hash);  // Only enforced in production
    }
}
// Fuzzer can now explore code beyond this check

Step 3: Verify Coverage Improvement

After patching:

  1. Rebuild with fuzzing instrumentation
  2. Run the fuzzer for a short time
  3. Compare coverage to the unpatched version
  4. Confirm new code paths are being explored

Step 4: Assess False Positive Risk

Consider whether skipping the check introduces impossible program states:

  • Does code after the check assume validated properties?
  • Could skipping validation cause crashes that cannot occur in production?
  • Is there implicit state dependency?

If false positives are likely, consider a more targeted patch (see Common Patterns below).

Common Patterns

Pattern: Bypass Checksum Validation

Use Case: Hash/checksum blocks all fuzzer progress

Before:

uint32_t computed = hash_function(data, size);
if (computed != expected_checksum) {
    return ERROR_INVALID_HASH;
}
process_data(data, size);

After:

uint32_t computed = hash_function(data, size);
if (computed != expected_checksum) {
#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
    return ERROR_INVALID_HASH;
#endif
}
process_data(data, size);

False positive risk: LOW - If data processing doesn't depend on checksum correctness

Pattern: Deterministic PRNG Seeding

Use Case: Non-deterministic random state prevents reproducibility

Before:

void initialize() {
    srand(time(NULL));  // Different seed each run
}

After:

void initialize() {
#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
    srand(12345);  // Fixed seed for fuzzing
#else
    srand(time(NULL));
#endif
}

False positive risk: LOW - Fuzzer can explore all code paths with fixed seed

Pattern: Careful Validation Skip

Use Case: Validation must be skipped but downstream code has assumptions

Before (Dangerous):

#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
if (!validate_config(&config)) {
    return -1;  // Ensures config.x != 0
}
#endif

int32_t result = 100 / config.x;  // CRASH: Division by zero in fuzzing!

After (Safe):

#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
if (!validate_config(&config)) {
    return -1;
}
#else
// During fuzzing, use safe defaults for failed validation
if (!validate_config(&config)) {
    config.x = 1;  // Prevent division by zero
    config.y = 1;
}
#endif

int32_t result = 100 / config.x;  // Safe in both builds

False positive risk: MITIGATED - Provides safe defaults instead of skipping

Pattern: Bypass Complex Format Validation

Use Case: Multi-step validation makes valid input generation nearly impossible

Rust Example:

// Before: Multiple validation stages
pub fn parse_message(data: &[u8]) -> Result<Message, Error> {
    validate_magic_bytes(data)?;
    validate_structure(data)?;
    validate_checksums(data)?;
    validate_crypto_signature(data)?;

    deserialize_message(data)
}

// After: Skip expensive validation during fuzzing
pub fn parse_message(data: &[u8]) -> Result<Message, Error> {
    validate_magic_bytes(data)?;  // Keep cheap checks

    if !cfg!(fuzzing) {
        validate_structure(data)?;
        validate_checksums(data)?;
        validate_crypto_signature(data)?;
    }

    deserialize_message(data)
}

False positive risk: MEDIUM - Deserialization must handle malformed data gracefully

Advanced Usage

Tips and Tricks

TipWhy It Helps
Keep cheap validationMagic bytes and size checks guide fuzzer without much cost
Use fixed seeds for PRNGsMakes behavior deterministic while exploring all code paths
Patch incrementallySkip one obstacle at a time and measure coverage impact
Add defensive defaultsWhen skipping validation, provide safe fallback values
Document all patchesFuture maintainers need to understand fuzzing vs. production differences

Real-World Examples

OpenSSL: Uses FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION to modify cryptographic algorithm behavior. For example, in crypto/cmp/cmp_vfy.c, certain signature checks are relaxed during fuzzing to allow deeper exploration of certificate validation logic.

ogg crate (Rust): Uses cfg!(fuzzing) to skip checksum verification during fuzzing. This allows the fuzzer to explore audio processing code without spending effort guessing correct checksums.

Measuring Patch Effectiveness

After applying patches, quantify the improvement:

  1. Line coverage: Use llvm-cov or cargo-cov to see new reachable lines
  2. Basic block coverage: More fine-grained than line coverage
  3. Function coverage: How many more functions are now reachable?
  4. Corpus size: Does the fuzzer generate more diverse inputs?

Effective patches typically increase coverage by 10-50% or more.

Combining with Other Techniques

Obstacle patching works well with:

  • Corpus seeding: Provide valid inputs that get past initial parsing
  • Dictionaries: Help fuzzer learn magic bytes and common values
  • Structure-aware fuzzing: Use protobuf or grammar definitions for complex formats
  • Harness improvements: Better harness can sometimes avoid obstacles entirely

Anti-Patterns

Anti-PatternProblemCorrect Approach
Skip all validation wholesaleCreates false positives and unstable fuzzingSkip only specific obstacles that block coverage
No risk assessmentFalse positives waste time and hide real bugsAnalyze downstream code for assumptions
Forget to document patchesFuture maintainers don't understand the differencesAdd comments explaining why patch is safe
Patch without measuringDon't know if it helpedCompare coverage before and after
Over-patchingMakes fuzzing build diverge too much from productionMinimize differences between builds

Tool-Specific Guidance

libFuzzer

libFuzzer automatically defines FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION during compilation.

# C++ compilation
clang++ -g -fsanitize=fuzzer,address -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION \
    harness.cc target.cc -o fuzzer

# The macro is usually defined automatically by -fsanitize=fuzzer
clang++ -g -fsanitize=fuzzer,address harness.cc target.cc -o fuzzer

Integration tips:

  • The macro is defined automatically; manual definition is usually unnecessary
  • Use #ifdef to check for the macro
  • Combine with sanitizers to detect bugs in newly reachable code

AFL++

AFL++ also defines FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION when using its compiler wrappers.

# Compilation with AFL++ wrappers
afl-clang-fast++ -g -fsanitize=address target.cc harness.cc -o fuzzer

# The macro is defined automatically by afl-clang-fast

Integration tips:

  • Use afl-clang-fast or afl-clang-lto for automatic macro definition
  • Persistent mode harnesses benefit most from obstacle patching
  • Consider using AFL_LLVM_LAF_ALL for additional input-to-state transformations

honggfuzz

honggfuzz also supports the macro when building targets.

# Compilation
hfuzz-clang++ -g -fsanitize=address target.cc harness.cc -o fuzzer

Integration tips:

  • Use hfuzz-clang or hfuzz-clang++ wrappers
  • The macro is available for conditional compilation
  • Combine with honggfuzz's feedback-driven fuzzing

cargo-fuzz (Rust)

cargo-fuzz automatically sets the fuzzing cfg option during builds.

# Build fuzz target (cfg!(fuzzing) is automatically set)
cargo fuzz build fuzz_target_name

# Run fuzz target
cargo fuzz run fuzz_target_name

Integration tips:

  • Use cfg!(fuzzing) for runtime checks in production builds
  • Use #[cfg(fuzzing)] for compile-time conditional compilation
  • The fuzzing cfg is only set during cargo fuzz builds, not regular cargo build
  • Can be manually enabled with RUSTFLAGS="--cfg fuzzing" for testing

LibAFL

LibAFL supports the C/C++ macro for targets written in C/C++.

# Compilation
clang++ -g -fsanitize=address -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION \
    target.cc -c -o target.o

Integration tips:

  • Define the macro manually or use compiler flags
  • Works the same as with libFuzzer
  • Useful when building custom LibAFL-based fuzzers

Troubleshooting

IssueCauseSolution
Coverage doesn't improve after patchingWrong obstacle identifiedProfile execution to find actual bottleneck
Many false positive crashesDownstream code has assumptionsAdd defensive defaults or partial validation
Code compiles differentlyMacro not defined in all build configsVerify macro in all source files and dependencies
Fuzzer finds bugs in patched codePatch introduced invalid statesReview patch for state invariants; consider safer approach
Can't reproduce production bugsBuild differences too largeMinimize patches; keep validation for state-critical checks

Related Skills

Tools That Use This Technique

SkillHow It Applies
libfuzzerDefines FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION automatically
aflppSupports the macro via compiler wrappers
honggfuzzUses the macro for conditional compilation
cargo-fuzzSets cfg!(fuzzing) for Rust conditional compilation

Related Techniques

SkillRelationship
fuzz-harness-writingBetter harnesses may avoid obstacles; patching enables deeper exploration
coverage-analysisUse coverage to identify obstacles and measure patch effectiveness
corpus-seedingSeed corpus can help overcome obstacles without patching
dictionary-generationDictionaries help with magic bytes but not checksums or complex validation

Resources

Key External Resources

OpenSSL Fuzzing Documentation OpenSSL's fuzzing infrastructure demonstrates large-scale use of FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION. The project uses this macro to modify cryptographic validation, certificate parsing, and other security-critical code paths to enable deeper fuzzing while maintaining production correctness.

LibFuzzer Documentation on Flags Official LLVM documentation for libFuzzer, including how the fuzzer defines compiler macros and how to use them effectively. Covers integration with sanitizers and coverage instrumentation.

Rust cfg Attribute Reference Complete reference for Rust conditional compilation, including cfg!(fuzzing) and cfg!(test). Explains compile-time vs. runtime conditional compilation and best practices.

Genotoxic

Combines mutation testing and necessist (test statement removal) with code graph analysis to triage findings into actionable categories: false positives, missing unit tests, and fuzzing targets.

When to Use

  • After mutation testing reveals survived mutants that need triage
  • Identifying where unit tests would have the highest impact
  • Finding functions that need fuzz harnesses instead of unit tests
  • Prioritizing test improvements using data flow context
  • Filtering out harmless mutants from actionable ones
  • Finding unnecessary test statements that indicate weak assertions (necessist)

When NOT to Use

  • Codebase has no existing test suite (write tests first)
  • Pure documentation or configuration changes
  • Single-file scripts with trivial logic

Prerequisites

  • trailmark installed — if uv run trailmark fails, run:
    uv tool install trailmark
    

Python snippets: uv run --with trailmark python - (a tool env is not importable)

**DO NOT** fall back to "manual verification" or "manual analysis"
as a substitute for running trailmark. Install it first. If installation
fails, report the error instead of switching to manual analysis.
- A **mutation testing framework** for the target language — if the framework
command fails (not found, not installed), install it using the instructions
in [references/mutation-frameworks.md](references/mutation-frameworks.md).
**DO NOT** fall back to "manual mutation analysis" or skip mutation testing.
Install the framework first. If installation fails, report the error
instead of switching to manual mutation analysis.
- **necessist** (optional, recommended) — if the target language is
supported (Go, Rust, Solidity/Foundry, TypeScript/Hardhat,
TypeScript/Vitest, Rust/Anchor), install with `cargo install necessist`.
See [references/mutation-frameworks.md](references/mutation-frameworks.md)
for details.
- An existing test suite that passes
- **macOS environment**: Run `ulimit -n 1024` before any `mull-runner`
invocation. macOS Tahoe (26+) sets unlimited file descriptors by
default, which crashes Mull's subprocess spawning. See
[references/mutation-frameworks.md](references/mutation-frameworks.md)
for details.

---

## Rationalizations to Reject

| Rationalization | Why It's Wrong | Required Action |
|-----------------|----------------|-----------------|
| "All survived mutants need tests" | Many are harmless or equivalent | Triage before writing tests |
| "Mutation testing is too noisy" | Noise means you're not triaging | Use graph data to filter |
| "Unit tests cover everything" | Complex data flows need fuzzing | Check entrypoint reachability |
| "Dead code mutants don't matter" | Dead code should be removed | Flag for cleanup |
| "Low complexity = low risk" | Boundary bugs hide in simple code | Check mutant location |
| "Tool isn't installed, I'll do it manually" | Manual analysis misses what tooling catches | Install the tool first |
| "Necessist isn't mutation testing, skip it" | Necessist finds what mutation testing misses: weak tests | Run both when the language supports it |

---

## Quick Start

```bash
# 1. Build the code graph
uv run trailmark analyze --language auto --summary {targetDir}

# 2. Run mutation testing (language-dependent)
# Python:
uv run mutmut run --paths-to-mutate {targetDir}/src
uv run mutmut results

# 2b. Run necessist (if language supported)
necessist

# 3. Analyze results with this skill's workflow (Phase 3)

Workflow Overview

Phase 1: Graph Build      → Parse codebase with trailmark
      ↓
Phase 2: Mutation Run     → Execute mutation testing framework
Phase 2b: Necessist Run   → Remove test statements (optional, parallel)
      ↓
Phase 3: Triage           → Classify findings using graph data
      ↓
Output: Categorized Report
  ├── Corroborated         (both tools flag same function — highest value)
  ├── False Positives      (harmless, skip)
  ├── Missing Tests        (write unit tests)
  └── Fuzzing Targets      (set up fuzz harnesses)

Decision Tree

├─ Need to set up mutation testing for a language?
│  └─ Read: references/mutation-frameworks.md
│
├─ Need to set up necessist or find weak test statements?
│  └─ Read: references/mutation-frameworks.md (Necessist section)
│
├─ Need to understand the triage criteria in depth?
│  └─ Read: references/triage-methodology.md
│
├─ Need to understand how graph data informs triage?
│  └─ Read: references/graph-analysis.md
│
└─ Already have results + graph? Use Phase 3 below.

Phase 1: Build Code Graph and Run Pre-Analysis

Parse the target codebase with trailmark and run pre-analysis before mutation testing. Pre-analysis computes blast radius, entry points, privilege boundaries, and taint propagation, which Phase 3 uses for triage.

uv run trailmark analyze --language auto --summary {targetDir}

Use the QueryEngine API to build the graph and run pre-analysis:

  1. QueryEngine.from_directory("{targetDir}", language="auto")
  2. Call engine.preanalysis() — mandatory before triage
  3. Export with engine.to_json() for cross-referencing with mutation results

If auto-detection is wrong for the target, rerun with an explicit language or comma-separated list such as python,rust.

See references/graph-analysis.md for the full API: node mapping, reachability queries, blast radius, and pre-analysis subgraph lookups.


Phase 2: Run Mutation Testing

Select and run the appropriate framework. See references/mutation-frameworks.md for language-specific setup.

Capture survived mutants. Each framework reports differently, but extract these fields per mutant:

FieldDescription
File pathSource file containing the mutant
Line numberLine where mutation was applied
Mutation typeWhat was changed (operator, value, etc.)
Statussurvived, killed, timeout, error

Filter to survived mutants only for Phase 3.


Phase 2b: Run Necessist (Optional)

If the target language is supported (Go, Rust, Solidity/Foundry, TypeScript/Hardhat, TypeScript/Vitest, Rust/Anchor), run necessist to find unnecessary test statements. This runs independently of Phase 2 and can execute in parallel.

# Auto-detect framework
necessist

# Or target specific test files
necessist tests/test_parser.rs

# Export results
necessist --dump

Filter to findings where the test passed after removal. See references/mutation-frameworks.md for framework-specific configuration and the normalized record format.

Map each removal to a production function using the algorithm in references/graph-analysis.md.


Phase 3: Triage Findings

For each survived mutant and each necessist removal, determine its triage bucket using graph data. Necessist removals must first be mapped to a production function (see references/graph-analysis.md).

Quick Classification (Mutation Testing)

SignalBucketReasoning
No callers in graphFalse PositiveDead code, mutant is unreachable
Only test callersFalse PositiveTest infrastructure, not production
Logging/display stringFalse PositiveCosmetic, no behavioral impact
Equivalent mutantFalse PositiveBehavior unchanged despite mutation
Simple function, low CC, no entrypoint pathMissing TestsUnit test is straightforward
Error handling pathMissing TestsShould have negative test cases
Boundary condition (off-by-one)Missing TestsProperty-based test candidate
Pure function, deterministicMissing TestsEasy to test, high value
High CC (>10), entrypoint reachableFuzzing TargetComplex + exposed = fuzz it
Parser/validator/deserializerFuzzing TargetStructured input handling
Many callers (>10) + moderate CCFuzzing TargetHigh blast radius
Binary/wire protocol handlingFuzzing TargetFuzzers excel at format testing

Quick Classification (Necessist)

SignalBucketReasoning
Redundant setup or debug callFalse PositiveStatement genuinely unnecessary
Cannot map to production functionFalse PositiveNo graph context for triage
Call removed, no assertion checks its effectMissing TestsTest has weak assertions
Assertion removed, test still passesMissing TestsRedundant or insufficient coverage
Maps to high-CC entrypoint-reachable functionFuzzing TargetComplex + exposed + weak test

When both mutation testing and necessist flag the same production function, mark as corroborated — highest confidence finding.

For detailed criteria, see references/triage-methodology.md.

Graph Queries for Triage

For each mutant, map it to its containing graph node and use pre-analysis subgraphs (tainted, high_blast_radius, privilege_boundary) from Phase 1 to classify it. The classification logic checks: no callers → false positive, privilege boundary → fuzzing, high CC + tainted → fuzzing, high blast radius → fuzzing, otherwise → missing tests.

See references/graph-analysis.md for the batch_triage implementation and node mapping functions.


Output Format

Generate a markdown report:

# Genotoxic Triage Report

## Summary
- Total survived mutants: N
- Total necessist removals: N
- Corroborated findings: N
- False positives: N (N%)
- Missing test coverage: N (N%)
- Fuzzing targets: N (N%)

## Corroborated Findings
| File | Line | Function | Mutation Signal | Necessist Signal | Action |
|------|------|----------|----------------|------------------|--------|

## False Positives
| File | Line | Mutation | Reason | Source |
|------|------|----------|--------|--------|

## Missing Test Coverage
| File | Line | Function | CC | Callers | Suggested Test | Source |
|------|------|----------|----|---------|----------------|--------|

## Fuzzing Targets
| File | Line | Function | CC | Entrypoint Path | Blast Radius | Source |
|------|------|----------|----|-----------------|--------------|--------|

The Source column is mutation, necessist, or corroborated.

Write the report to GENOTOXIC_REPORT.md in the working directory.


Quality Checklist

Before delivering:

  • Trailmark graph built for target language
  • Mutation framework ran to completion
  • Necessist ran (if language supported) or noted as not applicable
  • All survived mutants triaged (none unclassified)
  • All necessist removals triaged (if applicable)
  • Corroborated findings identified (if both tools ran)
  • False positives have clear justifications
  • Missing test items include suggested test type
  • Fuzzing targets include entrypoint paths and blast radius
  • Report file written to GENOTOXIC_REPORT.md
  • User notified with summary statistics

Integration

trailmark skill:

  • Phase 1: Build code graph, query complexity and entrypoints
  • Phase 3: Caller analysis, reachability, blast radius

property-based-testing skill:

  • Missing test coverage items involving boundary conditions
  • Roundtrip/idempotence properties for serialization mutants

testing-handbook-skills (fuzzing):

  • Fuzzing target items: use harness-writing, cargo-fuzz, atheris

Supporting Documentation


First-time users: Start with Phase 1 (graph build), then run mutations, then use the Quick Classification table in Phase 3.

Experienced users: Jump to Phase 3 and use the Decision Tree to load specific reference material.

Writing Fuzzing Harnesses

A fuzzing harness is the entrypoint function that receives random data from the fuzzer and routes it to your system under test (SUT). The quality of your harness directly determines which code paths get exercised and whether critical bugs are found. A poorly written harness can miss entire subsystems or produce non-reproducible crashes.

Overview

The harness is the bridge between the fuzzer's random byte generation and your application's API. It must parse raw bytes into meaningful inputs, call target functions, and handle edge cases gracefully. The most important part of any fuzzing setup is the harness—if written poorly, critical parts of your application may not be covered.

Key Concepts

ConceptDescription
HarnessFunction that receives fuzzer input and calls target code under test
SUTSystem Under Test—the code being fuzzed
Entry pointFunction signature required by the fuzzer (e.g., LLVMFuzzerTestOneInput)
FuzzedDataProviderHelper class for structured extraction of typed data from raw bytes
DeterminismProperty that ensures same input always produces same behavior
Interleaved fuzzingSingle harness that exercises multiple operations based on input

When to Apply

Apply this technique when:

  • Creating a new fuzz target for the first time
  • Fuzz campaign has low code coverage or isn't finding bugs
  • Crashes found during fuzzing are not reproducible
  • Target API requires complex or structured inputs
  • Multiple related functions should be tested together

Skip this technique when:

  • Using existing well-tested harnesses from your project
  • Tool provides automatic harness generation that meets your needs
  • Target already has comprehensive fuzzing infrastructure

Quick Reference

TaskPattern
Minimal C++ harnessextern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
Minimal Rust harness`fuzz_target!(
Size validationif (size < MIN_SIZE) return 0;
Cast to integersuint32_t val = *(uint32_t*)(data);
Use FuzzedDataProviderFuzzedDataProvider fuzzed_data(data, size);
Extract typed data (C++)auto val = fuzzed_data.ConsumeIntegral<uint32_t>();
Extract string (C++)auto str = fuzzed_data.ConsumeBytesWithTerminator<char>(32, 0xFF);

Step-by-Step

Step 1: Identify Entry Points

Find functions in your codebase that:

  • Accept external input (parsers, validators, protocol handlers)
  • Parse complex data formats (JSON, XML, binary protocols)
  • Perform security-critical operations (authentication, cryptography)
  • Have high cyclomatic complexity or many branches

Good targets are typically:

  • Protocol parsers
  • File format parsers
  • Serialization/deserialization functions
  • Input validation routines

Step 2: Write Minimal Harness

Start with the simplest possible harness that calls your target function:

C/C++:

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    target_function(data, size);
    return 0;
}

Rust:

#![no_main]
use libfuzzer_sys::fuzz_target;

fuzz_target!(|data: &[u8]| {
    target_function(data);
});

Step 3: Add Input Validation

Reject inputs that are too small or too large to be meaningful:

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    // Ensure minimum size for meaningful input
    if (size < MIN_INPUT_SIZE || size > MAX_INPUT_SIZE) {
        return 0;
    }
    target_function(data, size);
    return 0;
}

Rationale: The fuzzer generates random inputs of all sizes. Your harness must handle empty, tiny, huge, or malformed inputs without causing unexpected issues in the harness itself (crashes in the SUT are fine—that's what we're looking for).

Step 4: Structure the Input

For APIs that require typed data (integers, strings, etc.), use casting or helpers like FuzzedDataProvider:

Simple casting:

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    if (size != 2 * sizeof(uint32_t)) {
        return 0;
    }

    uint32_t numerator = *(uint32_t*)(data);
    uint32_t denominator = *(uint32_t*)(data + sizeof(uint32_t));

    divide(numerator, denominator);
    return 0;
}

Using FuzzedDataProvider:

#include "FuzzedDataProvider.h"

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    FuzzedDataProvider fuzzed_data(data, size);

    size_t allocation_size = fuzzed_data.ConsumeIntegral<size_t>();
    std::vector<char> str1 = fuzzed_data.ConsumeBytesWithTerminator<char>(32, 0xFF);
    std::vector<char> str2 = fuzzed_data.ConsumeBytesWithTerminator<char>(32, 0xFF);

    concat(&str1[0], str1.size(), &str2[0], str2.size(), allocation_size);
    return 0;
}

Step 5: Test and Iterate

Run the fuzzer and monitor:

  • Code coverage (are all interesting paths reached?)
  • Executions per second (is it fast enough?)
  • Crash reproducibility (can you reproduce crashes with saved inputs?)

Iterate on the harness to improve these metrics.

Common Patterns

Pattern: Beyond Byte Arrays—Casting to Integers

Use Case: When target expects primitive types like integers or floats

Implementation:

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    // Ensure exactly 2 4-byte numbers
    if (size != 2 * sizeof(uint32_t)) {
        return 0;
    }

    // Split input into two integers
    uint32_t numerator = *(uint32_t*)(data);
    uint32_t denominator = *(uint32_t*)(data + sizeof(uint32_t));

    divide(numerator, denominator);
    return 0;
}

Rust equivalent:

fuzz_target!(|data: &[u8]| {
    if data.len() != 2 * std::mem::size_of::<i32>() {
        return;
    }

    let numerator = i32::from_ne_bytes([data[0], data[1], data[2], data[3]]);
    let denominator = i32::from_ne_bytes([data[4], data[5], data[6], data[7]]);

    divide(numerator, denominator);
});

Why it works: Any 8-byte input is valid. The fuzzer learns that inputs must be exactly 8 bytes, and every bit flip produces a new, potentially interesting input.

Pattern: FuzzedDataProvider for Complex Inputs

Use Case: When target requires multiple strings, integers, or variable-length data

Implementation:

#include "FuzzedDataProvider.h"

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    FuzzedDataProvider fuzzed_data(data, size);

    // Extract different types of data
    size_t allocation_size = fuzzed_data.ConsumeIntegral<size_t>();

    // Consume variable-length strings with terminator
    std::vector<char> str1 = fuzzed_data.ConsumeBytesWithTerminator<char>(32, 0xFF);
    std::vector<char> str2 = fuzzed_data.ConsumeBytesWithTerminator<char>(32, 0xFF);

    char* result = concat(&str1[0], str1.size(), &str2[0], str2.size(), allocation_size);
    if (result != NULL) {
        free(result);
    }

    return 0;
}

Why it helps: FuzzedDataProvider handles the complexity of extracting structured data from a byte stream. It's particularly useful for APIs that need multiple parameters of different types.

Pattern: Interleaved Fuzzing

Use Case: When multiple related operations should be tested in a single harness

Implementation:

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    if (size < 1 + 2 * sizeof(int32_t)) {
        return 0;
    }

    // First byte selects operation
    uint8_t mode = data[0];

    // Next bytes are operands
    int32_t numbers[2];
    memcpy(numbers, data + 1, 2 * sizeof(int32_t));

    int32_t result = 0;
    switch (mode % 4) {
        case 0:
            result = add(numbers[0], numbers[1]);
            break;
        case 1:
            result = subtract(numbers[0], numbers[1]);
            break;
        case 2:
            result = multiply(numbers[0], numbers[1]);
            break;
        case 3:
            result = divide(numbers[0], numbers[1]);
            break;
    }

    // Prevent compiler from optimizing away the calls
    printf("%d", result);
    return 0;
}

Advantages:

  • Faster to write one harness than multiple individual harnesses
  • Single shared corpus means interesting inputs for one operation may be interesting for others
  • Can discover bugs in interactions between operations

When to use:

  • Operations share similar input types
  • Operations are logically related (e.g., arithmetic operations, CRUD operations)
  • Single corpus makes sense across all operations

Pattern: Structure-Aware Fuzzing with Arbitrary (Rust)

Use Case: When fuzzing Rust code that uses custom structs

Implementation:

use arbitrary::Arbitrary;

#[derive(Debug, Arbitrary)]
pub struct Name {
    data: String
}

impl Name {
    pub fn check_buf(&self) {
        let data = self.data.as_bytes();
        if data.len() > 0 && data[0] == b'a' {
            if data.len() > 1 && data[1] == b'b' {
                if data.len() > 2 && data[2] == b'c' {
                    process::abort();
                }
            }
        }
    }
}

Harness with arbitrary:

#![no_main]
use libfuzzer_sys::fuzz_target;

fuzz_target!(|data: your_project::Name| {
    data.check_buf();
});

Add to Cargo.toml:

[dependencies]
arbitrary = { version = "1", features = ["derive"] }

Why it helps: The arbitrary crate automatically handles deserialization of raw bytes into your Rust structs, reducing boilerplate and ensuring valid struct construction.

Limitation: The arbitrary crate doesn't offer reverse serialization, so you can't manually construct byte arrays that map to specific structs. This works best when starting from an empty corpus (fine for libFuzzer, problematic for AFL++).

Advanced Usage

Tips and Tricks

TipWhy It Helps
Start with parsersHigh bug density, clear entry points, easy to harness
Mock I/O operationsPrevents hangs from blocking I/O, enables determinism
Use FuzzedDataProviderSimplifies extraction of structured data from raw bytes
Reset global stateEnsures each iteration is independent and reproducible
Free resources in harnessPrevents memory exhaustion during long campaigns
Avoid logging in harnessLogging is slow—fuzzing needs 100s-1000s exec/sec
Test harness manually firstRun harness with known inputs before starting campaign
Check coverage earlyEnsure harness reaches expected code paths

Structure-Aware Fuzzing with Protocol Buffers

For highly structured input formats, consider using Protocol Buffers as an intermediate format with custom mutators:

// Define your input format in .proto file
// Use libprotobuf-mutator to generate valid mutations
// This ensures fuzzer mutates message contents, not the protobuf encoding itself

This approach is more setup but prevents the fuzzer from wasting time on unparseable inputs. See structure-aware fuzzing documentation for details.

Handling Non-Determinism

Problem: Random values or timing dependencies cause non-reproducible crashes.

Solutions:

  • Replace rand() with deterministic PRNG seeded from fuzzer input:
    uint32_t seed = fuzzed_data.ConsumeIntegral<uint32_t>();
    srand(seed);
    
  • Mock system calls that return time, PIDs, or random data
  • Avoid reading from /dev/random or /dev/urandom

Resetting Global State

If your SUT uses global state (singletons, static variables), reset it between iterations:

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    // Reset global state before each iteration
    global_reset();

    target_function(data, size);

    // Clean up resources
    global_cleanup();
    return 0;
}

Rationale: Global state can cause crashes after N iterations rather than on a specific input, making bugs non-reproducible.

Practical Harness Rules

Follow these rules to ensure effective fuzzing harnesses:

RuleRationale
Handle all input sizesFuzzer generates empty, tiny, huge inputs—harness must handle gracefully
Never call exit()Calling exit() stops the fuzzer process. Use abort() in SUT if needed
Join all threadsEach iteration must run to completion before next iteration starts
Be fastAim for 100s-1000s executions/sec. Avoid logging, high complexity, excess memory
Maintain determinismSame input must always produce same behavior for reproducibility
Avoid global stateGlobal state reduces reproducibility—reset between iterations if unavoidable
Use narrow targetsDon't fuzz PNG and TCP in same harness—different formats need separate targets
Free resourcesPrevent memory leaks that cause resource exhaustion during long campaigns

Note: These guidelines apply not just to harness code, but to the entire SUT. If the SUT violates these rules, consider patching it (see the fuzzing obstacles technique).

Anti-Patterns

Anti-PatternProblemCorrect Approach
Global state without resetNon-deterministic crashesReset all globals at start of harness
Blocking I/O or network callsHangs fuzzer, wastes timeMock I/O, use in-memory buffers
Memory leaks in harnessResource exhaustion kills campaignFree all allocations before returning
Calling exit() in SUTStops entire fuzzing processUse abort() or return error codes
Heavy logging in harnessReduces exec/sec by orders of magnitudeDisable logging during fuzzing
Too many operations per iterationSlows down fuzzerKeep iterations fast and focused
Mixing unrelated input formatsCorpus entries not useful across formatsSeparate harnesses for different formats
Not validating input sizeHarness crashes on edge casesCheck size before accessing data

Tool-Specific Guidance

libFuzzer

Harness signature:

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    // Your code here
    return 0;  // Non-zero return is reserved for future use
}

Compilation:

clang++ -fsanitize=fuzzer,address -g harness.cc -o fuzz_target

Integration tips:

  • Use FuzzedDataProvider.h for structured input extraction
  • Compile with -fsanitize=fuzzer to link the fuzzing runtime
  • Add sanitizers (-fsanitize=address,undefined) to detect more bugs
  • Use -g for better stack traces when crashes occur
  • libFuzzer can start with empty corpus—no seed inputs required

Running:

./fuzz_target corpus_dir/

Resources:

AFL++

AFL++ supports multiple harness styles. For best performance, use persistent mode:

Persistent mode harness:

#include <unistd.h>

int main(int argc, char **argv) {
    #ifdef __AFL_HAVE_MANUAL_CONTROL
        __AFL_INIT();
    #endif

    unsigned char buf[MAX_SIZE];

    while (__AFL_LOOP(10000)) {
        // Read input from stdin
        ssize_t len = read(0, buf, sizeof(buf));
        if (len <= 0) break;

        // Call target function
        target_function(buf, len);
    }

    return 0;
}

Compilation:

afl-clang-fast++ -g harness.cc -o fuzz_target

Integration tips:

  • Use persistent mode (__AFL_LOOP) for 10-100x speedup
  • Consider deferred initialization (__AFL_INIT()) to skip setup overhead
  • AFL++ requires at least one seed input in the corpus directory
  • Use AFL_USE_ASAN=1 or AFL_USE_UBSAN=1 for sanitizer builds

Running:

afl-fuzz -i seeds/ -o findings/ -- ./fuzz_target

cargo-fuzz (Rust)

Harness signature:

#![no_main]
use libfuzzer_sys::fuzz_target;

fuzz_target!(|data: &[u8]| {
    // Your code here
});

With structured input (arbitrary crate):

#![no_main]
use libfuzzer_sys::fuzz_target;

fuzz_target!(|data: YourStruct| {
    data.check();
});

Creating harness:

cargo fuzz init
cargo fuzz add my_target

Integration tips:

  • Use arbitrary crate for automatic struct deserialization
  • cargo-fuzz wraps libFuzzer, so all libFuzzer features work
  • Compile with sanitizers automatically via cargo-fuzz
  • Harnesses go in fuzz/fuzz_targets/ directory

Running:

cargo +nightly fuzz run my_target

Resources:

go-fuzz

Harness signature:

// +build gofuzz

package mypackage

func Fuzz(data []byte) int {
    // Call target function
    target(data)

    // Return codes:
    // -1 if input is invalid
    //  0 if input is valid but not interesting
    //  1 if input is interesting (e.g., added new coverage)
    return 0
}

Building:

go-fuzz-build

Integration tips:

  • Return 1 for inputs that add coverage (optional—fuzzer can detect automatically)
  • Return -1 for invalid inputs to deprioritize similar mutations
  • go-fuzz handles persistence automatically

Running:

go-fuzz -bin=./mypackage-fuzz.zip -workdir=fuzz

Troubleshooting

IssueCauseSolution
Low executions/secHarness is too slow (logging, I/O, complexity)Profile harness, remove bottlenecks, mock I/O
No crashes foundCoverage not reaching buggy codeCheck coverage, improve harness to reach more paths
Non-reproducible crashesNon-determinism or global stateRemove randomness, reset globals between iterations
Fuzzer exits immediatelyHarness calls exit()Replace exit() with abort() or return error
Out of memory errorsMemory leaks in harness or SUTFree allocations, use leak sanitizer to find leaks
Crashes on empty inputHarness doesn't validate sizeAdd if (size < MIN_SIZE) return 0;
Corpus not growingInputs too constrained or format too strictUse FuzzedDataProvider or structure-aware fuzzing

Related Skills

Tools That Use This Technique

SkillHow It Applies
libfuzzerUses LLVMFuzzerTestOneInput harness signature with FuzzedDataProvider
aflppSupports persistent mode harnesses with __AFL_LOOP for performance
cargo-fuzzUses Rust-specific fuzz_target! macro with arbitrary crate integration
atherisPython harness takes bytes, calls Python functions
ossfuzzRequires harnesses in specific directory structure for cloud fuzzing

Related Techniques

SkillRelationship
coverage-analysisMeasure harness effectiveness—are you reaching target code?
address-sanitizerDetects bugs found by harness (buffer overflows, use-after-free)
fuzzing-dictionaryProvide tokens to help fuzzer pass format checks in harness
fuzzing-obstaclesPatch SUT when it violates harness rules (exit, non-determinism)

Resources

Key External Resources

Split Inputs in libFuzzer - Google Fuzzing Docs Explains techniques for handling multiple input parameters in a single fuzzing harness, including use of magic separators and FuzzedDataProvider.

Structure-Aware Fuzzing with Protocol Buffers Advanced technique using protobuf as intermediate format with custom mutators to ensure fuzzer mutates message contents rather than format encoding.

libFuzzer Documentation Official LLVM documentation covering harness requirements, best practices, and advanced features.

cargo-fuzz Book Comprehensive guide to writing Rust fuzzing harnesses with cargo-fuzz and the arbitrary crate.

Video Resources

LibAFL

LibAFL is a modular fuzzing library that implements features from AFL-based fuzzers like AFL++. Unlike traditional fuzzers, LibAFL provides all functionality in a modular and customizable way as a Rust library. It can be used as a drop-in replacement for libFuzzer or as a library to build custom fuzzers from scratch.

When to Use

FuzzerBest ForComplexity
libFuzzerQuick setup, single-threadedLow
AFL++Multi-core, general purposeMedium
LibAFLCustom fuzzers, advanced features, researchHigh

Choose LibAFL when:

  • You need custom mutation strategies or feedback mechanisms
  • Standard fuzzers don't support your target architecture
  • You want to implement novel fuzzing techniques
  • You need fine-grained control over fuzzing components
  • You're conducting fuzzing research

Quick Start

LibAFL can be used as a drop-in replacement for libFuzzer with minimal setup:

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    // Call your code with fuzzer-provided data
    my_function(data, size);
    return 0;
}

Build LibAFL's libFuzzer compatibility layer:

git clone https://github.com/AFLplusplus/LibAFL
cd LibAFL/libafl_libfuzzer_runtime
./build.sh

Compile and run:

clang++ -DNO_MAIN -g -O2 -fsanitize=fuzzer-no-link libFuzzer.a harness.cc main.cc -o fuzz
./fuzz corpus/

Installation

Prerequisites

  • Clang/LLVM 15-18
  • Rust (via rustup)
  • Additional system dependencies

Linux/macOS

Install Clang:

apt install clang

Or install a specific version via apt.llvm.org:

wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh 15

Configure environment for Rust:

export RUSTFLAGS="-C linker=/usr/bin/clang-15"
export CC="clang-15"
export CXX="clang++-15"

Install Rust:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Install additional dependencies:

apt install libssl-dev pkg-config

For libFuzzer compatibility mode, install nightly Rust:

rustup toolchain install nightly --component llvm-tools

Verification

Build LibAFL to verify installation:

cd LibAFL/libafl_libfuzzer_runtime
./build.sh
# Should produce libFuzzer.a

Writing a Harness

LibAFL harnesses follow the same pattern as libFuzzer when using drop-in replacement mode:

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    // Your fuzzing target code here
    return 0;
}

When building custom fuzzers with LibAFL as a Rust library, harness logic is integrated directly into the fuzzer. See the "Writing a Custom Fuzzer" section below for the full pattern.

See Also: For detailed harness writing techniques, see the harness-writing technique skill.

Usage Modes

LibAFL supports two primary usage modes:

1. libFuzzer Drop-in Replacement

Use LibAFL as a replacement for libFuzzer with existing harnesses.

Compilation:

clang++ -DNO_MAIN -g -O2 -fsanitize=fuzzer-no-link libFuzzer.a harness.cc main.cc -o fuzz

Running:

./fuzz corpus/

Recommended for long campaigns:

./fuzz -fork=1 -ignore_crashes=1 corpus/

2. Custom Fuzzer as Rust Library

Build a fully customized fuzzer using LibAFL components.

Create project:

cargo init --lib my_fuzzer
cd my_fuzzer
cargo add libafl@0.13 libafl_targets@0.13 libafl_bolts@0.13 libafl_cc@0.13 \
  --features "libafl_targets@0.13/libfuzzer,libafl_targets@0.13/sancov_pcguard_hitcounts"

Configure Cargo.toml:

[lib]
crate-type = ["staticlib"]

Writing a Custom Fuzzer

See Also: For detailed harness writing techniques, patterns for handling complex inputs, and advanced strategies, see the fuzz-harness-writing technique skill.

Fuzzer Components

A LibAFL fuzzer consists of modular components:

  1. Observers - Collect execution feedback (coverage, timing)
  2. Feedback - Determine if inputs are interesting
  3. Objective - Define fuzzing goals (crashes, timeouts)
  4. State - Maintain corpus and metadata
  5. Mutators - Generate new inputs
  6. Scheduler - Select which inputs to mutate
  7. Executor - Run the target with inputs

Basic Fuzzer Structure

use libafl::prelude::*;
use libafl_bolts::prelude::*;
use libafl_targets::{libfuzzer_test_one_input, std_edges_map_observer};

#[no_mangle]
pub extern "C" fn libafl_main() {
    let mut run_client = |state: Option<_>, mut restarting_mgr, _core_id| {
        // 1. Setup observers
        let edges_observer = HitcountsMapObserver::new(
            unsafe { std_edges_map_observer("edges") }
        ).track_indices();
        let time_observer = TimeObserver::new("time");

        // 2. Define feedback
        let mut feedback = feedback_or!(
            MaxMapFeedback::new(&edges_observer),
            TimeFeedback::new(&time_observer)
        );

        // 3. Define objective
        let mut objective = feedback_or_fast!(
            CrashFeedback::new(),
            TimeoutFeedback::new()
        );

        // 4. Create or restore state
        let mut state = state.unwrap_or_else(|| {
            StdState::new(
                StdRand::new(),
                InMemoryCorpus::new(),
                OnDiskCorpus::new(&output_dir).unwrap(),
                &mut feedback,
                &mut objective,
            ).unwrap()
        });

        // 5. Setup mutator
        let mutator = StdScheduledMutator::new(havoc_mutations());
        let mut stages = tuple_list!(StdMutationalStage::new(mutator));

        // 6. Setup scheduler
        let scheduler = IndexesLenTimeMinimizerScheduler::new(
            &edges_observer,
            QueueScheduler::new()
        );

        // 7. Create fuzzer
        let mut fuzzer = StdFuzzer::new(scheduler, feedback, objective);

        // 8. Define harness
        let mut harness = |input: &BytesInput| {
            let buf = input.target_bytes().as_slice();
            libfuzzer_test_one_input(buf);
            ExitKind::Ok
        };

        // 9. Setup executor
        let mut executor = InProcessExecutor::with_timeout(
            &mut harness,
            tuple_list!(edges_observer, time_observer),
            &mut fuzzer,
            &mut state,
            &mut restarting_mgr,
            timeout,
        )?;

        // 10. Load initial inputs
        if state.must_load_initial_inputs() {
            state.load_initial_inputs(
                &mut fuzzer,
                &mut executor,
                &mut restarting_mgr,
                &input_dir
            )?;
        }

        // 11. Start fuzzing
        fuzzer.fuzz_loop(&mut stages, &mut executor, &mut state, &mut restarting_mgr)?;
        Ok(())
    };

    // Launch fuzzer
    Launcher::builder()
        .run_client(&mut run_client)
        .cores(&cores)
        .build()
        .launch()
        .unwrap();
}

Compilation

Verbose Mode

Manually specify all instrumentation flags:

clang++-15 -DNO_MAIN -g -O2 \
  -fsanitize-coverage=trace-pc-guard \
  -fsanitize=address \
  -Wl,--whole-archive target/release/libmy_fuzzer.a -Wl,--no-whole-archive \
  main.cc harness.cc -o fuzz

Compiler Wrapper (Recommended)

Create a LibAFL compiler wrapper to handle instrumentation automatically.

Create src/bin/libafl_cc.rs:

use libafl_cc::{ClangWrapper, CompilerWrapper, Configuration, ToolWrapper};

pub fn main() {
    let args: Vec<String> = env::args().collect();
    let mut cc = ClangWrapper::new();
    cc.cpp(is_cpp)
      .parse_args(&args)
      .link_staticlib(&dir, "my_fuzzer")
      .add_args(&Configuration::GenerateCoverageMap.to_flags().unwrap())
      .add_args(&Configuration::AddressSanitizer.to_flags().unwrap())
      .run()
      .unwrap();
}

Compile and use:

cargo build --release
target/release/libafl_cxx -DNO_MAIN -g -O2 main.cc harness.cc -o fuzz

See Also: For detailed sanitizer configuration, common issues, and advanced flags, see the address-sanitizer and undefined-behavior-sanitizer technique skills.

Running Campaigns

Basic Run

./fuzz --cores 0 --input corpus/

Multi-Core Fuzzing

./fuzz --cores 0,8-15 --input corpus/

This runs 9 clients: one on core 0, and 8 on cores 8-15.

With Options

./fuzz --cores 0-7 --input corpus/ --output crashes/ --timeout 1000

Text User Interface (TUI)

Enable graphical statistics view:

./fuzz -tui=1 corpus/

Interpreting Output

OutputMeaning
corpus: NNumber of interesting test cases found
objectives: NNumber of crashes/timeouts found
executions: NTotal number of target invocations
exec/sec: NCurrent execution throughput
edges: X%Code coverage percentage
clients: NNumber of parallel fuzzing processes

The fuzzer emits two main event types:

  • UserStats - Regular heartbeat with current statistics
  • Testcase - New interesting input discovered

Advanced Usage

Tips and Tricks

TipWhy It Helps
Use -fork=1 -ignore_crashes=1Continue fuzzing after first crash
Use InMemoryOnDiskCorpusPersist corpus across restarts
Enable TUI with -tui=1Better visualization of progress
Use specific LLVM versionAvoid compatibility issues
Set RUSTFLAGS correctlyPrevent linking errors

Crash Deduplication

Avoid storing duplicate crashes from the same bug:

Add backtrace observer:

let backtrace_observer = BacktraceObserver::owned(
    "BacktraceObserver",
    libafl::observers::HarnessType::InProcess
);

Update executor:

let mut executor = InProcessExecutor::with_timeout(
    &mut harness,
    tuple_list!(edges_observer, time_observer, backtrace_observer),
    &mut fuzzer,
    &mut state,
    &mut restarting_mgr,
    timeout,
)?;

Update objective with hash feedback:

let mut objective = feedback_and!(
    feedback_or_fast!(CrashFeedback::new(), TimeoutFeedback::new()),
    NewHashFeedback::new(&backtrace_observer)
);

This ensures only crashes with unique backtraces are saved.

Dictionary Fuzzing

Use dictionaries to guide fuzzing toward specific tokens:

Add tokens from file:

let mut tokens = Tokens::new();
if let Some(tokenfile) = &tokenfile {
    tokens.add_from_file(tokenfile)?;
}
state.add_metadata(tokens);

Update mutator:

let mutator = StdScheduledMutator::new(
    havoc_mutations().merge(tokens_mutations())
);

Hard-coded tokens example (PNG):

state.add_metadata(Tokens::from([
    vec![137, 80, 78, 71, 13, 10, 26, 10], // PNG header
    "IHDR".as_bytes().to_vec(),
    "IDAT".as_bytes().to_vec(),
    "PLTE".as_bytes().to_vec(),
    "IEND".as_bytes().to_vec(),
]));

See Also: For detailed dictionary creation strategies and format-specific dictionaries, see the fuzzing-dictionaries technique skill.

Auto Tokens

Automatically extract magic values and checksums from the program:

Enable in compiler wrapper:

cc.add_pass(LLVMPasses::AutoTokens)

Load auto tokens in fuzzer:

tokens += libafl_targets::autotokens()?;

Verify tokens section:

echo "p (uint8_t *)__token_start" | gdb fuzz

Performance Tuning

SettingImpact
Multi-core fuzzingLinear speedup with cores
InMemoryCorpusFaster but non-persistent
InMemoryOnDiskCorpusBalanced speed and persistence
Sanitizers2-5x slowdown, essential for bugs
Optimization level -O2Balance between speed and coverage

Debugging Fuzzer

Run fuzzer in single-process mode for easier debugging:

// Replace launcher with direct call
run_client(None, SimpleEventManager::new(monitor), 0).unwrap();

// Comment out:
// Launcher::builder()
//     .run_client(&mut run_client)
//     ...
//     .launch()

Then debug with GDB:

gdb --args ./fuzz --cores 0 --input corpus/

Real-World Examples

Example: libpng

Fuzzing libpng using LibAFL:

1. Get source code:

curl -L -O https://downloads.sourceforge.net/project/libpng/libpng16/1.6.37/libpng-1.6.37.tar.xz
tar xf libpng-1.6.37.tar.xz
cd libpng-1.6.37/
apt install zlib1g-dev

2. Set compiler wrapper:

export FUZZER_CARGO_DIR="/path/to/libafl/project"
export CC=$FUZZER_CARGO_DIR/target/release/libafl_cc
export CXX=$FUZZER_CARGO_DIR/target/release/libafl_cxx

3. Build static library:

./configure --enable-shared=no
make

4. Get harness:

curl -O https://raw.githubusercontent.com/glennrp/libpng/f8e5fa92b0e37ab597616f554bee254157998227/contrib/oss-fuzz/libpng_read_fuzzer.cc

5. Link fuzzer:

$CXX libpng_read_fuzzer.cc .libs/libpng16.a -lz -o fuzz

6. Prepare seeds:

mkdir seeds/
curl -o seeds/input.png https://raw.githubusercontent.com/glennrp/libpng/acfd50ae0ba3198ad734e5d4dec2b05341e50924/contrib/pngsuite/iftp1n3p08.png

7. Get dictionary (optional):

curl -O https://raw.githubusercontent.com/glennrp/libpng/2fff013a6935967960a5ae626fc21432807933dd/contrib/oss-fuzz/png.dict

8. Start fuzzing:

./fuzz --input seeds/ --cores 0 -x png.dict

Example: CMake Project

Integrate LibAFL with CMake build system:

CMakeLists.txt:

project(BuggyProgram)
cmake_minimum_required(VERSION 3.0)

add_executable(buggy_program main.cc)

add_executable(fuzz main.cc harness.cc)
target_compile_definitions(fuzz PRIVATE NO_MAIN=1)
target_compile_options(fuzz PRIVATE -g -O2)

Build non-instrumented binary:

cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ .
cmake --build . --target buggy_program

Build fuzzer:

export FUZZER_CARGO_DIR="/path/to/libafl/project"
cmake -DCMAKE_C_COMPILER=$FUZZER_CARGO_DIR/target/release/libafl_cc \
      -DCMAKE_CXX_COMPILER=$FUZZER_CARGO_DIR/target/release/libafl_cxx .
cmake --build . --target fuzz

Run fuzzing:

./fuzz --input seeds/ --cores 0

Troubleshooting

ProblemCauseSolution
No coverage increasesInstrumentation failedVerify compiler wrapper used, check for -fsanitize-coverage
Fuzzer won't startEmpty corpus with no interesting inputsProvide seed inputs that trigger code paths
Linker errors with libafl_mainRuntime not linkedUse -Wl,--whole-archive or -u libafl_main
LLVM version mismatchLibAFL requires LLVM 15-18Install compatible LLVM version, set environment variables
Rust compilation failsOutdated Rust or CargoUpdate Rust with rustup update
Slow fuzzingSanitizers enabledExpected 2-5x slowdown, necessary for finding bugs
Environment variable interferenceCC, CXX, RUSTFLAGS setUnset after building LibAFL project
Cannot attach debuggerMulti-process fuzzingRun in single-process mode (see Debugging section)

Related Skills

Technique Skills

SkillUse Case
fuzz-harness-writingDetailed guidance on writing effective harnesses
address-sanitizerMemory error detection during fuzzing
undefined-behavior-sanitizerUndefined behavior detection
coverage-analysisMeasuring and improving code coverage
fuzzing-corpusBuilding and managing seed corpora
fuzzing-dictionariesCreating dictionaries for format-aware fuzzing

Related Fuzzers

SkillWhen to Consider
libfuzzerSimpler setup, don't need LibAFL's advanced features
aflppMulti-core fuzzing without custom fuzzer development
cargo-fuzzFuzzing Rust projects with less setup

Resources

Official Documentation

Examples and Tutorials

libFuzzer

libFuzzer is an in-process, coverage-guided fuzzer that is part of the LLVM project. It's the recommended starting point for fuzzing C/C++ projects due to its simplicity and integration with the LLVM toolchain. While libFuzzer has been in maintenance-only mode since late 2022, it is easier to install and use than its alternatives, has wide support, and will be maintained for the foreseeable future.

When to Use

FuzzerBest ForComplexity
libFuzzerQuick setup, single-project fuzzingLow
AFL++Multi-core fuzzing, diverse mutationsMedium
LibAFLCustom fuzzers, research projectsHigh
HonggfuzzHardware-based coverageMedium

Choose libFuzzer when:

  • You need a simple, quick setup for C/C++ code
  • Project uses Clang for compilation
  • Single-core fuzzing is sufficient initially
  • Transitioning to AFL++ later is an option (harnesses are compatible)

Note: Fuzzing harnesses written for libFuzzer are compatible with AFL++, making it easy to transition if you need more advanced features like better multi-core support.

Quick Start

#include <stdint.h>
#include <stddef.h>

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    // Validate input if needed
    if (size < 1) return 0;

    // Call your target function with fuzzer-provided data
    my_target_function(data, size);

    return 0;
}

Compile and run:

clang++ -fsanitize=fuzzer,address -g -O2 harness.cc target.cc -o fuzz
mkdir corpus/
./fuzz corpus/

Installation

Prerequisites

  • LLVM/Clang compiler (includes libFuzzer)
  • LLVM tools for coverage analysis (optional)

Linux (Ubuntu/Debian)

apt install clang llvm

For the latest LLVM version:

# Add LLVM repository from apt.llvm.org
# Then install specific version, e.g.:
apt install clang-18 llvm-18

macOS

# Using Homebrew
brew install llvm

# Or using Nix
nix-env -i clang

Windows

Install Clang through Visual Studio. Refer to Microsoft's documentation for setup instructions.

Recommendation: If possible, fuzz on a local x86_64 VM or rent one on DigitalOcean, AWS, or Hetzner. Linux provides the best support for libFuzzer.

Verification

clang++ --version
# Should show LLVM version information

Writing a Harness

Harness Structure

The harness is the entry point for the fuzzer. libFuzzer calls the LLVMFuzzerTestOneInput function repeatedly with different inputs.

#include <stdint.h>
#include <stddef.h>

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    // 1. Optional: Validate input size
    if (size < MIN_REQUIRED_SIZE) {
        return 0;  // Reject inputs that are too small
    }

    // 2. Optional: Convert raw bytes to structured data
    // Example: Parse two integers from byte array
    if (size >= 2 * sizeof(uint32_t)) {
        uint32_t a = *(uint32_t*)(data);
        uint32_t b = *(uint32_t*)(data + sizeof(uint32_t));
        my_function(a, b);
    }

    // 3. Call target function
    target_function(data, size);

    // 4. Always return 0 (non-zero reserved for future use)
    return 0;
}

Harness Rules

DoDon't
Handle all input types (empty, huge, malformed)Call exit() - stops fuzzing process
Join all threads before returningLeave threads running
Keep harness fast and simpleAdd excessive logging or complexity
Maintain determinismUse random number generators or read /dev/random
Reset global state between runsRely on state from previous executions
Use narrow, focused targetsMix unrelated data formats (PNG + TCP) in one harness

Rationale:

  • Speed matters: Aim for 100s-1000s executions per second per core
  • Reproducibility: Crashes must be reproducible after fuzzing completes
  • Isolation: Each execution should be independent

Using FuzzedDataProvider for Complex Inputs

For complex inputs (strings, multiple parameters), use the FuzzedDataProvider helper:

#include <stdint.h>
#include <stddef.h>
#include "FuzzedDataProvider.h"  // From LLVM project

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    FuzzedDataProvider fuzzed_data(data, size);

    // Extract structured data
    size_t allocation_size = fuzzed_data.ConsumeIntegral<size_t>();
    std::vector<char> str1 = fuzzed_data.ConsumeBytesWithTerminator<char>(32, 0xFF);
    std::vector<char> str2 = fuzzed_data.ConsumeBytesWithTerminator<char>(32, 0xFF);

    // Call target with extracted data
    char* result = concat(&str1[0], str1.size(), &str2[0], str2.size(), allocation_size);
    if (result != NULL) {
        free(result);
    }

    return 0;
}

Download FuzzedDataProvider.h from the LLVM repository.

Interleaved Fuzzing

Use a single harness to test multiple related functions:

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    if (size < 1 + 2 * sizeof(int32_t)) {
        return 0;
    }

    uint8_t mode = data[0];
    int32_t numbers[2];
    memcpy(numbers, data + 1, 2 * sizeof(int32_t));

    // Select function based on first byte
    switch (mode % 4) {
        case 0: add(numbers[0], numbers[1]); break;
        case 1: subtract(numbers[0], numbers[1]); break;
        case 2: multiply(numbers[0], numbers[1]); break;
        case 3: divide(numbers[0], numbers[1]); break;
    }

    return 0;
}

See Also: For detailed harness writing techniques, patterns for handling complex inputs, structure-aware fuzzing, and protobuf-based fuzzing, see the fuzz-harness-writing technique skill.

Compilation

Basic Compilation

The key flag is -fsanitize=fuzzer, which:

  • Links the libFuzzer runtime (provides main function)
  • Enables SanitizerCoverage instrumentation for coverage tracking
  • Disables built-in functions like memcmp
clang++ -fsanitize=fuzzer -g -O2 harness.cc target.cc -o fuzz

Flags explained:

  • -fsanitize=fuzzer: Enable libFuzzer
  • -g: Add debug symbols (helpful for crash analysis)
  • -O2: Production-level optimizations (recommended for fuzzing)
  • -DNO_MAIN: Define macro if your code has a main function

With Sanitizers

AddressSanitizer (recommended):

clang++ -fsanitize=fuzzer,address -g -O2 -U_FORTIFY_SOURCE harness.cc target.cc -o fuzz

Multiple sanitizers:

clang++ -fsanitize=fuzzer,address,undefined -g -O2 harness.cc target.cc -o fuzz

See Also: For detailed sanitizer configuration, common issues, ASAN_OPTIONS flags, and advanced sanitizer usage, see the address-sanitizer and undefined-behavior-sanitizer technique skills.

Build Flags

FlagPurpose
-fsanitize=fuzzerEnable libFuzzer runtime and instrumentation
-fsanitize=addressEnable AddressSanitizer (memory error detection)
-fsanitize=undefinedEnable UndefinedBehaviorSanitizer
-fsanitize=fuzzer-no-linkInstrument without linking fuzzer (for libraries)
-gInclude debug symbols
-O2Production optimization level
-U_FORTIFY_SOURCEDisable fortification (can interfere with ASan)

Building Static Libraries

For projects that produce static libraries:

  1. Build the library with fuzzing instrumentation:
export CC=clang CFLAGS="-fsanitize=fuzzer-no-link -fsanitize=address"
export CXX=clang++ CXXFLAGS="$CFLAGS"
./configure --enable-shared=no
make
  1. Link the static library with your harness:
clang++ -fsanitize=fuzzer -fsanitize=address harness.cc libmylib.a -o fuzz

CMake Integration

project(FuzzTarget)
cmake_minimum_required(VERSION 3.0)

add_executable(fuzz main.cc harness.cc)
target_compile_definitions(fuzz PRIVATE NO_MAIN=1)
target_compile_options(fuzz PRIVATE -g -O2 -fsanitize=fuzzer -fsanitize=address)
target_link_libraries(fuzz -fsanitize=fuzzer -fsanitize=address)

Build with:

cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ .
cmake --build .

Corpus Management

Creating Initial Corpus

Create a directory for the corpus (can start empty):

mkdir corpus/

Optional but recommended: Provide seed inputs (valid example files):

# For a PNG parser:
cp examples/*.png corpus/

# For a protocol parser:
cp test_packets/*.bin corpus/

Benefits of seed inputs:

  • Fuzzer doesn't start from scratch
  • Reaches valid code paths faster
  • Significantly improves effectiveness

Corpus Structure

The corpus directory contains:

  • Input files that trigger unique code paths
  • Minimized versions (libFuzzer automatically minimizes)
  • Named by content hash (e.g., a9993e364706816aba3e25717850c26c9cd0d89d)

Corpus Minimization

libFuzzer automatically minimizes corpus entries during fuzzing. To explicitly minimize:

mkdir minimized_corpus/
./fuzz -merge=1 minimized_corpus/ corpus/

This creates a deduplicated, minimized corpus in minimized_corpus/.

See Also: For corpus creation strategies, seed selection, format-specific corpus building, and corpus maintenance, see the fuzzing-corpus technique skill.

Running Campaigns

Basic Run

./fuzz corpus/

This runs until a crash is found or you stop it (Ctrl+C).

Recommended: Continue After Crashes

./fuzz -fork=1 -ignore_crashes=1 corpus/

The -fork and -ignore_crashes flags (experimental but widely used) allow fuzzing to continue after finding crashes.

Common Options

Control input size:

./fuzz -max_len=4000 corpus/

Rule of thumb: 2x the size of minimal realistic input.

Set timeout:

./fuzz -timeout=2 corpus/

Abort test cases that run longer than 2 seconds.

Use a dictionary:

./fuzz -dict=./format.dict corpus/

Close stdout/stderr (speed up fuzzing):

./fuzz -close_fd_mask=3 corpus/

See all options:

./fuzz -help=1

Multi-Core Fuzzing

Option 1: Jobs and workers (recommended):

./fuzz -jobs=4 -workers=4 -fork=1 -ignore_crashes=1 corpus/
  • -jobs=4: Run 4 sequential campaigns
  • -workers=4: Process jobs in parallel with 4 processes
  • Test cases are shared between jobs

Option 2: Fork mode:

./fuzz -fork=4 -ignore_crashes=1 corpus/

Note: For serious multi-core fuzzing, consider switching to AFL++, Honggfuzz, or LibAFL.

Re-executing Test Cases

Re-run a single crash:

./fuzz ./crash-a9993e364706816aba3e25717850c26c9cd0d89d

Test all inputs in a directory without fuzzing:

./fuzz -runs=0 corpus/

Interpreting Output

When fuzzing runs, you'll see statistics like:

INFO: Seed: 3517090860
INFO: Loaded 1 modules (9 inline 8-bit counters)
#2      INITED cov: 3 ft: 4 corp: 1/1b exec/s: 0 rss: 26Mb
#57     NEW    cov: 4 ft: 5 corp: 2/4b lim: 4 exec/s: 0 rss: 26Mb
OutputMeaning
INITEDFuzzing initialized
NEWNew coverage found, added to corpus
REDUCEInput minimized while keeping coverage
cov: NNumber of coverage edges hit
corp: X/YbCorpus size: X entries, Y total bytes
exec/s: NExecutions per second
rss: NMbResident memory usage

On crash:

==11672== ERROR: libFuzzer: deadly signal
artifact_prefix='./'; Test unit written to ./crash-a9993e364706816aba3e25717850c26c9cd0d89d
0x61,0x62,0x63,
abc
Base64: YWJj

The crash is saved to ./crash-<hash> with the input shown in hex, UTF-8, and Base64.

Reproducibility: Use -seed=<value> to reproduce a fuzzing campaign (single-core only).

Fuzzing Dictionary

Dictionaries help the fuzzer discover interesting inputs faster by providing hints about the input format.

Dictionary Format

Create a text file with quoted strings (one per line):

# Lines starting with '#' are comments

# Magic bytes
magic="\x89PNG"
magic2="IEND"

# Keywords
"GET"
"POST"
"Content-Type"

# Hex sequences
delimiter="\xFF\xD8\xFF"

Using a Dictionary

./fuzz -dict=./format.dict corpus/

Generating a Dictionary

From header files:

grep -o '".*"' header.h > header.dict

From man pages:

man curl | grep -oP '^\s*(--|-)\K\S+' | sed 's/[,.]$//' | sed 's/^/"&/; s/$/&"/' | sort -u > man.dict

From binary strings:

strings ./binary | sed 's/^/"&/; s/$/&"/' > strings.dict

Using LLMs: Ask ChatGPT or similar to generate a dictionary for your format (e.g., "Generate a libFuzzer dictionary for a JSON parser").

See Also: For advanced dictionary generation, format-specific dictionaries, and dictionary optimization strategies, see the fuzzing-dictionaries technique skill.

Coverage Analysis

While libFuzzer shows basic coverage stats (cov: N), detailed coverage analysis requires additional tools.

Source-Based Coverage

1. Recompile with coverage instrumentation:

clang++ -fsanitize=fuzzer -fprofile-instr-generate -fcoverage-mapping harness.cc target.cc -o fuzz

2. Run fuzzer to collect coverage:

LLVM_PROFILE_FILE="coverage-%p.profraw" ./fuzz -runs=10000 corpus/

3. Merge coverage data:

llvm-profdata merge -sparse coverage-*.profraw -o coverage.profdata

4. Generate coverage report:

llvm-cov show ./fuzz -instr-profile=coverage.profdata

5. Generate HTML report:

llvm-cov show ./fuzz -instr-profile=coverage.profdata -format=html > coverage.html

Improving Coverage

Tips:

  • Provide better seed inputs in corpus
  • Use dictionaries for format-aware fuzzing
  • Check if harness properly exercises target
  • Consider structure-aware fuzzing for complex formats
  • Run longer campaigns (days/weeks)

See Also: For detailed coverage analysis techniques, identifying coverage gaps, systematic coverage improvement, and comparing coverage across fuzzers, see the coverage-analysis technique skill.

Sanitizer Integration

AddressSanitizer (ASan)

ASan detects memory errors like buffer overflows and use-after-free bugs. Highly recommended for fuzzing.

Enable ASan:

clang++ -fsanitize=fuzzer,address -g -O2 -U_FORTIFY_SOURCE harness.cc target.cc -o fuzz

Example ASan output:

==1276163==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6020000c4ab1
WRITE of size 1 at 0x6020000c4ab1 thread T0
    #0 0x55555568631a in check_buf(char*, unsigned long) main.cc:13:25
    #1 0x5555556860bf in LLVMFuzzerTestOneInput harness.cc:7:3

Configure ASan with environment variables:

ASAN_OPTIONS=verbosity=1:abort_on_error=1 ./fuzz corpus/

Important flags:

  • verbosity=1: Show ASan is active
  • detect_leaks=0: Disable leak detection (leaks reported at end)
  • abort_on_error=1: Call abort() instead of _exit() on errors

Drawbacks:

  • 2-4x slowdown
  • Requires ~20TB virtual memory (disable memory limits: -rss_limit_mb=0)
  • Best supported on Linux

See Also: For comprehensive ASan configuration, common pitfalls, symbolization, and combining with other sanitizers, see the address-sanitizer technique skill.

UndefinedBehaviorSanitizer (UBSan)

UBSan detects undefined behavior like integer overflow, null pointer dereference, etc.

Enable UBSan:

clang++ -fsanitize=fuzzer,undefined -g -O2 harness.cc target.cc -o fuzz

Combine with ASan:

clang++ -fsanitize=fuzzer,address,undefined -g -O2 harness.cc target.cc -o fuzz

MemorySanitizer (MSan)

MSan detects uninitialized memory reads. More complex to use (requires rebuilding all dependencies).

clang++ -fsanitize=fuzzer,memory -g -O2 harness.cc target.cc -o fuzz

Common Sanitizer Issues

IssueSolution
ASan slows fuzzing too muchUse -fsanitize-recover=address for non-fatal errors
Out of memorySet ASAN_OPTIONS=rss_limit_mb=0 or -rss_limit_mb=0
Stack exhaustionIncrease stack size: ASAN_OPTIONS=stack_size=8388608
False positives with _FORTIFY_SOURCEUse -U_FORTIFY_SOURCE flag
MSan reports in dependenciesRebuild all dependencies with -fsanitize=memory

Real-World Examples

Example 1: Fuzzing libpng

libpng is a widely-used library for reading/writing PNG images. Bugs can lead to security issues.

1. Get source code:

curl -L -O https://downloads.sourceforge.net/project/libpng/libpng16/1.6.37/libpng-1.6.37.tar.xz
tar xf libpng-1.6.37.tar.xz
cd libpng-1.6.37/

2. Install dependencies:

apt install zlib1g-dev

3. Compile with fuzzing instrumentation:

export CC=clang CFLAGS="-fsanitize=fuzzer-no-link -fsanitize=address"
export CXX=clang++ CXXFLAGS="$CFLAGS"
./configure --enable-shared=no
make

4. Get a harness (or write your own):

curl -O https://raw.githubusercontent.com/glennrp/libpng/f8e5fa92b0e37ab597616f554bee254157998227/contrib/oss-fuzz/libpng_read_fuzzer.cc

5. Prepare corpus and dictionary:

mkdir corpus/
curl -o corpus/input.png https://raw.githubusercontent.com/glennrp/libpng/acfd50ae0ba3198ad734e5d4dec2b05341e50924/contrib/pngsuite/iftp1n3p08.png
curl -O https://raw.githubusercontent.com/glennrp/libpng/2fff013a6935967960a5ae626fc21432807933dd/contrib/oss-fuzz/png.dict

6. Link and compile fuzzer:

clang++ -fsanitize=fuzzer -fsanitize=address libpng_read_fuzzer.cc .libs/libpng16.a -lz -o fuzz

7. Run fuzzing campaign:

./fuzz -close_fd_mask=3 -dict=./png.dict corpus/

Example 2: Simple Division Bug

Harness that finds a division-by-zero bug:

#include <stdint.h>
#include <stddef.h>

double divide(uint32_t numerator, uint32_t denominator) {
    // Bug: No check if denominator is zero
    return numerator / denominator;
}

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    if(size != 2 * sizeof(uint32_t)) {
        return 0;
    }

    uint32_t numerator = *(uint32_t*)(data);
    uint32_t denominator = *(uint32_t*)(data + sizeof(uint32_t));

    divide(numerator, denominator);

    return 0;
}

Compile and fuzz:

clang++ -fsanitize=fuzzer harness.cc -o fuzz
./fuzz

The fuzzer will quickly find inputs causing a crash.

Advanced Usage

Tips and Tricks

TipWhy It Helps
Start with single-core, switch to AFL++ for multi-corelibFuzzer harnesses work with AFL++
Use dictionaries for structured formats10-100x faster bug discovery
Close file descriptors with -close_fd_mask=3Speed boost if SUT writes output
Set reasonable -max_lenPrevents wasted time on huge inputs
Run for days/weeks, not minutesCoverage plateaus take time to break
Use seed corpus from test suitesStarts fuzzing from valid inputs

Structure-Aware Fuzzing

For highly structured inputs (e.g., complex protocols, file formats), use libprotobuf-mutator:

  • Define input structure using Protocol Buffers
  • libFuzzer mutates protobuf messages (structure-preserving mutations)
  • Harness converts protobuf to native format

See structure-aware fuzzing documentation for details.

Custom Mutators

libFuzzer allows custom mutators for specialized fuzzing:

extern "C" size_t LLVMFuzzerCustomMutator(uint8_t *Data, size_t Size,
                                          size_t MaxSize, unsigned int Seed) {
    // Custom mutation logic
    return new_size;
}

extern "C" size_t LLVMFuzzerCustomCrossOver(const uint8_t *Data1, size_t Size1,
                                            const uint8_t *Data2, size_t Size2,
                                            uint8_t *Out, size_t MaxOutSize,
                                            unsigned int Seed) {
    // Custom crossover logic
    return new_size;
}

Performance Tuning

SettingImpact
-close_fd_mask=3Closes stdout/stderr, speeds up fuzzing
-max_len=<reasonable_size>Avoids wasting time on huge inputs
-timeout=<seconds>Detects hangs, prevents stuck executions
Disable ASan for baseline2-4x speed boost (but misses memory bugs)
Use -jobs and -workersLimited multi-core support
Run on LinuxBest platform support and performance

Troubleshooting

ProblemCauseSolution
No crashes found after hoursPoor corpus, low coverageAdd seed inputs, use dictionary, check harness
Very slow executions/sec (<100)Target too complex, excessive loggingOptimize target, use -close_fd_mask=3, reduce logging
Out of memoryASan's 20TB virtual memorySet -rss_limit_mb=0 to disable RSS limit
Fuzzer stops after first crashDefault behaviorUse -fork=1 -ignore_crashes=1 to continue
Can't reproduce crashNon-determinism in harness/targetRemove random number generation, global state
Linking errors with -fsanitize=fuzzerMissing libFuzzer runtimeEnsure using Clang, check LLVM installation
GCC project won't compile with ClangGCC-specific codeSwitch to AFL++ with gcc_plugin instead
Coverage not improvingCorpus plateauRun longer, add dictionary, improve seeds, check coverage report
Crashes but ASan doesn't triggerMemory error not detected without ASanRecompile with -fsanitize=address

Related Skills

Technique Skills

SkillUse Case
fuzz-harness-writingDetailed guidance on writing effective harnesses, structure-aware fuzzing, and FuzzedDataProvider usage
address-sanitizerMemory error detection configuration, ASAN_OPTIONS, and troubleshooting
undefined-behavior-sanitizerDetecting undefined behavior during fuzzing
coverage-analysisMeasuring fuzzing effectiveness and identifying untested code paths
fuzzing-corpusBuilding and managing seed corpora, corpus minimization strategies
fuzzing-dictionariesCreating format-specific dictionaries for faster bug discovery

Related Fuzzers

SkillWhen to Consider
aflppWhen you need serious multi-core fuzzing, or when libFuzzer coverage plateaus
honggfuzzWhen you want hardware-based coverage feedback on Linux
libaflWhen building custom fuzzers or conducting fuzzing research

Resources

Official Documentation

Advanced Topics

Example Projects

OSS-Fuzz

OSS-Fuzz is an open-source project developed by Google that provides free distributed infrastructure for continuous fuzz testing. It streamlines the fuzzing process and facilitates simpler modifications. While only select projects are accepted into OSS-Fuzz, the project's core is open-source, allowing anyone to host their own instance for private projects.

Overview

OSS-Fuzz provides a simple CLI framework for building and starting harnesses or calculating their coverage. Additionally, OSS-Fuzz can be used as a service that hosts static web pages generated from fuzzing outputs such as coverage information.

Key Concepts

ConceptDescription
helper.pyCLI script for building images, building fuzzers, and running harnesses locally
Base ImagesHierarchical Docker images providing build dependencies and compilers
project.yamlConfiguration file defining project metadata for OSS-Fuzz enrollment
DockerfileProject-specific image with build dependencies
build.shScript that builds fuzzing harnesses for your project
Criticality ScoreMetric used by OSS-Fuzz team to evaluate project acceptance

When to Apply

Apply this technique when:

  • Setting up continuous fuzzing for an open-source project
  • Need distributed fuzzing infrastructure without managing servers
  • Want coverage reports and bug tracking integrated with fuzzing
  • Testing existing OSS-Fuzz harnesses locally
  • Reproducing crashes from OSS-Fuzz bug reports

Skip this technique when:

  • Project is closed-source (unless hosting your own OSS-Fuzz instance)
  • Project doesn't meet OSS-Fuzz's criticality score threshold
  • Need proprietary or specialized fuzzing infrastructure
  • Fuzzing simple scripts that don't warrant infrastructure

Quick Reference

TaskCommand
Clone OSS-Fuzzgit clone https://github.com/google/oss-fuzz
Build project imageuv run --no-project python infra/helper.py build_image --pull <project>
Build fuzzers with ASanuv run --no-project python infra/helper.py build_fuzzers --sanitizer=address <project>
Run specific harnessuv run --no-project python infra/helper.py run_fuzzer <project> <harness>
Generate coverage reportuv run --no-project python infra/helper.py coverage <project>
Check helper.py optionsuv run --no-project python infra/helper.py --help

OSS-Fuzz Project Components

OSS-Fuzz provides several publicly available tools and web interfaces:

Bug Tracker

The bug tracker allows you to:

  • Check bugs from specific projects (initially visible only to maintainers, later made public)
  • Create new issues and comment on existing ones
  • Search for similar bugs across all projects to understand issues

Build Status System

The build status system helps track:

  • Build statuses of all included projects
  • Date of last successful build
  • Build failures and their duration

Fuzz Introspector

Fuzz Introspector displays:

  • Coverage data for projects enrolled in OSS-Fuzz
  • Hit frequency for covered code
  • Performance analysis and blocker identification

Read this case study for examples and explanations.

Step-by-Step: Running a Single Harness

You don't need to host the whole OSS-Fuzz platform to use it. The helper script makes it easy to run individual harnesses locally.

Step 1: Clone OSS-Fuzz

git clone https://github.com/google/oss-fuzz
cd oss-fuzz
uv run --no-project python infra/helper.py --help

Step 2: Build Project Image

uv run --no-project python infra/helper.py build_image --pull <project-name>

This downloads and builds the base Docker image for the project.

Step 3: Build Fuzzers with Sanitizers

uv run --no-project python infra/helper.py build_fuzzers --sanitizer=address <project-name>

Sanitizer options:

Note: Fuzzers are built to /build/out/<project-name>/ containing the harness executables, dictionaries, corpus, and crash files.

Step 4: Run the Fuzzer

uv run --no-project python infra/helper.py run_fuzzer <project-name> <harness-name> [<fuzzer-args>]

The helper script automatically runs any missed steps if you skip them.

Step 5: Coverage Analysis (Optional)

First, install gsutil (skip gcloud initialization).

uv run --no-project python infra/helper.py build_fuzzers --sanitizer=coverage <project-name>
uv run --no-project python infra/helper.py coverage <project-name>

Use --no-corpus-download to use only local corpus. The command generates and hosts a coverage report locally.

See official OSS-Fuzz documentation for details.

Common Patterns

Pattern: Running irssi Example

Use Case: Testing OSS-Fuzz setup with a simple enrolled project

# Clone and navigate to OSS-Fuzz
git clone https://github.com/google/oss-fuzz
cd oss-fuzz

# Build and run irssi fuzzer
uv run --no-project python infra/helper.py build_image --pull irssi
uv run --no-project python infra/helper.py build_fuzzers --sanitizer=address irssi
uv run --no-project python infra/helper.py run_fuzzer irssi irssi-fuzz

Expected Output:

INFO:__main__:Running: docker run --rm --privileged --shm-size=2g --platform linux/amd64 -i -e FUZZING_ENGINE=libfuzzer -e SANITIZER=address -e RUN_FUZZER_MODE=interactive -e HELPER=True -v /private/tmp/oss-fuzz/build/out/irssi:/out -t gcr.io/oss-fuzz-base/base-runner run_fuzzer irssi-fuzz.
Using seed corpus: irssi-fuzz_seed_corpus.zip
/out/irssi-fuzz -rss_limit_mb=2560 -timeout=25 /tmp/irssi-fuzz_corpus -max_len=2048 < /dev/null
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 1531341664
INFO: Loaded 1 modules   (95687 inline 8-bit counters): 95687 [0x1096c80, 0x10ae247),
INFO: Loaded 1 PC tables (95687 PCs): 95687 [0x10ae248,0x1223eb8),
INFO:      719 files found in /tmp/irssi-fuzz_corpus
INFO: seed corpus: files: 719 min: 1b max: 170106b total: 367969b rss: 48Mb
#720        INITED cov: 409 ft: 1738 corp: 640/163Kb exec/s: 0 rss: 62Mb
#762        REDUCE cov: 409 ft: 1738 corp: 640/163Kb lim: 2048 exec/s: 0 rss: 63Mb L: 236/2048 MS: 2 ShuffleBytes-EraseBytes-

Pattern: Enrolling a New Project

Use Case: Adding your project to OSS-Fuzz (or private instance)

Create three files in projects/<your-project>/:

1. project.yaml - Project metadata:

homepage: "https://github.com/yourorg/yourproject"
language: c++
primary_contact: "your-email@example.com"
main_repo: "https://github.com/yourorg/yourproject"
fuzzing_engines:
  - libfuzzer
sanitizers:
  - address
  - undefined

2. Dockerfile - Build dependencies:

FROM gcr.io/oss-fuzz-base/base-builder
RUN apt-get update && apt-get install -y \
    autoconf \
    automake \
    libtool \
    pkg-config
RUN git clone --depth 1 https://github.com/yourorg/yourproject
WORKDIR yourproject
COPY build.sh $SRC/

3. build.sh - Build harnesses:

#!/bin/bash -eu
./autogen.sh
./configure --disable-shared
make -j$(nproc)

# Build harnesses
$CXX $CXXFLAGS -std=c++11 -I. \
    $SRC/yourproject/fuzz/harness.cc -o $OUT/harness \
    $LIB_FUZZING_ENGINE ./libyourproject.a

# Copy corpus and dictionary if available
cp $SRC/yourproject/fuzz/corpus.zip $OUT/harness_seed_corpus.zip
cp $SRC/yourproject/fuzz/dictionary.dict $OUT/harness.dict

Docker Images in OSS-Fuzz

Harnesses are built and executed in Docker containers. All projects share a runner image, but each project has its own build image.

Image Hierarchy

Images build on each other in this sequence:

  1. base_image - Specific Ubuntu version
  2. base_clang - Clang compiler; based on base_image
  3. base_builder - Build dependencies; based on base_clang
  4. Your project Docker image - Project-specific dependencies; based on base_builder or language variant

Runner Images (Used Separately)

Advanced Usage

Tips and Tricks

TipWhy It Helps
Don't manually copy source codeProject Dockerfile likely already pulls latest version
Check existing projectsBrowse oss-fuzz/projects for examples
Keep harnesses in separate repoLike curl-fuzzer - cleaner organization
Use specific compiler versionsBase images provide consistent build environment
Install dependencies in DockerfileMay require approval for OSS-Fuzz enrollment

Criticality Score

OSS-Fuzz uses a criticality score to evaluate project acceptance. See this example for how scoring works.

Projects with lower scores may still be added to private OSS-Fuzz instances.

Hosting Your Own Instance

Since OSS-Fuzz is open-source, you can host your own instance for:

  • Private projects not eligible for public OSS-Fuzz
  • Projects with lower criticality scores
  • Custom fuzzing infrastructure needs

Anti-Patterns

Anti-PatternProblemCorrect Approach
Manually pulling source in build.shDoesn't use latest versionLet Dockerfile handle git clone
Copying code to OSS-Fuzz repoHard to maintain, violates separationReference external harness repo
Ignoring base image versionsBuild inconsistenciesUse provided base images and compilers
Skipping local testingWastes CI resourcesUse helper.py locally before PR
Not checking build statusUnnoticed build failuresMonitor build status page regularly

Tool-Specific Guidance

libFuzzer

OSS-Fuzz primarily uses libFuzzer as the fuzzing engine for C/C++ projects.

Harness signature:

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    // Your fuzzing logic
    return 0;
}

Build in build.sh:

$CXX $CXXFLAGS -std=c++11 -I. \
    harness.cc -o $OUT/harness \
    $LIB_FUZZING_ENGINE ./libproject.a

Integration tips:

  • Use $LIB_FUZZING_ENGINE variable provided by OSS-Fuzz
  • Include -fsanitize=fuzzer is handled automatically
  • Link against static libraries when possible

AFL++

OSS-Fuzz supports AFL++ as an alternative fuzzing engine.

Enable in project.yaml:

fuzzing_engines:
  - afl
  - libfuzzer

Integration tips:

  • AFL++ harnesses work alongside libFuzzer harnesses
  • Use persistent mode for better performance
  • OSS-Fuzz handles engine-specific compilation flags

Atheris (Python)

For Python projects with C extensions.

Example from cbor2 integration:

Harness:

import atheris
import sys
import cbor2

@atheris.instrument_func
def TestOneInput(data):
    fdp = atheris.FuzzedDataProvider(data)
    try:
        cbor2.loads(data)
    except (cbor2.CBORDecodeError, ValueError):
        pass

def main():
    atheris.Setup(sys.argv, TestOneInput)
    atheris.Fuzz()

if __name__ == "__main__":
    main()

Build in build.sh:

# allow-legacy-python: build.sh runs inside the oss-fuzz container, where the shims are absent.
pip3 install .
for fuzzer in $(find $SRC -name 'fuzz_*.py'); do
  compile_python_fuzzer $fuzzer
done

Integration tips:

Rust Projects

Enable in project.yaml:

language: rust
fuzzing_engines:
  - libfuzzer
sanitizers:
  - address  # Only AddressSanitizer supported for Rust

Build in build.sh:

cargo fuzz build -O --debug-assertions
cp fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1 $OUT/

Integration tips:

Troubleshooting

IssueCauseSolution
Build fails with missing dependenciesDependencies not in DockerfileAdd apt-get install or equivalent in Dockerfile
Harness crashes immediatelyMissing input validationAdd size checks in harness
Coverage is 0%Harness not reaching target codeVerify harness actually calls target functions
Build timeoutComplex build processOptimize build.sh, consider parallel builds
Sanitizer errors in buildIncompatible flagsUse flags provided by OSS-Fuzz environment variables
Cannot find source codeWrong working directory in DockerfileSet WORKDIR or use absolute paths

Related Skills

Tools That Use This Technique

SkillHow It Applies
libfuzzerPrimary fuzzing engine used by OSS-Fuzz
aflppAlternative fuzzing engine supported by OSS-Fuzz
atherisUsed for fuzzing Python projects in OSS-Fuzz
cargo-fuzzUsed for Rust projects in OSS-Fuzz

Related Techniques

SkillRelationship
coverage-analysisOSS-Fuzz generates coverage reports via helper.py
address-sanitizerDefault sanitizer for OSS-Fuzz projects
fuzz-harness-writingEssential for enrolling projects in OSS-Fuzz
corpus-managementOSS-Fuzz maintains corpus for enrolled projects

Resources

Key External Resources

OSS-Fuzz Official Documentation Comprehensive documentation covering enrollment, harness writing, and troubleshooting for the OSS-Fuzz platform.

Getting Started Guide Step-by-step process for enrolling new projects into OSS-Fuzz, including requirements and approval process.

cbor2 OSS-Fuzz Integration PR Real-world example of enrolling a Python project with C extensions into OSS-Fuzz. Shows:

  • Initial proposal and project introduction
  • Criticality score evaluation
  • Complete implementation (project.yaml, Dockerfile, build.sh, harnesses)

Fuzz Introspector Case Studies Examples and explanations of using Fuzz Introspector to analyze coverage and identify fuzzing blockers.

Video Resources

Check OSS-Fuzz documentation for workshop recordings and tutorials on enrollment and harness development.

Property-Based Testing

An example test asserts one point. A property asserts a rule over the whole input domain and lets the generator hunt for the counterexample. That trade is worth making when the code has an algebraic shape — an inverse, an invariant, an oracle — and not otherwise. Code with no such shape gets example tests; saying so is a valid outcome.

Check first whether the shape is missing or merely buried. A calculation wrapped in I/O, a string built by concatenation, an in-place mutation — each has a property and no seam to assert it through. See references/refactoring.md before concluding there is nothing to assert.

Property catalog

PropertyFormulaWhere it applies
Roundtripdecode(encode(x)) == xSerialization, conversion pairs
Inversef(g(x)) == xencrypt/decrypt, compress/decompress
Oraclenew(x) == reference(x)Optimization, refactoring, reimplementation
Idempotencef(f(x)) == f(x)Normalization, formatting, sorting
InvariantHolds before and afterAny transformation, contract state
Easy to verifyis_sorted(sort(x))Complex algorithms with cheap checkers
Commutativityf(a, b) == f(b, a)Binary and set operations
Associativityf(f(a,b), c) == f(a, f(b,c))Combining operations
Identityf(x, e) == xOperations with a neutral element

Strength ordering, weakest to strongest: no crash → type preservation → invariant → idempotence → roundtrip / oracle.

Assert the strongest property the code supports. "No crash" alone rarely justifies the dependency — if that is all you can find, either a small rearrangement exposes something stronger, or the honest report is that this code is a poor PBT candidate. Rule out the first before settling for the second.

The two ways a property test asserts nothing

  • Tautology. assert add(a, b) == a + b restates the implementation; no bug they share can fail it. Pick a property that constrains the function without recomputing it. Note the exception: f(x) == f(x) is a genuine determinism property when f is not obviously pure — serializers over dicts or sets, hashing, anything reading the clock.
  • Vacuity. assume() that filters out nearly every input passes without exercising anything, and self-contradictory assume() passes having run zero cases. Push constraints into the strategy so the generator produces valid inputs directly.

Where to look next

Load the one that matches the task in front of you:

TaskFile
Writing new tests, designing strategiesreferences/generating.md
The code has no property to assert yetreferences/refactoring.md
Reviewing existing property testsreferences/reviewing.md
A property test just failedreferences/interpreting-failures.md
Library choice, Echidna and Medusareferences/libraries.md

Introducing PBT to a project that lacks it

If the project already uses a PBT library, just write the tests in it. If it does not, adding one is a dependency decision that belongs to the user — offer it once with the specific property you would write, and take the answer either way.

Ruzzy

Ruzzy is a coverage-guided fuzzer for Ruby built on libFuzzer. It enables fuzzing both pure Ruby code and Ruby C extensions with sanitizer support for detecting memory corruption and undefined behavior.

When to Use

Ruzzy is currently the only production-ready coverage-guided fuzzer for Ruby.

Choose Ruzzy when:

  • Fuzzing Ruby applications or libraries
  • Testing Ruby C extensions for memory safety issues
  • You need coverage-guided fuzzing for Ruby code
  • Working with Ruby gems that have native extensions

Quick Start

Set up environment:

export ASAN_OPTIONS="allocator_may_return_null=1:detect_leaks=0:use_sigaltstack=0"

Test with the included toy example:

LD_PRELOAD=$(ruby -e 'require "ruzzy"; print Ruzzy::ASAN_PATH') \
    ruby -e 'require "ruzzy"; Ruzzy.dummy'

This should quickly find a crash demonstrating that Ruzzy is working correctly.

Installation

Platform Support

Ruzzy supports Linux x86-64 and AArch64/ARM64. For macOS or Windows, use the Dockerfile or development environment.

Prerequisites

  • Linux x86-64 or AArch64/ARM64
  • Recent version of clang (tested back to 14.0.0, latest release recommended)
  • Ruby with gem installed

Installation Command

Install Ruzzy with clang compiler flags:

MAKE="make --environment-overrides V=1" \
CC="/path/to/clang" \
CXX="/path/to/clang++" \
LDSHARED="/path/to/clang -shared" \
LDSHAREDXX="/path/to/clang++ -shared" \
    gem install ruzzy

Environment variables explained:

  • MAKE: Overrides make to respect subsequent environment variables
  • CC, CXX, LDSHARED, LDSHAREDXX: Ensure proper clang binaries are used for latest features

Troubleshooting Installation

If installation fails, enable debug output:

RUZZY_DEBUG=1 gem install --verbose ruzzy

Verification

Verify installation by running the toy example (see Quick Start section).

Writing a Harness

Fuzzing Pure Ruby Code

Pure Ruby fuzzing requires two scripts due to Ruby interpreter implementation details.

Tracer script (test_tracer.rb):

# frozen_string_literal: true

require 'ruzzy'

Ruzzy.trace('test_harness.rb')

Harness script (test_harness.rb):

# frozen_string_literal: true

require 'ruzzy'

def fuzzing_target(input)
  # Your code to fuzz here
  if input.length == 4
    if input[0] == 'F'
      if input[1] == 'U'
        if input[2] == 'Z'
          if input[3] == 'Z'
            raise
          end
        end
      end
    end
  end
end

test_one_input = lambda do |data|
  fuzzing_target(data)
  return 0
end

Ruzzy.fuzz(test_one_input)

Run with:

LD_PRELOAD=$(ruby -e 'require "ruzzy"; print Ruzzy::ASAN_PATH') \
    ruby test_tracer.rb

Fuzzing Ruby C Extensions

C extensions can be fuzzed with a single harness file, no tracer needed.

Example harness for msgpack (fuzz_msgpack.rb):

# frozen_string_literal: true

require 'msgpack'
require 'ruzzy'

test_one_input = lambda do |data|
  begin
    MessagePack.unpack(data)
  rescue Exception
    # We're looking for memory corruption, not Ruby exceptions
  end
  return 0
end

Ruzzy.fuzz(test_one_input)

Run with:

LD_PRELOAD=$(ruby -e 'require "ruzzy"; print Ruzzy::ASAN_PATH') \
    ruby fuzz_msgpack.rb

Harness Rules

DoDon't
Catch Ruby exceptions if testing C extensionsLet Ruby exceptions crash the fuzzer
Return 0 from test_one_input lambdaReturn other values
Keep harness deterministicUse randomness or time-based logic
Use tracer script for pure RubySkip tracer for pure Ruby code

See Also: For detailed harness writing techniques, patterns for handling complex inputs, and advanced strategies, see the fuzz-harness-writing technique skill.

Compilation

Installing Gems with Sanitizers

When installing Ruby gems with C extensions for fuzzing, compile with sanitizer flags:

MAKE="make --environment-overrides V=1" \
CC="/path/to/clang" \
CXX="/path/to/clang++" \
LDSHARED="/path/to/clang -shared" \
LDSHAREDXX="/path/to/clang++ -shared" \
CFLAGS="-fsanitize=address,fuzzer-no-link -fno-omit-frame-pointer -fno-common -fPIC -g" \
CXXFLAGS="-fsanitize=address,fuzzer-no-link -fno-omit-frame-pointer -fno-common -fPIC -g" \
    gem install <gem-name>

Build Flags

FlagPurpose
-fsanitize=address,fuzzer-no-linkEnable AddressSanitizer and fuzzer instrumentation
-fno-omit-frame-pointerImprove stack trace quality
-fno-commonBetter compatibility with sanitizers
-fPICPosition-independent code for shared libraries
-gInclude debug symbols

Running Campaigns

Environment Setup

Before running any fuzzing campaign, set ASAN_OPTIONS:

export ASAN_OPTIONS="allocator_may_return_null=1:detect_leaks=0:use_sigaltstack=0"

Options explained:

  1. allocator_may_return_null=1: Skip common low-impact allocation failures (DoS)
  2. detect_leaks=0: Ruby interpreter leaks data, ignore these for now
  3. use_sigaltstack=0: Ruby recommends disabling sigaltstack with ASan

Basic Run

LD_PRELOAD=$(ruby -e 'require "ruzzy"; print Ruzzy::ASAN_PATH') \
    ruby harness.rb

Note: LD_PRELOAD is required for sanitizer injection. Unlike ASAN_OPTIONS, do not export it as it may interfere with other programs.

With Corpus

LD_PRELOAD=$(ruby -e 'require "ruzzy"; print Ruzzy::ASAN_PATH') \
    ruby harness.rb /path/to/corpus

Passing libFuzzer Options

All libFuzzer options can be passed as arguments:

LD_PRELOAD=$(ruby -e 'require "ruzzy"; print Ruzzy::ASAN_PATH') \
    ruby harness.rb /path/to/corpus -max_len=1024 -timeout=10

See libFuzzer options for full reference.

Reproducing Crashes

Re-run a crash case by passing the crash file:

LD_PRELOAD=$(ruby -e 'require "ruzzy"; print Ruzzy::ASAN_PATH') \
    ruby harness.rb ./crash-253420c1158bc6382093d409ce2e9cff5806e980

Interpreting Output

OutputMeaning
INFO: Running with entropic power scheduleFuzzing campaign started
ERROR: AddressSanitizer: heap-use-after-freeMemory corruption detected
SUMMARY: libFuzzer: fuzz target exitedRuby exception occurred
artifact_prefix='./'; Test unit written to ./crash-*Crash input saved
Base64: ...Base64 encoding of crash input

Sanitizer Integration

AddressSanitizer (ASan)

Ruzzy includes a pre-compiled AddressSanitizer library:

LD_PRELOAD=$(ruby -e 'require "ruzzy"; print Ruzzy::ASAN_PATH') \
    ruby harness.rb

Use ASan for detecting:

  • Heap buffer overflows
  • Stack buffer overflows
  • Use-after-free
  • Double-free
  • Memory leaks (disabled by default in Ruzzy)

UndefinedBehaviorSanitizer (UBSan)

Ruzzy also includes UBSan:

LD_PRELOAD=$(ruby -e 'require "ruzzy"; print Ruzzy::UBSAN_PATH') \
    ruby harness.rb

Use UBSan for detecting:

  • Signed integer overflow
  • Null pointer dereferences
  • Misaligned memory access
  • Division by zero

Common Sanitizer Issues

IssueSolution
Ruby interpreter leak warningsUse ASAN_OPTIONS=detect_leaks=0
Sigaltstack conflictsUse ASAN_OPTIONS=use_sigaltstack=0
Allocation failure spamUse ASAN_OPTIONS=allocator_may_return_null=1
LD_PRELOAD interferes with toolsDon't export it; set inline with ruby command

See Also: For detailed sanitizer configuration, common issues, and advanced flags, see the address-sanitizer and undefined-behavior-sanitizer technique skills.

Real-World Examples

Example: msgpack-ruby

Fuzzing the msgpack MessagePack parser for memory corruption.

Install with sanitizers:

MAKE="make --environment-overrides V=1" \
CC="/path/to/clang" \
CXX="/path/to/clang++" \
LDSHARED="/path/to/clang -shared" \
LDSHAREDXX="/path/to/clang++ -shared" \
CFLAGS="-fsanitize=address,fuzzer-no-link -fno-omit-frame-pointer -fno-common -fPIC -g" \
CXXFLAGS="-fsanitize=address,fuzzer-no-link -fno-omit-frame-pointer -fno-common -fPIC -g" \
    gem install msgpack

Harness (fuzz_msgpack.rb):

# frozen_string_literal: true

require 'msgpack'
require 'ruzzy'

test_one_input = lambda do |data|
  begin
    MessagePack.unpack(data)
  rescue Exception
    # We're looking for memory corruption, not Ruby exceptions
  end
  return 0
end

Ruzzy.fuzz(test_one_input)

Run:

export ASAN_OPTIONS="allocator_may_return_null=1:detect_leaks=0:use_sigaltstack=0"
LD_PRELOAD=$(ruby -e 'require "ruzzy"; print Ruzzy::ASAN_PATH') \
    ruby fuzz_msgpack.rb

Example: Pure Ruby Target

Fuzzing pure Ruby code with a custom parser.

Tracer (test_tracer.rb):

# frozen_string_literal: true

require 'ruzzy'

Ruzzy.trace('test_harness.rb')

Harness (test_harness.rb):

# frozen_string_literal: true

require 'ruzzy'
require_relative 'my_parser'

test_one_input = lambda do |data|
  begin
    MyParser.parse(data)
  rescue StandardError
    # Expected exceptions from malformed input
  end
  return 0
end

Ruzzy.fuzz(test_one_input)

Run:

export ASAN_OPTIONS="allocator_may_return_null=1:detect_leaks=0:use_sigaltstack=0"
LD_PRELOAD=$(ruby -e 'require "ruzzy"; print Ruzzy::ASAN_PATH') \
    ruby test_tracer.rb

Troubleshooting

ProblemCauseSolution
Installation failsWrong clang version or pathVerify clang path, use clang 14.0.0+
cannot open shared object fileLD_PRELOAD not setSet LD_PRELOAD inline with ruby command
Fuzzer immediately exitsMissing corpus directoryCreate corpus directory or pass as argument
No coverage progressPure Ruby needs tracerUse tracer script for pure Ruby code
Leak detection spamRuby interpreter leaksSet ASAN_OPTIONS=detect_leaks=0
Installation debug neededCompilation errorsUse RUZZY_DEBUG=1 gem install --verbose ruzzy

Related Skills

Technique Skills

SkillUse Case
fuzz-harness-writingDetailed guidance on writing effective harnesses
address-sanitizerMemory error detection during fuzzing
undefined-behavior-sanitizerDetecting undefined behavior in C extensions
libfuzzerUnderstanding libFuzzer options (Ruzzy is built on libFuzzer)

Related Fuzzers

SkillWhen to Consider
libfuzzerWhen fuzzing Ruby C extension code directly in C/C++
aflppAlternative approach for fuzzing Ruby by instrumenting Ruby interpreter

Resources

Key External Resources

Introducing Ruzzy, a coverage-guided Ruby fuzzer Official Trail of Bits blog post announcing Ruzzy, covering motivation, architecture, and initial results.

Ruzzy GitHub Repository Source code, additional examples, and development instructions.

libFuzzer Documentation Since Ruzzy is built on libFuzzer, understanding libFuzzer options and behavior is valuable.

Fuzzing Ruby C extensions Detailed guide on fuzzing C extensions with compilation flags and examples.

Fuzzing pure Ruby code Detailed guide on the tracer pattern required for pure Ruby fuzzing.

Testing Handbook Skill Generator

Generate and maintain Claude Code skills from the Trail of Bits Testing Handbook.

When to Use

Invoke this skill when:

  • Creating new security testing skills from handbook content
  • User mentions "testing handbook", "appsec.guide", or asks about generating skills
  • Bulk skill generation or refresh is needed

Do NOT use for:

  • General security testing questions (use the generated skills)
  • Non-handbook skill creation

Handbook Location

The skill needs the Testing Handbook repository. See discovery.md for full details.

Quick reference: Check ./testing-handbook, ../testing-handbook, ~/testing-handbook → ask user → clone as last resort.

Repository: https://github.com/trailofbits/testing-handbook

Workflow Overview

Phase 0: Setup              Phase 1: Discovery
┌─────────────────┐        ┌─────────────────┐
│ Locate handbook │   →    │ Analyze handbook│
│ - Find or clone │        │ - Scan sections │
│ - Confirm path  │        │ - Classify types│
└─────────────────┘        └─────────────────┘
         ↓                          ↓
Phase 3: Generation        Phase 2: Planning
┌─────────────────┐        ┌─────────────────┐
│ TWO-PASS GEN    │   ←    │ Generate plan   │
│ Pass 1: Content │        │ - New skills    │
│ Pass 2: X-refs  │        │ - Updates       │
│ - Write to gen/ │        │ - Present user  │
└─────────────────┘        └─────────────────┘
         ↓
Phase 4: Testing           Phase 5: Finalize
┌─────────────────┐        ┌─────────────────┐
│ Validate skills │   →    │ Post-generation │
│ - Run validator │        │ - Update README │
│ - Test activation│       │ - Update X-refs │
│ - Fix issues    │        │ - Self-improve  │
└─────────────────┘        └─────────────────┘

Scope Restrictions

ONLY modify these locations:

  • plugins/testing-handbook-skills/skills/[skill-name]/* - Generated skills (as siblings to testing-handbook-generator)
  • plugins/testing-handbook-skills/skills/testing-handbook-generator/* - Self-improvement
  • Repository root README.md - Add generated skills to table

NEVER modify or analyze:

  • Other plugins (plugins/property-based-testing/, plugins/static-analysis/, etc.)
  • Other skills outside this plugin

Do not scan or pull into context any skills outside of testing-handbook-skills/. Generate skills based solely on handbook content and resources referenced from it.

Quick Reference

Section → Skill Type Mapping

Handbook SectionSkill TypeTemplate
/static-analysis/[tool]/Tool Skilltool-skill.md
/fuzzing/[lang]/[fuzzer]/Fuzzer Skillfuzzer-skill.md
/fuzzing/techniques/Technique Skilltechnique-skill.md
/crypto/[tool]/Domain Skilldomain-skill.md
/web/[tool]/Tool Skilltool-skill.md

Skill Candidate Signals

SignalIndicates
_index.md with bookCollapseSection: trueMajor tool/topic
Numbered files (00-, 10-, 20-)Structured content
techniques/ subsectionMethodology content
99-resources.md or 91-resources.mdHas external links

Exclusion Signals

SignalAction
draft: true in frontmatterSkip section
Empty directorySkip section
Template/placeholder fileSkip section
GUI-only tool (e.g., web/burp/)Skip section (Claude cannot operate GUI tools)

Decision Tree

Starting skill generation?

├─ Need to analyze handbook and build plan?
│  └─ Read: discovery.md
│     (Handbook analysis methodology, plan format)
│
├─ Spawning skill generation agents?
│  └─ Read: agent-prompt.md
│     (Full prompt template, variable reference, validation checklist)
│
├─ Generating a specific skill type?
│  └─ Read appropriate template:
│     ├─ Tool (Semgrep, CodeQL) → templates/tool-skill.md
│     ├─ Fuzzer (libFuzzer, AFL++) → templates/fuzzer-skill.md
│     ├─ Technique (harness, coverage) → templates/technique-skill.md
│     └─ Domain (crypto, web) → templates/domain-skill.md
│
├─ Validating generated skills?
│  └─ Run: scripts/validate-skills.py
│     Then read: testing.md for activation testing
│
├─ Finalizing after generation?
│  └─ See: Post-Generation Tasks below
│     (Update main README, update Skills Cross-Reference, self-improvement)
│
└─ Quick generation from specific section?
   └─ Use Quick Reference above, apply template directly

Two-Pass Generation (Phase 3)

Generation uses a two-pass approach to solve forward reference problems (skills referencing other skills that don't exist yet).

Pass 1: Content Generation (Parallel)

Generate all skills in parallel without the Related Skills section:

Pass 1 - Generating 5 skills in parallel:
├─ Agent 1: libfuzzer (fuzzer) → skills/libfuzzer/SKILL.md
├─ Agent 2: aflpp (fuzzer) → skills/aflpp/SKILL.md
├─ Agent 3: semgrep (tool) → skills/semgrep/SKILL.md
├─ Agent 4: harness-writing (technique) → skills/harness-writing/SKILL.md
└─ Agent 5: wycheproof (domain) → skills/wycheproof/SKILL.md

Each agent uses: pass=1 (content only, Related Skills left empty)

Pass 1 agents:

  • Generate all sections EXCEPT Related Skills
  • Leave a placeholder: ## Related Skills\n\n<!-- PASS2: populate after all skills exist -->
  • Output report includes references: DEFERRED

Pass 2: Cross-Reference Population (Sequential)

After all Pass 1 agents complete, run Pass 2 to populate Related Skills:

Pass 2 - Populating cross-references:
├─ Read all generated skill names from skills/*/SKILL.md
├─ For each skill, determine related skills based on:
│   ├─ related_sections from discovery (handbook structure)
│   ├─ Skill type relationships (fuzzers → techniques)
│   └─ Explicit mentions in content
└─ Update each SKILL.md's Related Skills section

Pass 2 process:

  1. Collect all generated skill names: ls -d skills/*/SKILL.md
  2. For each skill, identify related skills using the mapping from discovery
  3. Edit each SKILL.md to replace the placeholder with actual links
  4. Validate cross-references exist (no broken links)

Agent Prompt Template

See agent-prompt.md for the full prompt template with:

  • Variable substitution reference (including pass variable)
  • Pre-write validation checklist
  • Hugo shortcode conversion rules
  • Line count splitting rules
  • Error handling guidance
  • Output report format

Collecting Results

After Pass 1: Aggregate output reports, verify all skills generated. After Pass 2: Run validator to check cross-references.

Handling Agent Failures

If an agent fails or produces invalid output:

Failure TypeDetectionRecovery Action
Agent crashedNo output reportRe-run single agent with same inputs
Validation failedOutput report shows errorsCheck gaps/warnings, manually patch or re-run
Wrong skill typeContent doesn't match templateRe-run with corrected type parameter
Missing contentOutput report lists gapsAccept if minor, or provide additional related_sections
Pass 2 broken refValidator shows missing skillCheck if skill was skipped, update reference

Important: Do NOT re-run the entire parallel batch for a single agent failure. Fix individual failures independently.

Single-Skill Regeneration

To regenerate a single skill without re-running the entire batch:

# Regenerate single skill (Pass 1 - content only)
"Use testing-handbook-generator to regenerate the {skill-name} skill from section {section_path}"

# Example:
"Use testing-handbook-generator to regenerate the libfuzzer skill from section fuzzing/c-cpp/10-libfuzzer"

Regeneration workflow:

  1. Re-read the handbook section for fresh content
  2. Apply the appropriate template
  3. Write to skills/{skill-name}/SKILL.md (overwrites existing)
  4. Re-run Pass 2 for that skill only to update cross-references
  5. Run validator on the single skill: uv run scripts/validate-skills.py --skill {skill-name}

Output Location

Generated skills are written to:

skills/[skill-name]/SKILL.md

Each skill gets its own directory for potential supporting files (as siblings to testing-handbook-generator).

Quality Checklist

Before delivering generated skills:

  • All handbook sections analyzed (Phase 1)
  • Plan presented to user before generation (Phase 2)
  • Parallel agents launched - one per skill (Phase 3)
  • Templates applied correctly per skill type
  • Validator passes: uv run scripts/validate-skills.py
  • Activation testing passed - see testing.md
  • Main README.md updated with generated skills table
  • README.md Skills Cross-Reference graph updated
  • Self-improvement notes captured
  • User notified with summary

Post-Generation Tasks

1. Update Main README

After generating skills, update the repository's main README.md to list them.

Format: Add generated skills to the same "Available Plugins" table, directly after testing-handbook-skills. Use plain text testing-handbook-generator as the author (no link).

Example:

| Plugin | Description | Author |
|--------|-------------|--------|
| ... other plugins ... |
| [testing-handbook-skills](plugins/testing-handbook-skills/) | Meta-skill that generates skills from the Testing Handbook | Paweł Płatek |
| [libfuzzer](plugins/testing-handbook-skills/skills/libfuzzer/) | Coverage-guided fuzzing with libFuzzer for C/C++ | testing-handbook-generator |
| [aflpp](plugins/testing-handbook-skills/skills/aflpp/) | Multi-core fuzzing with AFL++ | testing-handbook-generator |
| [semgrep](plugins/testing-handbook-skills/skills/semgrep/) | Fast static analysis for finding bugs | testing-handbook-generator |

2. Update Skills Cross-Reference

After generating skills, update the README.md's Skills Cross-Reference section with the mermaid graph showing skill relationships.

Process:

  1. Read each generated skill's SKILL.md and extract its ## Related Skills section
  2. Build the mermaid graph with nodes grouped by skill type (Fuzzers, Techniques, Tools, Domain)
  3. Add edges based on the Related Skills relationships:
    • Solid arrows (-->) for primary technique dependencies
    • Dashed arrows (-.->) for alternative tool suggestions
  4. Replace the existing mermaid code block in README.md

Edge classification:

RelationshipArrow StyleExample
Fuzzer → Technique-->libfuzzer --> harness-writing
Tool → Tool (alternative)-.->semgrep -.-> codeql
Fuzzer → Fuzzer (alternative)-.->libfuzzer -.-> aflpp
Technique → Technique-->harness-writing --> coverage-analysis

Validation: After updating, run validate-skills.py to verify all referenced skills exist.

3. Self-Improvement

After each generation run, reflect on what could improve future runs.

Capture improvements to:

  • Templates (missing sections, better structure)
  • Discovery logic (missed patterns, false positives)
  • Content extraction (shortcodes not handled, formatting issues)

Update process:

  1. Note issues encountered during generation
  2. Identify patterns that caused problems
  3. Update relevant files:
    • SKILL.md - Workflow, decision tree, quick reference updates
    • templates/*.md - Template improvements
    • discovery.md - Detection logic updates
    • testing.md - New validation checks
  4. Document the improvement in commit message

Example self-improvement:

Issue: libFuzzer skill missing sanitizer flags table
Fix: Updated templates/fuzzer-skill.md to include ## Compiler Flags section

Example Usage

Full Discovery and Generation

User: "Generate skills from the testing handbook"

1. Locate handbook (check common locations, ask user, or clone)
2. Read discovery.md for methodology
3. Scan handbook at {handbook_path}/content/docs/
4. Build candidate list with types
5. Present plan to user
6. On approval, generate each skill using appropriate template
7. Validate generated skills
8. Update main README.md with generated skills table
9. Update README.md Skills Cross-Reference graph from Related Skills sections
10. Self-improve: note any template/discovery issues for future runs
11. Report results

Single Section Generation

User: "Create a skill for the libFuzzer section"

1. Read /testing-handbook/content/docs/fuzzing/c-cpp/10-libfuzzer/
2. Identify type: Fuzzer Skill
3. Read templates/fuzzer-skill.md
4. Extract content, apply template
5. Write to skills/libfuzzer/SKILL.md
6. Validate and report

Tips

Do:

  • Always present plan before generating
  • Use appropriate template for skill type
  • Preserve code blocks exactly
  • Validate after generation

Don't:

  • Generate without user approval
  • Skip fetching non-video external resources (use WebFetch)
  • Fetch video URLs (YouTube, Vimeo - titles only)
  • Include handbook images directly
  • Skip validation step
  • Exceed 500 lines per SKILL.md

For first-time use: Start with discovery.md to understand the handbook analysis process.

For template reference: See templates/ directory for skill type templates.

For validation: See testing.md for quality assurance methodology.

Vector Forge

Uses mutation testing to systematically identify gaps in test vector coverage, then generates new test vectors that close those gaps. Measures effectiveness by comparing mutation kill rates before and after.

When to Use

  • Generating test vectors for cryptographic algorithms or protocols
  • Evaluating how well existing test vectors cover an implementation
  • Finding implementation code paths that no test vector exercises
  • Creating Wycheproof-style cross-implementation test vectors
  • Measuring the concrete coverage value of a test vector suite

When NOT to Use

  • No implementations exist yet (need code to mutate)
  • Single trivial implementation with no edge cases
  • Testing application logic rather than algorithm implementations
  • The algorithm has no public test vectors to compare against

Prerequisites

  • trailmark installed — if uv run trailmark fails, run:
    uv tool install trailmark
    

Python snippets: uv run --with trailmark python - (a tool env is not importable)

- At least one implementation of the target algorithm in a
language with mutation testing support
- A test harness that consumes test vectors and exercises
the implementation
- A mutation testing framework for the target language

---

## Rationalizations to Reject

| Rationalization | Why It's Wrong | Required Action |
|-----------------|----------------|-----------------|
| "We have enough test vectors" | Mutation testing proves otherwise | Run the baseline first |
| "The implementation's own tests are sufficient" | Own tests often share blind spots with the impl | Cross-impl vectors catch different bugs |
| "FFI crates can be mutation tested at the binding layer" | Mutations to wrappers don't affect the underlying impl | Mutate the actual implementation language |
| "Timeouts mean the mutation was caught" | Timeouts are ambiguous — could be killed or alive | Resolve timeouts before drawing conclusions |
| "All mutants are equivalent" | Most aren't — verify by reading the mutation | Classify each escaped mutant individually |
| "Checking valid vectors is enough" | Permissive mutations survive without negative assertions | Assert rejection for every invalid vector |
| "Manual analysis is fine" | Manual analysis misses what tooling catches | Install and run the tools |

---

## Workflow Overview

Phase 1: Discovery → Find implementations to test ↓ Phase 2: Harness → Write/adapt test vector harness for each impl ↓ Phase 3: Baseline → Run mutation testing with existing vectors ↓ Phase 4: Escape Analysis → Classify escaped mutants by code path ↓ Phase 5: Vector Gen → Create test vectors targeting escapes ↓ Phase 6: Validation → Re-run mutation testing, compare before/after ↓ Output: Coverage Report + New Test Vectors


---

## Phase 1: Discovery

Find implementations of the target algorithm. Look for:

1. **Pure implementations** in high-level languages (Go, Rust, Python)
   — these are the best mutation testing targets
2. **FFI wrapper crates** — identify these early so you don't waste
   time mutating wrapper glue code
3. **Reference implementations** — useful for cross-verification but
   may not be the best mutation targets

For each implementation, note:
- Language and mutation testing framework
- Whether it's pure code or FFI wrappers
- Existing test suite size and coverage
- Which API surface the test vectors will exercise

### Implementation Type Classification

| Type | Mutation Value | Example |
|------|---------------|---------|
| Pure implementation | High | zkcrypto/bls12_381 (Rust), gnark-crypto (Go) |
| FFI bindings to C/asm | Low at binding layer | blst Rust crate |
| C/C++ implementation | High (use Mull) | blst C library |
| Generated code | Medium (mutations may be equivalent) | gnark-crypto generated field arithmetic |

**Key insight:** If an implementation delegates to another language
via FFI, you must mutate the *underlying* implementation, not the
bindings. For C/C++ underneath Rust/Go/Python, use Mull or similar.

---

## Phase 2: Harness

For each implementation, create a test harness that:

1. Reads test vectors from JSON files (Wycheproof format recommended)
2. Exercises the implementation's API for each vector
3. Asserts **both acceptance and rejection**:
   - Valid vectors: deserialization succeeds, output matches expected
   - Invalid vectors: deserialization fails or verification rejects
4. Adds **roundtrip assertions** for valid deserialization vectors:
   `serialize(deserialize(bytes)) == bytes`
5. Reports pass/fail per vector with test IDs

**Critical:** A harness that only checks valid vectors will miss all
permissive mutations (e.g., `&` → `|` in validation). See
[references/lessons-learned.md](references/lessons-learned.md) §7.

The harness must be runnable by the mutation testing framework.
For most frameworks this means:
- **Go:** A `_test.go` file in the same package as the implementation
- **Rust:** An integration test in `tests/` or inline `#[test]` functions
- **Python:** A pytest test file
- **C/C++:** A test binary linked against the implementation

### Harness Placement

The harness must live *inside the implementation's package* so the
mutation framework can see it. This usually means:

```bash
# Go: add test file to the package being mutated
cp wycheproof_test.go /path/to/impl/package/

# Rust: add integration test
cp wycheproof.rs /path/to/crate/tests/

# Python: add test to the test directory
cp test_wycheproof.py /path/to/package/tests/

Handling Existing Vectors

If the implementation already has test vectors:

  1. Run mutation testing with ONLY the existing vectors (baseline)
  2. Run mutation testing with ONLY your new vectors
  3. Run mutation testing with BOTH combined
  4. The delta between (1) and (3) shows the new vectors' value

Phase 3: Baseline

Run mutation testing with existing test vectors only.

Framework Selection

See references/mutation-frameworks.md for language-specific setup.

LanguageFrameworkCommand
Gogremlinsgremlins unleash ./path/to/package
Rustcargo-mutantscargo mutants -j N --timeout T
Pythonmutmutmutmut run --paths-to-mutate src/
C/C++Mullmull-runner -test-framework=GoogleTest binary

Parallelism

Always use parallel execution for large codebases:

  • cargo mutants -j 8 (Rust, 8 parallel workers)
  • gremlins unleash --timeout-coefficient 3 (Go, increase timeouts)
  • mutmut run --runner "pytest -x -q" (Python, fail-fast)

Recording Baseline Results

Capture these metrics per implementation:

MetricDescription
Total mutantsNumber of mutations generated
KilledMutants caught by tests
Survived/LivedMutants NOT caught (these are the targets)
Not coveredCode paths no test reaches at all
Timed outAmbiguous — resolve before comparing
Efficacy %Killed / (Killed + Survived)
Coverage %(Total - Not covered) / Total

Save the full mutation log for Phase 4 analysis.


Phase 4: Escape Analysis (Graph-Informed Triage)

Classify each escaped (survived + not covered) mutant using the Trailmark call graph for reachability and blast radius analysis.

This phase MUST use the genotoxic skill's triage methodology. The call graph transforms mutation results from a flat list of survived mutants into an actionable, prioritized set of vector targets.

Step 1: Build the Call Graph

Build a Trailmark code graph for each implementation before triaging mutations:

# Go
uv run trailmark analyze --language go --summary {targetDir}

# Rust
uv run trailmark analyze --language rust --summary {targetDir}

The graph provides:

  • Caller chains — trace from public API entry points to mutated functions to determine reachability
  • Cyclomatic complexity — prioritize high-CC functions
  • Blast radius — functions with many callers have wider impact if their mutations survive

Step 2: Filter to Relevant Code

Mutation frameworks test the entire package. Filter results to only the files/functions that test vectors should exercise:

# Go (gremlins)
grep -E "(LIVED|NOT COVERED)" baseline.log \
  | grep -E " at (relevant|files)" \
  | sort

# Rust (cargo-mutants)
cat mutants.out/missed.txt | grep "src/relevant"

Step 3: Graph-Informed Classification

For each escaped mutant, map it to its containing function in the call graph and apply the genotoxic triage criteria:

Graph SignalClassificationAction
No callers in graphFalse PositiveDead code, skip
Only test callersFalse PositiveTest infrastructure
Logging/display/formattingFalse PositiveCosmetic
Cross-package callers but NOT COVEREDCross-Package GapSee below
Reachable from public API, low CCMissing VectorDesign targeted vector
Reachable from public API, high CC (>10)Fuzzing TargetBoth vector + fuzz harness
Validation/error-handling pathNegative VectorCraft invalid input that triggers path
Optimization path (GLV, SIMD, batch)Edge-Case VectorInput that triggers optimization threshold
|→^ after left shift (e.g. (t<<1) | carry)Equivalent MutantSkip — bit 0 always 0, OR=XOR
ct_eq &→| on Montgomery limbsAPI-UnreachableNeeds library-internal tests, not vectors
Equivalent mutation (behavior unchanged)False PositiveSkip

Step 4: Identify Cross-Package Test Gaps

Critical pitfall: Mutation frameworks often only run tests within the same package as the mutation. For Go (gremlins) and Rust (cargo-mutants), this means:

  • A mutation in hash_to_curve/g2.go only runs tests in the hash_to_curve package, NOT tests in the parent bls12381 package that imports it
  • Functions that are fully exercised by cross-package tests will appear as NOT COVERED — these are false positives
  • To confirm: check if the mutated function is called from a test in a different package that wouldn't be run

To resolve cross-package gaps:

  1. Add a thin test in the sub-package that calls through the same code path as the cross-package test
  2. Or run gremlins with --test-pkg ./... (if supported)
  3. Or document as a framework limitation in the report

Step 5: Prioritize by Security Impact

Using the call graph, rank surviving mutants by impact:

PriorityCriteriaExample
P0 — CriticalMutant weakens validation/equality/authenticationct_eq: & → | makes equality permissive
P1 — HighMutant in deserialization flag parsingfrom_compressed: & → | accepts invalid flags
P2 — MediumMutant in field arithmetic internalsFp::square: | → ^ corrupts computation
P3 — LowMutant in optimization pathphi endomorphism: only affects performance path
SkipFormatting, display, equivalent mutationDebug::fmt return value replacement

Step 6: Group by Vector Strategy

Group escaped mutants by the code path they represent and the type of test vector needed:

Deserialization flag validation (P1):
  - g1.rs:339,363-365,384 — from_compressed_unchecked flags
  → Need: valid-point-wrong-flag vectors

Field arithmetic (P2):
  - fp.rs:371-376,406,635-643 — subtract_p, neg, square
  → Need: field arithmetic KATs with edge-case values

Optimization thresholds (P3):
  - g1.go:68, g2.go:75 — GLV vs windowed multiplication
  → Need: scalar multiplication with large scalars

Cross-package (framework limitation):
  - hash_to_curve/g2.go:242-278 — isogeny, sgn0
  → Document as false positive or add sub-package test

Each group becomes a target for new test vectors in Phase 5.


Phase 5: Vector Generation

For each escaped code path group, design test vectors that force execution through that path.

Vector Design Patterns

Code Path TypeVector Strategy
Point deserializationMalformed points: wrong length, invalid field elements, off-curve, wrong subgroup, identity point
Signature verificationValid sig + all single-bit corruptions of sig, pk, msg
Hash-to-curveKnown answer tests (KATs) with edge-case inputs: empty, single byte, max length
Aggregate operations1 signer, many signers, duplicate signers, mixed valid/invalid
Error handlingEvery error path should have a vector that triggers it
Arithmetic edge casesZero, one, field modulus - 1, points at infinity
Serialization flagsEvery valid flag combination + every invalid flag combination
Roundtrip integrityFor every valid deser vector, assert serialize(deserialize(b)) == b
Carry/reduction faultsReimplement at reduced limb widths, inject faults, extract distinguishing inputs

Single-Fault Negative Vectors

Each negative vector should have exactly one defect with everything else valid — this isolates which validation check is being tested. See references/vector-patterns.md for per-flag construction examples.

Fault Simulation (Limb-Width Reimplementation)

When mutation testing only applies local operator swaps, deeper architectural bugs (carry propagation, reduction overflow) go untested. To close this gap, reimplement the target algorithm at reduced limb widths (8, 16, 25, 32 bits) and deliberately inject faults — then generate vectors that catch them.

See references/fault-simulation.md for the full methodology: limb-width selection, fault injection catalog, vector extraction, and validation workflow.

Cross-Implementation Verification

Every new test vector MUST be verified against at least two independent implementations before being added to the suite:

  1. Generate the vector using implementation A
  2. Verify with implementation B (different codebase, ideally different language)
  3. If B disagrees, investigate — one implementation has a bug

Vector Format

Use Wycheproof JSON format (algorithm, testGroups[].tests[] with tcId, comment, result, flags). See references/vector-patterns.md for the full schema.

Wycheproof contributions: Use Wycheproof's vectorgen tool rather than formatting vector files directly. Supply the generated changes as an envelope. The vectorgen tool can add, update, or replace vectors while handling tcId assignment, test counts, canonical formatting, and schema validation. Go-based generators can avoid the vectorgen CLI tool and instead call the programmatic github.com/c2sp/wycheproof/vectorgen API.

See references/lessons-learned.md §14 and the upstream vectorgen guide for the current workflow and commands.


Phase 6: Validation

Re-run mutation testing with the new test vectors included.

Tip: Use per-file mutation testing for fast iteration during vector development (see references/lessons-learned.md §12). Only run full-crate tests for the final comparison.

Before/After Comparison

MetricBaselineWith New VectorsDelta
KilledXYY - X
SurvivedABA - B (should decrease)
Not CoveredCDC - D (should decrease)
Efficacy %E%F%F - E

Success Criteria

Vectors have both retroactive value (killing mutants in existing code) and proactive value (catching bugs in future implementations). Generate both kinds — boundary-condition vectors may not improve kill rates in mature libraries but will catch bugs in new implementations. See references/lessons-learned.md §13.

Retroactive (measurable): previously survived/uncovered mutants become killed, no regressions.

If kill rates don't change: the implementation's own tests likely already cover those paths. The vectors still add cross-implementation verification value. Document which case applies.


Output Format

Write VECTOR_FORGE_REPORT.md covering: target algorithm, implementations tested, baseline results, escape analysis, new vectors generated, after results, before/after delta, and conclusions. See references/report-template.md for the full template.


Quality Checklist

Before delivering:

  • At least one pure implementation mutation-tested (not just FFI wrappers)
  • Baseline run completed with existing vectors
  • Trailmark call graph built for each implementation
  • All escaped mutants triaged using graph-informed classification
  • Cross-package false positives identified and documented
  • Security-critical mutations (ct_eq, validation, auth) prioritized as P0/P1
  • Fault simulation and mutation-derived vectors cross-verified against 2+ implementations
  • After run completed with new vectors included
  • Before/after delta computed and explained
  • Report written to VECTOR_FORGE_REPORT.md
  • New test vectors saved in standard format (Wycheproof JSON)

Integration

SkillRelationship
genotoxic (required for Phase 4)Provides graph-informed triage — call graph cuts actionable mutants by 30-50%
mutation-testing (mewt/muton)Use for Solidity; Vector Forge is language-agnostic
property-based-testingBetter than hand-crafted vectors for bitwise mutations in field arithmetic
testing-handbook-skills (fuzzing)Functions with CC > 10 and surviving mutants need both vectors and fuzz harnesses

Supporting Documentation

Wycheproof

Wycheproof is an extensive collection of test vectors designed to verify the correctness of cryptographic implementations and test against known attacks. Originally developed by Google, it is now a community-managed project where contributors can add test vectors for specific cryptographic constructions.

Background

Key Concepts

ConceptDescription
Test vectorInput/output pair for validating crypto implementation correctness
Test groupCollection of test vectors sharing attributes (key size, IV size, curve)
Result flagIndicates if test should pass (valid), fail (invalid), or is acceptable
Edge case testingTesting for known vulnerabilities and attack patterns

Why This Matters

Cryptographic implementations are notoriously difficult to get right. Even small bugs can:

  • Expose private keys
  • Allow signature forgery
  • Enable message decryption
  • Create consensus problems when different implementations accept/reject the same inputs

Wycheproof has found vulnerabilities in major libraries including OpenJDK's SHA1withDSA, Bouncy Castle's ECDHC, and the elliptic npm package.

When to Use

Apply Wycheproof when:

  • Testing cryptographic implementations (AES-GCM, ECDSA, ECDH, RSA, etc.)
  • Validating that crypto code handles edge cases correctly
  • Verifying implementations against known attack vectors
  • Setting up CI/CD for cryptographic libraries
  • Auditing third-party crypto code for correctness

Consider alternatives when:

  • Testing for timing side-channels (use constant-time testing tools instead)
  • Finding new unknown bugs (use fuzzing instead)
  • Testing custom/experimental cryptographic algorithms (Wycheproof only covers established algorithms)

Quick Reference

ScenarioRecommended ApproachNotes
AES-GCM implementationUse aes_gcm_test.json316 test vectors across 44 test groups
ECDSA verificationUse ecdsa_*_test.json for specific curvesTests signature malleability, DER encoding
ECDH key exchangeUse ecdh_*_test.jsonTests invalid curve attacks
RSA signaturesUse rsa_*_test.jsonTests padding oracle attacks
ChaCha20-Poly1305Use chacha20_poly1305_test.jsonTests AEAD implementation

Testing Workflow

Phase 1: Setup                 Phase 2: Parse Test Vectors
┌─────────────────┐          ┌─────────────────┐
│ Add Wycheproof  │    →     │ Load JSON file  │
│ as submodule    │          │ Filter by params│
└─────────────────┘          └─────────────────┘
         ↓                            ↓
Phase 4: CI Integration        Phase 3: Write Harness
┌─────────────────┐          ┌─────────────────┐
│ Auto-update     │    ←     │ Test valid &    │
│ test vectors    │          │ invalid cases   │
└─────────────────┘          └─────────────────┘

Repository Structure

The Wycheproof repository is organized as follows:

┣ 📜 README.md       : Project overview
┣ 📂 doc             : Documentation
┣ 📂 java            : Java JCE interface testing harness
┣ 📂 javascript      : JavaScript testing harness
┣ 📂 schemas         : Test vector schemas
┣ 📂 testvectors     : Test vectors
┗ 📂 testvectors_v1  : Updated test vectors (more detailed)

The essential folders are testvectors and testvectors_v1. While both contain similar files, testvectors_v1 includes more detailed information and is recommended for new integrations.

Supported Algorithms

Wycheproof provides test vectors for a wide range of cryptographic algorithms:

CategoryAlgorithms
Symmetric EncryptionAES-GCM, AES-EAX, ChaCha20-Poly1305
SignaturesECDSA, EdDSA, RSA-PSS, RSA-PKCS1
Key ExchangeECDH, X25519, X448
HashingHMAC, HKDF
Curvessecp256k1, secp256r1, secp384r1, secp521r1, ed25519, ed448

Test File Structure

Each JSON test file tests a specific cryptographic construction. All test files share common attributes:

"algorithm"         : The name of the algorithm tested
"schema"            : The JSON schema (found in schemas folder)
"generatorVersion"  : The version number
"numberOfTests"     : The total number of test vectors in this file
"header"            : Detailed description of test vectors
"notes"             : In-depth explanation of flags in test vectors
"testGroups"        : Array of one or multiple test groups

Test Groups

Test groups group sets of tests based on shared attributes such as:

  • Key sizes
  • IV sizes
  • Public keys
  • Curves

This classification allows extracting tests that meet specific criteria relevant to the construction being tested.

Test Vector Attributes

Shared Attributes

All test vectors contain four common fields:

  • tcId: Unique identifier for the test vector within a file
  • comment: Additional information about the test case
  • flags: Descriptions of specific test case types and potential dangers (referenced in notes field)
  • result: Expected outcome of the test

The result field can take three values:

ResultMeaning
validTest case should succeed
acceptableTest case is allowed to succeed but contains non-ideal attributes
invalidTest case should fail
Unique Attributes

Unique attributes are specific to the algorithm being tested:

AlgorithmUnique Attributes
AES-GCMkey, iv, aad, msg, ct, tag
ECDH secp256k1public, private, shared
ECDSAmsg, sig, result
EdDSAmsg, sig, pk

Implementation Guide

Phase 1: Add Wycheproof to Your Project

Option 1: Git Submodule (Recommended)

Adding Wycheproof as a git submodule ensures automatic updates:

git submodule add https://github.com/C2SP/wycheproof.git

Option 2: Fetch Specific Test Vectors

If submodules aren't possible, fetch specific JSON files:

#!/bin/bash

TMP_WYCHEPROOF_FOLDER=".wycheproof/"
TEST_VECTORS=('aes_gcm_test.json' 'aes_eax_test.json')
BASE_URL="https://raw.githubusercontent.com/C2SP/wycheproof/master/testvectors_v1/"

# Create wycheproof folder
mkdir -p $TMP_WYCHEPROOF_FOLDER

# Request all test vector files if they don't exist
for i in "${TEST_VECTORS[@]}"; do
  if [ ! -f "${TMP_WYCHEPROOF_FOLDER}${i}" ]; then
    curl -o "${TMP_WYCHEPROOF_FOLDER}${i}" "${BASE_URL}${i}"
    if [ $? -ne 0 ]; then
      echo "Failed to download ${i}"
      exit 1
    fi
  fi
done

Phase 2: Parse Test Vectors

Identify the test file for your algorithm and parse the JSON:

Python Example:

import json

def load_wycheproof_test_vectors(path: str):
    testVectors = []
    try:
        with open(path, "r") as f:
            wycheproof_json = json.loads(f.read())
    except FileNotFoundError:
        print(f"No Wycheproof file found at: {path}")
        return testVectors

    # Attributes that need hex-to-bytes conversion
    convert_attr = {"key", "aad", "iv", "msg", "ct", "tag"}

    for testGroup in wycheproof_json["testGroups"]:
        # Filter test groups based on implementation constraints
        if testGroup["ivSize"] < 64 or testGroup["ivSize"] > 1024:
            continue

        for tv in testGroup["tests"]:
            # Convert hex strings to bytes
            for attr in convert_attr:
                if attr in tv:
                    tv[attr] = bytes.fromhex(tv[attr])
            testVectors.append(tv)

    return testVectors

JavaScript Example:

const fs = require('fs').promises;

async function loadWycheproofTestVectors(path) {
  const tests = [];

  try {
    const fileContent = await fs.readFile(path);
    const data = JSON.parse(fileContent.toString());

    data.testGroups.forEach(testGroup => {
      testGroup.tests.forEach(test => {
        // Add shared test group properties to each test
        test['pk'] = testGroup.publicKey.pk;
        tests.push(test);
      });
    });
  } catch (err) {
    console.error('Error reading or parsing file:', err);
    throw err;
  }

  return tests;
}

Phase 3: Write Testing Harness

Create test functions that handle both valid and invalid test cases.

Python/pytest Example:

import pytest
from cryptography.hazmat.primitives.ciphers.aead import AESGCM

tvs = load_wycheproof_test_vectors("wycheproof/testvectors_v1/aes_gcm_test.json")

@pytest.mark.parametrize("tv", tvs, ids=[str(tv['tcId']) for tv in tvs])
def test_encryption(tv):
    try:
        aesgcm = AESGCM(tv['key'])
        ct = aesgcm.encrypt(tv['iv'], tv['msg'], tv['aad'])
    except ValueError as e:
        # Implementation raised error - verify test was expected to fail
        assert tv['result'] != 'valid', tv['comment']
        return

    if tv['result'] == 'valid':
        assert ct[:-16] == tv['ct'], f"Ciphertext mismatch: {tv['comment']}"
        assert ct[-16:] == tv['tag'], f"Tag mismatch: {tv['comment']}"
    elif tv['result'] == 'invalid' or tv['result'] == 'acceptable':
        assert ct[:-16] != tv['ct'] or ct[-16:] != tv['tag']

@pytest.mark.parametrize("tv", tvs, ids=[str(tv['tcId']) for tv in tvs])
def test_decryption(tv):
    try:
        aesgcm = AESGCM(tv['key'])
        decrypted_msg = aesgcm.decrypt(tv['iv'], tv['ct'] + tv['tag'], tv['aad'])
    except ValueError:
        assert tv['result'] != 'valid', tv['comment']
        return
    except InvalidTag:
        assert tv['result'] != 'valid', tv['comment']
        assert 'ModifiedTag' in tv['flags'], f"Expected 'ModifiedTag' flag: {tv['comment']}"
        return

    assert tv['result'] == 'valid', f"No invalid test case should pass: {tv['comment']}"
    assert decrypted_msg == tv['msg'], f"Decryption mismatch: {tv['comment']}"

JavaScript/Mocha Example:

const assert = require('assert');

function testFactory(tcId, tests) {
  it(`[${tcId + 1}] ${tests[tcId].comment}`, function () {
    const test = tests[tcId];
    const ed25519 = new eddsa('ed25519');
    const key = ed25519.keyFromPublic(toArray(test.pk, 'hex'));

    let sig;
    if (test.result === 'valid') {
      sig = key.verify(test.msg, test.sig);
      assert.equal(sig, true, `[${test.tcId}] ${test.comment}`);
    } else if (test.result === 'invalid') {
      try {
        sig = key.verify(test.msg, test.sig);
      } catch (err) {
        // Point could not be decoded
        sig = false;
      }
      assert.equal(sig, false, `[${test.tcId}] ${test.comment}`);
    }
  });
}

// Generate tests for all test vectors
for (var tcId = 0; tcId < tests.length; tcId++) {
  testFactory(tcId, tests);
}

Phase 4: CI Integration

Ensure test vectors stay up to date by:

  1. Using git submodules: Update submodule in CI before running tests
  2. Fetching latest vectors: Run fetch script before test execution
  3. Scheduled updates: Set up weekly/monthly updates to catch new test vectors

Common Vulnerabilities Detected

Wycheproof test vectors are designed to catch specific vulnerability patterns:

VulnerabilityDescriptionAffected AlgorithmsExample CVE
Signature malleabilityMultiple valid signatures for same messageECDSA, EdDSACVE-2024-42459
Invalid DER encodingAccepting non-canonical DER signaturesECDSACVE-2024-42460, CVE-2024-42461
Invalid curve attacksECDH with invalid curve pointsECDHCommon in many libraries
Padding oracleTiming leaks in padding validationRSA-PKCS1Historical OpenSSL issues
Tag forgeryAccepting modified authentication tagsAES-GCM, ChaCha20-Poly1305Various implementations

Signature Malleability: Deep Dive

Problem: Implementations that don't validate signature encoding can accept multiple valid signatures for the same message.

Example (EdDSA): Appending or removing zeros from signature:

Valid signature:   ...6a5c51eb6f946b30d
Invalid signature: ...6a5c51eb6f946b30d0000  (should be rejected)

How to detect:

# Add signature length check
if len(sig) != 128:  # EdDSA signatures must be exactly 64 bytes (128 hex chars)
    return False

Impact: Can lead to consensus problems when different implementations accept/reject the same signatures.

Related Wycheproof tests:

  • EdDSA: tcId 37 - "removing 0 byte from signature"
  • ECDSA: tcId 06 - "Legacy: ASN encoding of r misses leading 0"

Case Study: Elliptic npm Package

This case study demonstrates how Wycheproof found three CVEs in the popular elliptic npm package (3000+ dependents, millions of weekly downloads).

Overview

The elliptic library is an elliptic-curve cryptography library written in JavaScript, supporting ECDH, ECDSA, and EdDSA. Using Wycheproof test vectors on version 6.5.6 revealed multiple vulnerabilities:

  • CVE-2024-42459: EdDSA signature malleability (appending/removing zeros)
  • CVE-2024-42460: ECDSA DER encoding - invalid bit placement
  • CVE-2024-42461: ECDSA DER encoding - leading zero in length field

Methodology

  1. Identify supported curves: ed25519 for EdDSA
  2. Find test vectors: testvectors_v1/ed25519_test.json
  3. Parse test vectors: Load JSON and extract tests
  4. Write test harness: Create parameterized tests
  5. Run tests: Identify failures
  6. Analyze root causes: Examine implementation code
  7. Propose fixes: Add validation checks

Key Findings

EdDSA Issue (CVE-2024-42459):

  • Missing signature length validation
  • Allowed trailing zeros in signatures
  • Fix: Add if(sig.length !== 128) return false;

ECDSA Issue 1 (CVE-2024-42460):

  • Missing check for first bit being zero in DER-encoded r and s values
  • Fix: Add if ((data[p.place] & 128) !== 0) return false;

ECDSA Issue 2 (CVE-2024-42461):

  • DER length field accepted leading zeros
  • Fix: Add if(buf[p.place] === 0x00) return false;

Impact

All three vulnerabilities allowed multiple valid signatures for a single message, leading to consensus problems across implementations.

Lessons learned:

  • Wycheproof catches subtle encoding bugs
  • Reusable test harnesses pay dividends
  • Test vector comments and flags help diagnose issues
  • Even popular libraries benefit from systematic test vector validation

Advanced Usage

Tips and Tricks

TipWhy It Helps
Filter test groups by parametersFocus on test vectors relevant to your implementation constraints
Use test vector flagsUnderstand specific vulnerability patterns being tested
Check the notes fieldGet detailed explanations of flag meanings
Test both encrypt/decrypt and sign/verifyEnsure bidirectional correctness
Run tests in CICatch regressions and benefit from new test vectors
Use parameterized testsGet clear failure messages with tcId and comment

Common Mistakes

MistakeWhy It's WrongCorrect Approach
Only testing valid casesMisses vulnerabilities where invalid inputs are acceptedTest all result types: valid, invalid, acceptable
Ignoring "acceptable" resultImplementation might have subtle bugsTreat acceptable as warnings worth investigating
Not filtering test groupsWastes time on unsupported parametersFilter by keySize, ivSize, etc. based on your implementation
Not updating test vectorsMiss new vulnerability patternsUse submodules or scheduled fetches
Testing only one directionEncrypt/sign might work but decrypt/verify failsTest both operations

Related Skills

Tool Skills

SkillPrimary Use in Wycheproof Testing
pytestPython testing framework for parameterized tests
mochaJavaScript testing framework for test generation
constant-time-testingComplement Wycheproof with timing side-channel testing
cryptofuzzFuzz-based crypto testing to find additional bugs

Technique Skills

SkillWhen to Apply
coverage-analysisEnsure test vectors cover all code paths in crypto implementation
property-based-testingTest mathematical properties (e.g., encrypt/decrypt round-trip)
fuzz-harness-writingCreate harnesses for crypto parsers (complements Wycheproof)

Related Domain Skills

SkillRelationship
crypto-testingWycheproof is a key tool in comprehensive crypto testing methodology
fuzzingUse fuzzing to find bugs Wycheproof doesn't cover (new edge cases)

Skill Dependency Map

                    ┌─────────────────────┐
                    │    wycheproof       │
                    │   (this skill)      │
                    └──────────┬──────────┘
                               │
           ┌───────────────────┼───────────────────┐
           │                   │                   │
           ▼                   ▼                   ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│  pytest/mocha   │ │ constant-time   │ │   cryptofuzz    │
│ (test framework)│ │   testing       │ │   (fuzzing)     │
└────────┬────────┘ └────────┬────────┘ └────────┬────────┘
         │                   │                   │
         └───────────────────┼───────────────────┘
                             │
                             ▼
              ┌──────────────────────────┐
              │   Technique Skills       │
              │ coverage, harness, PBT   │
              └──────────────────────────┘

Resources

Official Repository

Wycheproof GitHub Repository

The official repository contains:

  • All test vectors in testvectors/ and testvectors_v1/
  • JSON schemas in schemas/
  • Reference implementations in Java and JavaScript
  • Documentation in doc/

Real-World Examples

pycryptodome

The pycryptodome library integrates Wycheproof test vectors in their test suite, demonstrating best practices for Python crypto implementations.

Community Resources

  • C2SP Community - Cryptographic specifications and standards community maintaining Wycheproof
  • Wycheproof issues tracker - Report bugs in test vectors or suggest new constructions

Summary

Wycheproof is an essential tool for validating cryptographic implementations against known attack vectors and edge cases. By integrating Wycheproof test vectors into your testing workflow:

  1. Catch subtle encoding and validation bugs
  2. Prevent signature malleability issues
  3. Ensure consistent behavior across implementations
  4. Benefit from community-contributed test vectors
  5. Protect against known cryptographic vulnerabilities

The investment in writing a reusable testing harness pays dividends through continuous validation as new test vectors are added to the Wycheproof repository.

Sponsors

Details

Marketplace
marslanalvi/SkillsStack
Source
plugins/security/fuzzing
Category
Security
Keywords
security, fuzzing

More from marslanalvi/SkillsStack

All 45