OLD | NEW |
(Empty) | |
| 1 # Efficient Fuzzer |
| 2 |
| 3 This document describes ways to determine your fuzzer efficiency and ways |
| 4 to improve it. |
| 5 |
| 6 ## Overview |
| 7 |
| 8 Being a coverage-driven fuzzer, Libfuzzer considers a certain input *interesting
* |
| 9 if it results in new coverage. The set of all interesting inputs is called |
| 10 *corpus*. |
| 11 Items in corpus are constantly mutated in search of new interesting input. |
| 12 Corpus is usually maintained between multiple fuzzer runs. |
| 13 |
| 14 There are several metrics you should look at to determine your fuzzer effectiven
ess: |
| 15 |
| 16 * fuzzer speed (exec/s) |
| 17 * corpus size |
| 18 * coverage |
| 19 |
| 20 You can collect these metrics manually or take them from [ClusterFuzz status] |
| 21 pages. |
| 22 |
| 23 ## Fuzzer Speed |
| 24 |
| 25 Fuzzer speed is printed while fuzzer runs: |
| 26 |
| 27 ``` |
| 28 #19346 NEW cov: 2815 bits: 1082 indir: 43 units: 150 exec/s: 19346 L: 62 |
| 29 ``` |
| 30 |
| 31 Because Libfuzzer performs randomized search, it is critical to have it as fast |
| 32 as possible. You should try to get to at least 1,000 exec/s. Profile the fuzzer |
| 33 using any standard tool to see where it spends its time. |
| 34 |
| 35 ### Initialization/Cleanup |
| 36 |
| 37 Try to keep your fuzzing function as simple as possible. Prefer to use static |
| 38 initialization and shared resources rather than bringing environment up and down |
| 39 every single run. |
| 40 |
| 41 Fuzzers don't have to shutdown gracefully (we either kill them or they crash |
| 42 because sanitizer has found a problem). You can skip freeing static resource. |
| 43 |
| 44 Of course all resources allocated withing `LLVMFuzzerTestOneInput` function |
| 45 should be deallocated since this function is called millions of times during |
| 46 one fuzzing session. |
| 47 |
| 48 |
| 49 ## Corpus Size |
| 50 |
| 51 After running for a while the fuzzer would reach a plateau and won't discover |
| 52 new interesting input. Corpus for a reasonably complex functionality |
| 53 should contain hundreds (if not thousands) of items. |
| 54 |
| 55 Too small corpus size indicates some code barrier that |
| 56 libfuzzer is having problems penetrating. Common cases include: checksums, |
| 57 magic numbers etc. The easiest way to diagnose this problem is to generate a |
| 58 [coverage report](#Coverage). To fix the issue you can: |
| 59 |
| 60 * change the code (e.g. disable crc checks while fuzzing) |
| 61 * prepare fuzzer dictionary |
| 62 * prepare [corpus seed](#Corpus-Seed). |
| 63 |
| 64 ## Coverage |
| 65 |
| 66 You can easily generate source-level coverage report for a given corpus: |
| 67 |
| 68 ``` |
| 69 ASAN_OPTIONS=coverage=1:html_cov_report=1:sancov_path=./third_party/llvm-build/R
elease+Asserts/bin/sancov \ |
| 70 ./out/libfuzzer/my_fuzzer -runs=0 ~/tmp/my_fuzzer_corpus |
| 71 ``` |
| 72 |
| 73 This will produce an .html file with colored source-code. It can be used to |
| 74 determine where your fuzzer is "stuck". |
| 75 |
| 76 ## Corpus Seed |
| 77 |
| 78 You can pass a corpus directory to a fuzzer that you run manually: |
| 79 |
| 80 ``` |
| 81 ./out/libfuzzer/my_fuzzer ~/tmp/my_fuzzer_corpus |
| 82 ``` |
| 83 |
| 84 The directory can initially be empty. The fuzzer would store all the interesting |
| 85 items it finds in the directory. You can help the fuzzer by "seeding" the corpus
: |
| 86 simply copy interesting inputs for your function to the corpus directory before |
| 87 running. This works especially well for file-parsing functionality: just |
| 88 use some valid files from your test suite. |
| 89 |
| 90 After discovering new and interesting items, [upload corpus to Clusterfuzz]. |
| 91 |
| 92 |
| 93 [ClusterFuzz status]: ./clusterfuzz.md#Fuzzer-Status |
| 94 [upload corpus to Clusterfuzz]: ./clusterfuzz.md#Upload-Corpus |
OLD | NEW |