OLD | NEW |
---|---|
(Empty) | |
1 # Efficient Fuzzer | |
2 | |
3 This documents describes ways to determine your fuzzer efficiency and ways | |
mmoroz
2016/03/17 11:58:13
nit: s/documents/document
aizatsky
2016/03/17 23:39:25
Done.
| |
4 to improve it. | |
5 | |
6 ## Overview | |
7 | |
8 Being a coverage-driven fuzzer, Libfuzzer considers a certain input *interesting * | |
9 if it results in new coverage. The set of all interesting inputs is called | |
10 *corpus*. | |
11 Items in corpus are constantly mutated in search of new interesting input. | |
12 Corpus is usually maintained between multiple fuzzer runs. | |
13 | |
14 There are several metrics you should look at to determine your fuzzer effectiven ess: | |
15 | |
16 * fuzzer speed (exec/s) | |
17 * corpus size | |
18 * coverage | |
19 | |
20 You can collect these metrics manually or take them from [ClusterFuzz status] | |
21 pages. | |
22 | |
23 ## Fuzzer Speed | |
24 | |
25 Fuzzer speed is printed while fuzzer runs: | |
26 | |
27 ``` | |
28 #19346 NEW cov: 2815 bits: 1082 indir: 43 units: 150 exec/s: 19346 L: 62 | |
29 ``` | |
30 | |
31 Because Libfuzzer performs randomized search, it is critical to have it as fast | |
32 as possible. You should try to get to at least 1,000 exec/s. Profile the fuzzer | |
33 using any standard tool to see where it spends its time. | |
34 | |
35 ### Initialization/Cleanup | |
36 | |
37 Try to keep your fuzzing function as simple as possible. Prefer to use static | |
38 initialization and shared resources rather than brining environment up and down | |
inferno
2016/03/17 16:49:36
typo brining
aizatsky
2016/03/17 23:39:25
Done.
| |
39 every single run. | |
40 | |
41 Fuzzers don't have to shutdown gracefully (we either kill them or they crash | |
42 because sanitizer has found a problem). You can skip freeing static resource. | |
43 | |
44 Of course all resources allocated withing `LLVMFuzzerTestOneInput` function | |
45 should be deallocated since this function is called millions of times during | |
46 one fuzzing session. | |
47 | |
48 | |
49 ## Corpus Size | |
50 | |
51 After running for a while the fuzzer would reach a plateau and won't discover | |
52 new interesting input. Corpus for a reasonably complex functionality | |
53 should contain hundreds (if not thousands) of items. | |
54 | |
55 Too small corpus size indicates some code barrier that | |
56 libfuzzer is having problems penetrating. Common cases include: checksums, | |
57 magic numbers etc. The easiest way to diagnose this problem is to generate a | |
58 [coverage report](#Coverage). To fix the issue you can: | |
59 | |
60 * change the code (e.g. disable crc checks while fuzzing) | |
61 * prepare fuzzer dictionary | |
62 * prepare [corpus seed](#Corpus-Seed). | |
63 | |
64 ## Coverage | |
65 | |
66 You can easily generate source-level coverage report for a given corpus: | |
67 | |
68 ``` | |
69 ASAN_OPTIONS=coverage=1:html_cov_report=1:sancov_path=./third_party/llvm-build/R elease+Asserts/bin/sancov \ | |
70 ./out/libfuzzer/my_fuzzer -runs=0 ~/tmp/my_fuzzer_corpus | |
71 ``` | |
72 | |
73 This will produce an .html file with colored source-code. It can be used to | |
74 determine where your fuzzer is "stuck". | |
75 | |
76 ## Corpus Seed | |
77 | |
78 You can pass a corpus directory to a fuzzer that you run manually: | |
79 | |
80 ``` | |
81 ./out/libfuzzer/my_fuzzer ~/tmp/my_fuzzer_corpus | |
82 ``` | |
83 | |
84 The directory can initially be empty. The fuzzer would store all the interesting | |
85 items it finds in the directory. You can help the fuzzer by "seeding" the corpus : | |
86 simply copy interesting inputs for your function to the corpus directory before | |
87 running. This works especially well for file-parsing functionality: just | |
88 use some valid files from your test suite. | |
89 | |
90 After discovering new and interesting items, [upload corpus to Clusterfuzz]. | |
91 | |
92 | |
93 [ClusterFuzz status]: ./clusterfuzz.md#Fuzzer-Status | |
94 [upload corpus to Clusterfuzz]: ./clusterfuzz.md#Upload-Corpus | |
OLD | NEW |