Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(460)

Side by Side Diff: testing/libfuzzer/efficient_fuzzer.md

Issue 1809843002: [libfuzzer] First part of libfuzzer-chrome documentation. (Closed) Base URL: https://chromium.googlesource.com/chromium/src.git@master
Patch Set: review Created 4 years, 9 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « testing/libfuzzer/clusterfuzz.md ('k') | testing/libfuzzer/getting_started.md » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
(Empty)
1 # Efficient Fuzzer
2
3 This document describes ways to determine your fuzzer efficiency and ways
4 to improve it.
5
6 ## Overview
7
8 Being a coverage-driven fuzzer, Libfuzzer considers a certain input *interesting *
9 if it results in new coverage. The set of all interesting inputs is called
10 *corpus*.
11 Items in corpus are constantly mutated in search of new interesting input.
12 Corpus is usually maintained between multiple fuzzer runs.
13
14 There are several metrics you should look at to determine your fuzzer effectiven ess:
15
16 * fuzzer speed (exec/s)
17 * corpus size
18 * coverage
19
20 You can collect these metrics manually or take them from [ClusterFuzz status]
21 pages.
22
23 ## Fuzzer Speed
24
25 Fuzzer speed is printed while fuzzer runs:
26
27 ```
28 #19346 NEW cov: 2815 bits: 1082 indir: 43 units: 150 exec/s: 19346 L: 62
29 ```
30
31 Because Libfuzzer performs randomized search, it is critical to have it as fast
32 as possible. You should try to get to at least 1,000 exec/s. Profile the fuzzer
33 using any standard tool to see where it spends its time.
34
35 ### Initialization/Cleanup
36
37 Try to keep your fuzzing function as simple as possible. Prefer to use static
38 initialization and shared resources rather than bringing environment up and down
39 every single run.
40
41 Fuzzers don't have to shutdown gracefully (we either kill them or they crash
42 because sanitizer has found a problem). You can skip freeing static resource.
43
44 Of course all resources allocated withing `LLVMFuzzerTestOneInput` function
45 should be deallocated since this function is called millions of times during
46 one fuzzing session.
47
48
49 ## Corpus Size
50
51 After running for a while the fuzzer would reach a plateau and won't discover
52 new interesting input. Corpus for a reasonably complex functionality
53 should contain hundreds (if not thousands) of items.
54
55 Too small corpus size indicates some code barrier that
56 libfuzzer is having problems penetrating. Common cases include: checksums,
57 magic numbers etc. The easiest way to diagnose this problem is to generate a
58 [coverage report](#Coverage). To fix the issue you can:
59
60 * change the code (e.g. disable crc checks while fuzzing)
61 * prepare fuzzer dictionary
62 * prepare [corpus seed](#Corpus-Seed).
63
64 ## Coverage
65
66 You can easily generate source-level coverage report for a given corpus:
67
68 ```
69 ASAN_OPTIONS=coverage=1:html_cov_report=1:sancov_path=./third_party/llvm-build/R elease+Asserts/bin/sancov \
70 ./out/libfuzzer/my_fuzzer -runs=0 ~/tmp/my_fuzzer_corpus
71 ```
72
73 This will produce an .html file with colored source-code. It can be used to
74 determine where your fuzzer is "stuck".
75
76 ## Corpus Seed
77
78 You can pass a corpus directory to a fuzzer that you run manually:
79
80 ```
81 ./out/libfuzzer/my_fuzzer ~/tmp/my_fuzzer_corpus
82 ```
83
84 The directory can initially be empty. The fuzzer would store all the interesting
85 items it finds in the directory. You can help the fuzzer by "seeding" the corpus :
86 simply copy interesting inputs for your function to the corpus directory before
87 running. This works especially well for file-parsing functionality: just
88 use some valid files from your test suite.
89
90 After discovering new and interesting items, [upload corpus to Clusterfuzz].
91
92
93 [ClusterFuzz status]: ./clusterfuzz.md#Fuzzer-Status
94 [upload corpus to Clusterfuzz]: ./clusterfuzz.md#Upload-Corpus
OLDNEW
« no previous file with comments | « testing/libfuzzer/clusterfuzz.md ('k') | testing/libfuzzer/getting_started.md » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698