Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(110)

Unified Diff: testing/libfuzzer/efficient_fuzzer.md

Issue 1809843002: [libfuzzer] First part of libfuzzer-chrome documentation. (Closed) Base URL: https://chromium.googlesource.com/chromium/src.git@master
Patch Set: review Created 4 years, 9 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View side-by-side diff with in-line comments
Download patch
« no previous file with comments | « testing/libfuzzer/clusterfuzz.md ('k') | testing/libfuzzer/getting_started.md » ('j') | no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
Index: testing/libfuzzer/efficient_fuzzer.md
diff --git a/testing/libfuzzer/efficient_fuzzer.md b/testing/libfuzzer/efficient_fuzzer.md
new file mode 100644
index 0000000000000000000000000000000000000000..d72b752c1e751c963f66bfc0d12914fc20dfce60
--- /dev/null
+++ b/testing/libfuzzer/efficient_fuzzer.md
@@ -0,0 +1,94 @@
+# Efficient Fuzzer
+
+This document describes ways to determine your fuzzer efficiency and ways
+to improve it.
+
+## Overview
+
+Being a coverage-driven fuzzer, Libfuzzer considers a certain input *interesting*
+if it results in new coverage. The set of all interesting inputs is called
+*corpus*.
+Items in corpus are constantly mutated in search of new interesting input.
+Corpus is usually maintained between multiple fuzzer runs.
+
+There are several metrics you should look at to determine your fuzzer effectiveness:
+
+* fuzzer speed (exec/s)
+* corpus size
+* coverage
+
+You can collect these metrics manually or take them from [ClusterFuzz status]
+pages.
+
+## Fuzzer Speed
+
+Fuzzer speed is printed while fuzzer runs:
+
+```
+#19346 NEW cov: 2815 bits: 1082 indir: 43 units: 150 exec/s: 19346 L: 62
+```
+
+Because Libfuzzer performs randomized search, it is critical to have it as fast
+as possible. You should try to get to at least 1,000 exec/s. Profile the fuzzer
+using any standard tool to see where it spends its time.
+
+### Initialization/Cleanup
+
+Try to keep your fuzzing function as simple as possible. Prefer to use static
+initialization and shared resources rather than bringing environment up and down
+every single run.
+
+Fuzzers don't have to shutdown gracefully (we either kill them or they crash
+because sanitizer has found a problem). You can skip freeing static resource.
+
+Of course all resources allocated withing `LLVMFuzzerTestOneInput` function
+should be deallocated since this function is called millions of times during
+one fuzzing session.
+
+
+## Corpus Size
+
+After running for a while the fuzzer would reach a plateau and won't discover
+new interesting input. Corpus for a reasonably complex functionality
+should contain hundreds (if not thousands) of items.
+
+Too small corpus size indicates some code barrier that
+libfuzzer is having problems penetrating. Common cases include: checksums,
+magic numbers etc. The easiest way to diagnose this problem is to generate a
+[coverage report](#Coverage). To fix the issue you can:
+
+* change the code (e.g. disable crc checks while fuzzing)
+* prepare fuzzer dictionary
+* prepare [corpus seed](#Corpus-Seed).
+
+## Coverage
+
+You can easily generate source-level coverage report for a given corpus:
+
+```
+ASAN_OPTIONS=coverage=1:html_cov_report=1:sancov_path=./third_party/llvm-build/Release+Asserts/bin/sancov \
+ ./out/libfuzzer/my_fuzzer -runs=0 ~/tmp/my_fuzzer_corpus
+```
+
+This will produce an .html file with colored source-code. It can be used to
+determine where your fuzzer is "stuck".
+
+## Corpus Seed
+
+You can pass a corpus directory to a fuzzer that you run manually:
+
+```
+./out/libfuzzer/my_fuzzer ~/tmp/my_fuzzer_corpus
+```
+
+The directory can initially be empty. The fuzzer would store all the interesting
+items it finds in the directory. You can help the fuzzer by "seeding" the corpus:
+simply copy interesting inputs for your function to the corpus directory before
+running. This works especially well for file-parsing functionality: just
+use some valid files from your test suite.
+
+After discovering new and interesting items, [upload corpus to Clusterfuzz].
+
+
+[ClusterFuzz status]: ./clusterfuzz.md#Fuzzer-Status
+[upload corpus to Clusterfuzz]: ./clusterfuzz.md#Upload-Corpus
« no previous file with comments | « testing/libfuzzer/clusterfuzz.md ('k') | testing/libfuzzer/getting_started.md » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698