Add American Fuzzy Lop (afl) to third_party/afl/

third_party/afl/src/docs/notes_for_asan.txt

+==================================
+Notes for using ASAN with afl-fuzz
+==================================
+
+  This file discusses some of the caveats for fuzzing under ASAN, and suggests
+  a handful of alternatives. See README for the general instruction manual.
+
+1) Short version
+----------------
+
+ASAN on 64-bit systems requests a lot of memory in a way that can't be easily
+distinguished from a misbehaving program bent on crashing your system.
+
+Because of this, fuzzing with ASAN is recommended only in four scenarios:
+
+  - On 32-bit systems, where we can always enforce a reasonable memory limit
+    (-m 800 or so is a good starting point),
+
+  - On 64-bit systems only if you can do one of the following:
+
+    - Compile the binary in 32-bit mode (gcc -m32),
+
+    - Precisely gauge memory needs using http://jwilk.net/software/recidivm .
+
+    - Limit the memory available to process using cgroups on Linux (see
+      experimental/asan_cgroups).
+
+To compile with ASAN, set AFL_USE_ASAN=1 before calling 'make clean all'. The
+afl-gcc / afl-clang wrappers will pick that up and add the appropriate flags.
+Note that ASAN is incompatible with -static, so be mindful of that.
+
+(You can also use AFL_USE_MSAN=1 to enable MSAN instead.)
+
+There is also the option of generating a corpus using a non-ASAN binary, and
+then feeding it to an ASAN-instrumented one to check for bugs. This is faster,
+and can give you somewhat comparable results.
+
+2) Long version
+---------------
+
+ASAN allocates a huge region of virtual address space for bookkeeping purposes.
+Most of this is never actually accessed, so the OS never has to allocate any
+real pages of memory for the process, and the VM grabbed by ASAN is essentially
+"free" - but the mapping counts against the standard OS-enforced limit
+(RLIMIT_AS, aka ulimit -v).
+
+On our end, afl-fuzz tries to protect you from processes that go off-rails
+and start consuming all the available memory in a vain attempt to parse a
+malformed input file. This happens surprisingly often, so enforcing such a limit
+is important for almost any fuzzer: the alternative is for the kernel OOM
+handler to step in and start killing random processes to free up resources.
+Needless to say, that's not a very nice prospect to live with.
+
+Unfortunately, un*x systems offer no portable way to limit the amount of
+pages actually given to a process in a way that distinguishes between that
+and the harmless "land grab" done by ASAN. In principle, there are three standard
+ways to limit the size of the heap:
+
+  - The RLIMIT_AS mechanism (ulimit -v) caps the size of the virtual space -
+    but as noted, this pays no attention to the number of pages actually
+    in use by the process, and doesn't help us here.
+
+  - The RLIMIT_DATA mechanism (ulimit -d) seems like a good fit, but it applies
+    only to the traditional sbrk() / brk() methods of requesting heap space;
+    modern allocators, including the one in glibc, routinely rely on mmap()
+    instead, and circumvent this limit completely.
+
+  - Finally, the RLIMIT_RSS limit (ulimit -m) sounds like what we need, but
+    doesn't work on Linux - mostly because nobody felt like implementing it.
+
+There are also cgroups, but they are Linux-specific, not universally available
+even on Linux systems, and they require root permissions to set up; I'm a bit
+hesitant to make afl-fuzz require root permissions just for that. That said,
+if you are on Linux and want to use cgroups, check out the contributed script
+that ships in experimental/asan_cgroups/.
+
+In settings where cgroups aren't available, we have no nice, portable way to
+avoid counting the ASAN allocation toward the limit. On 32-bit systems, or for
+binaries compiled in 32-bit mode (-m32), this is not a big deal: ASAN needs
+around 600-800 MB or so, depending on the compiler - so all you need to do is
+to specify -m that is a bit higher than that.
+
+On 64-bit systems, the situation is more murky, because the ASAN allocation
+is completely outlandish - around 17.5 TB in older versions, and closer to
+20 TB with newest ones. The actual amount of memory on your system is
+(probably!) just a tiny fraction of that - so unless you dial the limit
+with surgical precision, you will get no protection from OOM bugs.
+
+On my system, the amount of memory grabbed by ASAN with a slightly older
+version of gcc is around 17,825,850 MB; for newest clang, it's 20,971,600.
+But there is no guarantee that these numbers are stable, and if you get them
+wrong by "just" a couple gigs or so, you will be at risk.
+
+To get the precise number, you can use the recidivm tool developed by Jakub
+Wilk (http://jwilk.net/software/recidivm). In absence of this, ASAN is *not*
+recommended when fuzzing 64-bit binaries, unless you are confident that they
+are robust and enforce reasonable memory limits (in which case, you can
+specify '-m none' when calling afl-fuzz).
+
+Using recidivm or running with no limits aside, there are two other decent
+alternatives: build a corpus of test cases using a non-ASAN binary, and then
+examine them with ASAN, Valgrind, or other heavy-duty tools in a more
+controlled setting; or compile the target program with -m32 (32-bit mode)
+if your system supports that.
+
+3) Interactions with the QEMU mode
+----------------------------------
+
+ASAN, MSAN, and other sanitizers appear to be incompatible with QEMU user
+emulation, so please do not try to use them with the -Q option; QEMU doesn't
+seem to appreciate the shadow VM trick used by these tools, and will likely
+just allocate all your physical memory, then crash.
+
+4) What about UBSAN?
+--------------------
+
+Some folks expressed interest in fuzzing with UBSAN. This isn't officially
+supported, because many installations of UBSAN don't offer a consistent way
+to abort() on fault conditions or to terminate with a distinctive exit code.
+
+That said, some versions of the library can be binary-patched to address this
+issue, while newer releases support explicit compile-time flags - see this
+mailing list thread for tips:
+
+  https://groups.google.com/forum/#!topic/afl-users/GyeSBJt4M38