Stop reporting OOM as errors in libpng fuzzers

testing/libfuzzer/fuzzers/libpng_read_fuzzer.cc

Index: testing/libfuzzer/fuzzers/libpng_read_fuzzer.cc
diff --git a/testing/libfuzzer/fuzzers/libpng_read_fuzzer.cc b/testing/libfuzzer/fuzzers/libpng_read_fuzzer.cc
index c1b62ebaf4cfe00cacb453c921de90ceee20aee7..a68637aff62430970d300792d31814f1f22a3c2f 100644
--- a/testing/libfuzzer/fuzzers/libpng_read_fuzzer.cc
+++ b/testing/libfuzzer/fuzzers/libpng_read_fuzzer.cc
@@ -12,6 +12,23 @@
 #define PNG_INTERNAL
 #include "third_party/libpng/png.h"
 
+#ifdef MEMORY_SANITIZER

[kcc2, 2017/04/17 18:14:28]

Why #ifdef MEMORY_SANITIZER? 
Isn't this relevant to any other build mode? 

[scroggo_chromium, 2017/04/17 18:51:53]

I would have thought so, too, but
https://bugs.chromium.org/p/chromium/issues/detail?id=673082#c18 suggests we
only set the limit when using MEMORY_SANITIZER. I supposed that to mean that
this was only a problem when using msan - i.e. we successfully allocate this
large amount otherwise. When I run locally, not using msan, we do not crash
trying to decode the test image, but a LargeAlloc is reported.

+void* limited_malloc(png_structp, png_alloc_size_t size) {
+  // libpng may allocate large amounts of memory that the fuzzer reports as
+  // an error. In order to silence these errors, make libpng fail when trying
+  // to allocate a large amount.
+  // This number is chosen to match the default png_user_chunk_malloc_max.
+  if (size > 8000000)
+    return nullptr;
+
+  return malloc(size);
+}
+
+void default_free(png_structp, png_voidp ptr) {
+  return free(ptr);
+}
+#endif  // MEMORY_SANITIZER
+
 #ifndef PNG_FUZZ_PROGRESSIVE
 
 // Read sequentially, with png_read_row.
@@ -56,6 +73,10 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
   // To avoid OOM with MSan (crbug.com/648073). These values are recommended as
   // safe settings by https://github.com/glennrp/libpng/blob/libpng16/pngusr.dfa
   png_set_user_limits(png_ptr, 65535, 65535);
+
+  // Not all potential OOM are due to images with large widths and heights.
+  // Use a custom allocator that fails for large allocations.
+  png_set_mem_fn(png_ptr, nullptr, limited_malloc, default_free);
 #endif
 
   png_set_crc_action(png_ptr, PNG_CRC_QUIET_USE, PNG_CRC_QUIET_USE);