Make SyzyAsan support the allocation > 1GB

syzygy/agent/asan/heap_managers/block_heap_manager.cc

 // Copyright 2014 Google Inc. All Rights Reserved.
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
 //
 //     http://www.apache.org/licenses/LICENSE-2.0
 //
 // Unless required by applicable law or agreed to in writing, software
 // distributed under the License is distributed on an "AS IS" BASIS,
@@ skipping 27 matching lines @@
 
 typedef HeapManagerInterface::HeapId HeapId;
 using heaps::LargeBlockHeap;
 using heaps::ZebraBlockHeap;
 
 // For now, the overbudget size is always set to 20% of the size of the
 // quarantine.
 // TODO(georgesak): allow this to be changed through the parameters.
 enum : uint32_t { kOverbudgetSizePercentage = 20 };
 
+// The maximum allocation size that we can handle in SyzyAsan, this is
+// constrained by the number of bits used to store the block size in the block
+// header structure.
+const size_t kMaxAllocSize = 0x8000000;

[chrisha, 2016/11/23 19:37:17]

This should be part of block.h, and tied to a #defined constant that is used to
set the bitsize of the "size" field. With some static_assert or something to
make sure that everything works well?

static constexpr BlockHeader kDummyHeader = {};
static_assert(kDummyHeader.size - 1 == kMaxAllocSize);

Something like that?

(And this value should (1 << 31) - 1 = 0x7FFFFFFF, no?)

[Sébastien Marchand, 2016/11/23 22:36:01]

I've tried this but it doesn't seem to work, doing an overflow on purpose seems
to use the whole |unsigned| length (and not just the 31 bits), e.g. in a
unittest:
error: Value of: ~block_header.body_size
Actual: 4294967295
Expected: kBlockMaxAllocSize
Which is: 2147483647

(same behavior in a static_assert with block_header.body_size - 1)

[chrisha, 2016/11/24 17:23:22]

Acknowledged.

+
 // Return the position of the most significant bit in a 32 bit unsigned value.
 size_t GetMSBIndex(size_t n) {
   // Algorithm taken from
   // http://graphics.stanford.edu/~seander/bithacks.html#IntegerLog
   size_t r = 0;
   size_t shift = 0;
   r = (n > 0xFFFF) << 4;
   n >>= r;
   shift = (n > 0xFF) << 3;
   n >>= shift;
@@ skipping 111 matching lines @@
     heaps_.erase(heaps_.find(heap));
   }
 
   return true;
 }
 
 void* BlockHeapManager::Allocate(HeapId heap_id, uint32_t bytes) {
   DCHECK(initialized_);
   DCHECK(IsValidHeapId(heap_id, false));
 
+  // Prevent from trying to allocate a memory block bigger than what we can
+  // represent in the block header.
+  if (bytes > kMaxAllocSize)
+    return nullptr;

[chrisha, 2016/11/23 19:37:17]

I'd just push this through to the underlying heap without instrumentation. It
could be that *that* heap can make the allocation.

[Sébastien Marchand, 2016/11/23 22:36:01]

I think that this should live in block.cc::BlockPlanLayout, as this is a limit
imposed by the Block format.

+
   // Some allocations can pass through without instrumentation.
   if (parameters_.allocation_guard_rate < 1.0 &&
       base::RandDouble() >= parameters_.allocation_guard_rate) {
     BlockHeapInterface* heap = GetHeapFromId(heap_id);
     void* alloc = heap->Allocate(bytes);
     if ((heap->GetHeapFeatures() &
         HeapInterface::kHeapReportsReservations) != 0) {
       shadow_->Unpoison(alloc, bytes);
     }
     return alloc;
@@ skipping 29 matching lines @@
     alloc = heap->AllocateBlock(
         bytes,
         0,
         parameters_.trailer_padding_size + sizeof(BlockTrailer),
         &block_layout);
     if (alloc != nullptr) {
       heap_id = heaps[i];
       break;
     }
   }
 

[chrisha, 2016/11/24 17:23:22]

If you're moving the logic to BlockPlanLayout, we should have a fallback to try
the allocation using the *uninstrumented* heap here. The block may be too big
for our BlockHeaps to allocate, but it may still be able to be allocated by the
underlying heap we're wrapping.

So try a last ditch effort to make an unprotected allocation. If that then fails
we can bail by returning nullptr.

[Sébastien Marchand, 2016/11/25 02:38:21]

Done.

   // The allocation can fail if we're out of memory.
   if (alloc == nullptr)
     return nullptr;
 
   DCHECK_NE(static_cast<void*>(nullptr), alloc);
   DCHECK_EQ(0u, reinterpret_cast<size_t>(alloc) % kShadowRatio);
   BlockInfo block = {};
   BlockInitialize(block_layout, alloc, &block);
 
   // Poison the redzones in the shadow memory as early as possible.
@@ skipping 873 matching lines @@
   if (trailer_has_valid_heap_id)
     return block_info->trailer->heap_id;
 
   // Unfortunately, there's no way to know which heap this block belongs to.
   return 0;
 }
 
 }  // namespace heap_managers
 }  // namespace asan
 }  // namespace agent