Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(93)

Side by Side Diff: base/allocator/README.md

Issue 1719433002: Introduce allocator shim for Android (Closed) Base URL: https://chromium.googlesource.com/chromium/src.git@shim_traceintegration
Patch Set: Add readme changes, do NOT enabled by default in this cl Created 4 years, 8 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « base/allocator/BUILD.gn ('k') | base/allocator/allocator.gyp » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 This document describes how malloc / new calls are routed in the various Chrome 1 This document describes how malloc / new calls are routed in the various Chrome
2 platforms. 2 platforms.
3 3
4 Bare in mind that the chromium codebase does not always just use `malloc()`. 4 Bare in mind that the chromium codebase does not always just use `malloc()`.
5 Some examples: 5 Some examples:
6 - Large parts of the renderer (Blink) use two home-brewed allocators, 6 - Large parts of the renderer (Blink) use two home-brewed allocators,
7 PartitionAlloc and BlinkGC (Oilpan). 7 PartitionAlloc and BlinkGC (Oilpan).
8 - Some subsystems, such as the V8 JavaScript engine, handle memory management 8 - Some subsystems, such as the V8 JavaScript engine, handle memory management
9 autonomously. 9 autonomously.
10 - Various parts of the codebase use abstractions such as `SharedMemory` or 10 - Various parts of the codebase use abstractions such as `SharedMemory` or
(...skipping 67 matching lines...) Expand 10 before | Expand all | Expand 10 after
78 78
79 The tcmalloc library originates outside of Chromium and exists in 79 The tcmalloc library originates outside of Chromium and exists in
80 `../../third_party/tcmalloc` (currently, the actual location is defined in the 80 `../../third_party/tcmalloc` (currently, the actual location is defined in the
81 allocator.gyp file). The third party sources use a vendor-branch SCM pattern to 81 allocator.gyp file). The third party sources use a vendor-branch SCM pattern to
82 track Chromium-specific changes independently from upstream changes. 82 track Chromium-specific changes independently from upstream changes.
83 83
84 The general intent is to push local changes upstream so that over 84 The general intent is to push local changes upstream so that over
85 time we no longer need any forked files. 85 time we no longer need any forked files.
86 86
87 87
88 Unified allocator shim
89 ----------------------
90 On most platform, Chrome overrides the malloc / operator new symbols (and
91 corresponding free / delete and other variants). This is to enforce security
92 checks and lately to enable the
93 [memory-infra heap profiler][url-memory-infra-heap-profiler].
94 Historically each platform had its special logic for defining the allocator
95 symbols in different places of the codebase. The unified allocator shim is
96 a project aimed to unify the symbol definition and allocator routing logic in
97 a central place.
98
99 - Full documentation: [Allocator shim design doc][url-allocator-shim].
100 - Current state: Available and enabled by default on Linux and CrOS.
101 - Tracking bug: [https://crbug.com/550886][crbug.com/550886].
102 - Build-time flag: `use_experimental_allocator_shim`.
103
104 **Overview of the unified allocator shim**
105 The allocator shim consists of three stages:
106 ```
107 +-------------------------+ +-----------------------+ +----------------+
108 | malloc & friends | -> | shim layer | -> | Routing to |
109 | symbols definition | | implementation | | allocator |
110 +-------------------------+ +-----------------------+ +----------------+
111 | - libc symbols (malloc, | | - Security checks | | - tcmalloc |
112 | calloc, free, ...) | | - Chain of dispatchers| | - glibc |
113 | - C++ symbols (operator | | that can intercept | | - Android |
114 | new, delete, ...) | | and override | | bionic |
115 | - glibc weak symbols | | allocations | | - WinHeap |
116 | (__libc_malloc, ...) | +-----------------------+ +----------------+
117 +-------------------------+
118 ```
119
120 **1. malloc symbols definition**
121 This stage takes care of overriding the symbols `malloc`, `free`,
122 `operator new`, `operator delete` and friends and routing those calls inside the
123 allocator shim (next point).
124 This is taken care of by the headers in `allocator_shim_override_*`.
125
126 *On Linux/CrOS*: the allocator symbols are defined as exported global symbols
127 in `allocator_shim_override_libc_symbols.h` (for `malloc`, `free` and friends)
128 and in `allocator_shim_override_cpp_symbols.h` (for `operator new`,
129 `operator delete` and friends).
130 This enables proper interposition of malloc symbols referenced by the main
131 executable and any third party libraries. Symbol resolution on Linux is a breadt h first search that starts from the root link unit, that is the executable
132 (see EXECUTABLE AND LINKABLE FORMAT (ELF) - Portable Formats Specification).
133 Additionally, when tcmalloc is the default allocator, some extra glibc symbols
134 are also defined in `allocator_shim_override_glibc_weak_symbols.h`, for subtle
135 reasons explained in that file.
136 The Linux/CrOS shim was introduced by
137 [crrev.com/1675143004](https://crrev.com/1675143004).
138
139 *On Android*: load-time symbol interposition (unlike the Linux/CrOS case) is not
140 possible. This is because Android processes are `fork()`-ed from the Android
141 zygote, which pre-loads libc.so and only later native code gets loaded via
142 `dlopen()` (symbols from `dlopen()`-ed libraries get a different resolution
143 scope).
144 In this case, the approach instead of wrapping symbol resolution at link time
145 (i.e. during the build), via the `--Wl,-wrap,malloc` linker flag.
146 The use of this wrapping flag causes:
147 - All references to allocator symbols in the Chrome codebase to be rewritten as
148 references to `__wrap_malloc` and friends. The `__wrap_malloc` symbols are
149 defined in the `allocator_shim_override_linker_wrapped_symbols.h` and
150 route allocator calls inside the shim layer.
151 - The reference to the original `malloc` symbols (which typically is defined by
152 the system's libc.so) are accessible via the special `__real_malloc` and
153 friends symbols (which will be relocated, at load time, against `malloc`).
154
155 In summary, this approach is transparent to the dynamic loader, which still sees
156 undefined symbol references to malloc symbols.
157 These symbols will be resolved against libc.so as usual.
158 More details in [crrev.com/1719433002](https://crrev.com/1719433002).
159
160 **2. Shim layer implementation**
161 This stage contains the actual shim implementation. This consists of:
162 - A singly linked list of dispatchers (structs with function pointers to `malloc `-like functions). Dispatchers can be dynamically inserted at runtime
163 (using the `InsertAllocatorDispatch` API). They can intercept and override
164 allocator calls.
165 - The security checks (suicide on malloc-failure via `std::new_handler`, etc).
166 This happens inside `allocator_shim.cc`
167
168 **3. Final allocator routing**
169 The final element of the aforementioned dispatcher chain is statically defined
170 at build time and ultimately routes the allocator calls to the actual allocator
171 (as described in the *Background* section above). This is taken care of by the
172 headers in `allocator_shim_default_dispatch_to_*` files.
173
174
88 Appendixes 175 Appendixes
89 ---------- 176 ----------
90 **How does the Windows shim layer replace the malloc symbols?** 177 **How does the Windows shim layer replace the malloc symbols?**
91 The mechanism for hooking LIBCMT in Windows is rather tricky. The core 178 The mechanism for hooking LIBCMT in Windows is rather tricky. The core
92 problem is that by default, the Windows library does not declare malloc and 179 problem is that by default, the Windows library does not declare malloc and
93 free as weak symbols. Because of this, they cannot be overridden. To work 180 free as weak symbols. Because of this, they cannot be overridden. To work
94 around this, we start with the LIBCMT.LIB, and manually remove all allocator 181 around this, we start with the LIBCMT.LIB, and manually remove all allocator
95 related functions from it using the visual studio library tool. Once removed, 182 related functions from it using the visual studio library tool. Once removed,
96 we can now link against the library and provide custom versions of the 183 we can now link against the library and provide custom versions of the
97 allocator related functionality. 184 allocator related functionality.
98 See the script `preb_libc.py` in this folder. 185 See the script `preb_libc.py` in this folder.
99 186
100 Related links 187 Related links
101 ------------- 188 -------------
102 - [Allocator Cleanup Doc - Jan 2016][url-allocator-cleanup] 189 - [Unified allocator shim doc - Feb 2016][url-allocator-shim]
190 - [Allocator cleanup doc - Jan 2016][url-allocator-cleanup]
103 - [Proposal to use PartitionAlloc as default allocator](https://crbug.com/339604 ) 191 - [Proposal to use PartitionAlloc as default allocator](https://crbug.com/339604 )
104 - [Memory-Infra: Tools to profile memory usage in Chrome](components/tracing/doc s/memory_infra.md) 192 - [Memory-Infra: Tools to profile memory usage in Chrome](components/tracing/doc s/memory_infra.md)
105 193
106 [url-allocator-cleanup]: https://docs.google.com/document/d/1V77Kgp_4tfaaWPEZVxN evoD02wXiatnAv7Ssgr0hmjg/edit?usp=sharing 194 [url-allocator-cleanup]: https://docs.google.com/document/d/1V77Kgp_4tfaaWPEZVxN evoD02wXiatnAv7Ssgr0hmjg/edit?usp=sharing
195 [url-memory-infra-heap-profiler]: components/tracing/docs/heap_profiler.md
196 [url-allocator-shim]: https://docs.google.com/document/d/1yKlO1AO4XjpDad9rjcBOI1 5EKdAGsuGO_IeZy0g0kxo/edit?usp=sharing
OLDNEW
« no previous file with comments | « base/allocator/BUILD.gn ('k') | base/allocator/allocator.gyp » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698