Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(553)

Side by Side Diff: docs/linux_crash_dumping.md

Issue 1324603002: [Docs] Another round of stylistic fixes. (Closed) Base URL: https://chromium.googlesource.com/chromium/src.git@master
Patch Set: Created 5 years, 3 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « docs/linux_chromium_packages.md ('k') | docs/linux_debugging.md » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 Official builds of Chrome support crash dumping and reporting using the Google c rash servers. This is a guide to how this works. 1 # Linux Crash Dumping
2 2
3 Official builds of Chrome support crash dumping and reporting using the Google
4 crash servers. This is a guide to how this works.
5
6 [TOC]
7
3 ## Breakpad 8 ## Breakpad
4 9
5 Breakpad is an open source library which we use for crash reporting across all t hree platforms (Linux, Mac and Windows). For Linux, a substantial amount of work was required to support cross-process dumping. At the time of writing this code is currently forked from the upstream breakpad repo. While this situation remai ns, the forked code lives in <tt>breakpad/linux</tt>. The upstream repo is mirro red in <tt>breakpad/src</tt>. 10 Breakpad is an open source library which we use for crash reporting across all
11 three platforms (Linux, Mac and Windows). For Linux, a substantial amount of
12 work was required to support cross-process dumping. At the time of writing this
13 code is currently forked from the upstream breakpad repo. While this situation
14 remains, the forked code lives in `breakpad/linux`. The upstream repo is
15 mirrored in `breakpad/src`.
6 16
7 The code currently supports i386 only. Getting x86-64 to work should only be a m inor amount of work. 17 The code currently supports i386 only. Getting x86-64 to work should only be a
18 minor amount of work.
8 19
9 ### Minidumps 20 ### Minidumps
10 21
11 Breakpad deals in a file format called 'minidumps'. This is a Microsoft format a nd thus is defined by in-memory structures which are dumped, raw, to disk. The m ain header file for this file format is <tt>breakpad/src/google_breakpad/common/ minidump_format.h</tt>. 22 Breakpad deals in a file format called 'minidumps'. This is a Microsoft format
23 and thus is defined by in-memory structures which are dumped, raw, to disk. The
24 main header file for this file format is
25 `breakpad/src/google_breakpad/common/minidump_format.h`.
12 26
13 At the top level, the minidump file format is a list of key-value pairs. Many of the keys are defined by the minidump format and contain cross-platform represen tations of stacks, threads etc. For Linux we also define a number of custom keys containing <tt>/proc/cpuinfo</tt>, <tt>lsb-release</tt> etc. These are defined in <tt>breakpad/linux/minidump_format_linux.h</tt>. 27 At the top level, the minidump file format is a list of key-value pairs. Many of
28 the keys are defined by the minidump format and contain cross-platform
29 representations of stacks, threads etc. For Linux we also define a number of
30 custom keys containing `/proc/cpuinfo`, `lsb-release` etc. These are defined in
31 `breakpad/linux/minidump_format_linux.h`.
14 32
15 ### Catching exceptions 33 ### Catching exceptions
16 34
17 Exceptional conditions (such as invalid memory references, floating point except ions, etc) are signaled by synchronous signals to the thread which caused them. Synchronous signals are always run on the thread which triggered them as opposed to asynchronous signals which can be handled by any thread in a thread-group wh ich hasn't masked that signal. 35 Exceptional conditions (such as invalid memory references, floating point
36 exceptions, etc) are signaled by synchronous signals to the thread which caused
37 them. Synchronous signals are always run on the thread which triggered them as
38 opposed to asynchronous signals which can be handled by any thread in a
39 thread-group which hasn't masked that signal.
18 40
19 All the signals that we wish to catch are synchronous except SIGABRT, and we can always arrange to send SIGABRT to a specific thread. Thus, we find the crashing thread by looking at the current thread in the signal handler. 41 All the signals that we wish to catch are synchronous except SIGABRT, and we can
42 always arrange to send SIGABRT to a specific thread. Thus, we find the crashing
43 thread by looking at the current thread in the signal handler.
20 44
21 The signal handlers run on a pre-allocated stack in case the crash was triggered by a stack overflow. 45 The signal handlers run on a pre-allocated stack in case the crash was triggered
46 by a stack overflow.
22 47
23 Once we have started handling the signal, we have to assume that the address spa ce is compromised. In order not to fall prey to this and crash (again) in the cr ash handler, we observe some rules: 48 Once we have started handling the signal, we have to assume that the address
24 1. We don't enter the dynamic linker. This, observably, can trigger crashes in the crash handler. Unfortunately, entering the dynamic linker is very easy and can be triggered by calling a function from a shared library who's resolution ha sn't been cached yet. Since we can't know which functions have been cached we av oid calling any of these functions with one exception: <tt>memcpy</tt>. Since th e compiler can emit calls to <tt>memcpy</tt> we can't really avoid it. 49 space is compromised. In order not to fall prey to this and crash (again) in the
25 1. We don't allocate memory via malloc as the heap may be corrupt. Instead we use a custom allocator (in <tt>breadpad/linux/memory.h</tt>) which gets clean pa ges directly from the kernel. 50 crash handler, we observe some rules:
26 51
27 In order to avoid calling into libc we have a couple of header files which wrap the system calls (<tt>linux_syscall_support.h</tt>) and reimplement a tiny subse t of libc (<tt>linux_libc_support.h</tt>). 52 1. We don't enter the dynamic linker. This, observably, can trigger crashes in
53 the crash handler. Unfortunately, entering the dynamic linker is very easy
54 and can be triggered by calling a function from a shared library who's
55 resolution hasn't been cached yet. Since we can't know which functions have
56 been cached we avoid calling any of these functions with one exception:
57 `memcpy`. Since the compiler can emit calls to `memcpy` we can't really
58 avoid it.
59 1. We don't allocate memory via malloc as the heap may be corrupt. Instead we
60 use a custom allocator (in `breadpad/linux/memory.h`) which gets clean pages
61 directly from the kernel.
62
63 In order to avoid calling into libc we have a couple of header files which wrap
64 the system calls (`linux_syscall_support.h`) and reimplement a tiny subset of
65 libc (`linux_libc_support.h`).
28 66
29 ### Self dumping 67 ### Self dumping
30 68
31 The simple case occurs when the browser process crashes. Here we catch the signa l and <tt>clone</tt> a new process to perform the dumping. We have to use a new process because a process cannot ptrace itself. 69 The simple case occurs when the browser process crashes. Here we catch the
70 signal and `clone` a new process to perform the dumping. We have to use a new
71 process because a process cannot ptrace itself.
32 72
33 The dumping process then ptrace attaches to all the threads in the crashed proce ss and writes out a minidump to <tt>/tmp</tt>. This is generic breakpad code. 73 The dumping process then ptrace attaches to all the threads in the crashed
74 process and writes out a minidump to `/tmp`. This is generic breakpad code.
34 75
35 Then we reach the Chrome specific parts in <tt>chrome/app/breakpad_linux.cc</tt> . Here we construct another temporary file and write a MIME wrapping of the cras h dump ready for uploading. We then fork off <tt>wget</tt> to upload the file. B ased on Debian popcorn, <tt>wget</tt> is very commonly installed (much more so t han <tt>libcurl</tt>) and <tt>wget</tt> handles the HTTPS gubbins for us. 76 Then we reach the Chrome specific parts in `chrome/app/breakpad_linux.cc`. Here
77 we construct another temporary file and write a MIME wrapping of the crash dump
78 ready for uploading. We then fork off `wget` to upload the file. Based on Debian
79 popcorn, `wget` is very commonly installed (much more so than `libcurl`) and
80 `wget` handles the HTTPS gubbins for us.
36 81
37 ### Renderer dumping 82 ### Renderer dumping
38 83
39 In the case of a crash in the renderer, we don't want the renderer handling the crash dumping itself. In the future we will sandbox the renderer and allowing it the authority to crash dump itself is too much. 84 In the case of a crash in the renderer, we don't want the renderer handling the
85 crash dumping itself. In the future we will sandbox the renderer and allowing it
86 the authority to crash dump itself is too much.
40 87
41 Thus, we split the crash dumping in two parts: the gathering of information whic h is done in process and the external dumping which is done out of process. In t he case above, the latter half was done in a <tt>clone</tt>d child. In this case , the browser process handles it. 88 Thus, we split the crash dumping in two parts: the gathering of information
89 which is done in process and the external dumping which is done out of process.
90 In the case above, the latter half was done in a `clone`d child. In this case,
91 the browser process handles it.
42 92
43 When renderers are forked off, they have a UNIX DGRAM socket in file descriptor 4. The signal handler then calls into Chrome specific code (<tt>chrome/renderer/ render_crash_handler_linux.cc</tt>) when it would otherwise <tt>clone</tt>. The Chrome specific code sends a datagram to the socket which contains: 93 When renderers are forked off, they have a `UNIX DGRAM` socket in file
44 * Information which is only available to the signal handler (such as the <tt>u context</tt> structure). 94 descriptor 4. The signal handler then calls into Chrome specific code
45 * A file descriptor to a pipe which it then blocks on reading from. 95 (`chrome/renderer/render_crash_handler_linux.cc`) when it would otherwise
46 * A <tt>CREDENTIALS</tt> structure giving its PID. 96 `clone`. The Chrome specific code sends a datagram to the socket which contains:
47 97
48 The kernel enforces that the renderer isn't lying in the <tt>CREDENTIALS</tt> st ructure so it can't ask the browser to crash dump another process. 98 * Information which is only available to the signal handler (such as the
99 `ucontext` structure).
100 * A file descriptor to a pipe which it then blocks on reading from.
101 * A `CREDENTIALS` structure giving its PID.
49 102
50 The browser then performs the ptrace and minidump writing which would otherwise be performed in the <tt>clone</tt>d process and does the MIME wrapping the uploa ding as normal. 103 The kernel enforces that the renderer isn't lying in the `CREDENTIALS` structure
104 so it can't ask the browser to crash dump another process.
51 105
52 Once the browser has finished getting information from the crashed renderer via ptrace, it writes a byte to the file descriptor which was passed from the render er. The renderer than wakes up (because it was blocking on reading from the othe r end) and rethrows the signal to itself. It then appears to crash 'normally' an d other parts of the browser notice the abnormal termination and display the sad tab. 106 The browser then performs the ptrace and minidump writing which would otherwise
107 be performed in the `clone`d process and does the MIME wrapping the uploading as
108 normal.
109
110 Once the browser has finished getting information from the crashed renderer via
111 ptrace, it writes a byte to the file descriptor which was passed from the
112 renderer. The renderer than wakes up (because it was blocking on reading from
113 the other end) and rethrows the signal to itself. It then appears to crash
114 'normally' and other parts of the browser notice the abnormal termination and
115 display the sad tab.
53 116
54 ## How to test Breakpad support in Chromium 117 ## How to test Breakpad support in Chromium
55 118
56 * Build Chromium with the gyp option `-Dlinux_breakpad=1`. 119 * Build Chromium with the gyp option `-Dlinux_breakpad=1`.
57 ``` 120
58 ./build/gyp_chromium -Dlinux_breakpad=1 121 ```shell
59 ninja -C out/Debug chrome 122 ./build/gyp_chromium -Dlinux_breakpad=1
60 ``` 123 ninja -C out/Debug chrome
61 * Run the browser with the environment variable [CHROME\_HEADLESS=1](http://co de.google.com/p/chromium/issues/detail?id=19663). This enables crash dumping bu t prevents crash dumps from being uploaded and deleted. 124 ```
62 ``` 125 * Run the browser with the environment variable
63 env CHROME_HEADLESS=1 ./out/Debug/chrome-wrapper 126 [CHROME_HEADLESS=1](https://crbug.com/19663). This enables crash dumping but
64 ``` 127 prevents crash dumps from being uploaded and deleted.
65 * Visit the special URL `about:crash` to trigger a crash in the renderer proce ss. 128
66 * A crash dump file should appear in the directory `~/.config/chromium/Crash R eports`. 129 ```shell
130 env CHROME_HEADLESS=1 ./out/Debug/chrome-wrapper
131 ```
132 * Visit the special URL `about:crash` to trigger a crash in the renderer
133 process.
134 * A crash dump file should appear in the directory
135 `~/.config/chromium/Crash Reports`.
OLDNEW
« no previous file with comments | « docs/linux_chromium_packages.md ('k') | docs/linux_debugging.md » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698