crash-reporter: Capture and send recent update_engine logs when it crashes

crash-reporter: Capture and send recent update_engine logs when it crashes

We also disallow more than 4 nested crashes.  This way we know if core2md crashes for instance, but not if crash_reporter crashes while handling the core2md crash that came from crash_reporter.  Note that the kernel does not know about parent/child process relationships, it just counts concurrent handling, so it is possible that some of many crashing processes will be discarded.

See crash report id 473c22ed428d33a9 for an example report with update_engine logs.

Change-Id: I9ff9f03a94dc87d2d42840511c2e5e42ee37cea8

BUG=9176
TEST=UserCrash,CrashSender,unit tests

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=50ae33b

[petkov, 2010-12-13 19:27:57]

Mostly nits, I think.

Have you verified that you're getting the log on the crash server? I tried
downloading it and i couldn't.

It might make sense to update the CL description to mention that you're also
restricting the crash reports to one at a time.

http://codereview.chromium.org/5814001/diff/1/crash_collector_test.cc
File crash_collector_test.cc (right):

http://codereview.chromium.org/5814001/diff/1/crash_collector_test.cc#newcode388
crash_collector_test.cc:388: "foobar:sh -c 'echo hello there | sed -e
\"s/there/world/\"'";
do you intentionally use "sh -c" here given that the code executes the command
through sh?

http://codereview.chromium.org/5814001/diff/1/crash_reporter_logs.conf
File crash_reporter_logs.conf (right):

http://codereview.chromium.org/5814001/diff/1/crash_reporter_logs.conf#newcode1
crash_reporter_logs.conf:1: # Be careful to only run common unix commands here
as these commands
it doesn't hurt to add a Chromium OS copyright.

http://codereview.chromium.org/5814001/diff/1/crash_reporter_logs.conf#newcode2
crash_reporter_logs.conf:2: # will be run when a crash has recently occurred and
we should avoid
briefly document the format somewhere -- either her, or in the header file? a
one-liner will do.

http://codereview.chromium.org/5814001/diff/1/crash_reporter_logs.conf#newcode4
crash_reporter_logs.conf:4: update_engine:tail -c 50000
/var/log/update_engine.log
since update engine starts a new log every time it's started, can we do
something similar to what we do for chrome:system to collect data from the last
few logs? I think the command below will include the last 5 logs, at most 50KB
total.

cat $(ls -1tr /var/log/update_engine | tail -5 | sed
s.^./var/log/update_engine/.) | tail -c 50000

http://codereview.chromium.org/5814001/diff/1/user_collector.cc
File user_collector.cc (right):

http://codereview.chromium.org/5814001/diff/1/user_collector.cc#newcode34
user_collector.cc:34: // Set core_pipe_limit to 1 so that any crashes occurring
while crash_reporter is
80 chars

http://codereview.chromium.org/5814001/diff/1/user_collector.cc#newcode351
user_collector.cc:351: std::string log_contents;
unused? why are you not compiling with -Wall -Werror anyway? :)

[kmixter1, 2010-12-14 03:39:49]

PTAL

Btw, change description updated including an example crash report id for
reference.

http://codereview.chromium.org/5814001/diff/1/crash_collector_test.cc
File crash_collector_test.cc (right):

http://codereview.chromium.org/5814001/diff/1/crash_collector_test.cc#newcode388
crash_collector_test.cc:388: "foobar:sh -c 'echo hello there | sed -e
\"s/there/world/\"'";
On 2010/12/13 19:27:57, petkov wrote:
> do you intentionally use "sh -c" here given that the code executes the command
> through sh?

No, not sure why I was doing that.  Changed it.

http://codereview.chromium.org/5814001/diff/1/crash_reporter_logs.conf
File crash_reporter_logs.conf (right):

http://codereview.chromium.org/5814001/diff/1/crash_reporter_logs.conf#newcode1
crash_reporter_logs.conf:1: # Be careful to only run common unix commands here
as these commands
On 2010/12/13 19:27:57, petkov wrote:
> it doesn't hurt to add a Chromium OS copyright.

Done.

http://codereview.chromium.org/5814001/diff/1/crash_reporter_logs.conf#newcode2
crash_reporter_logs.conf:2: # will be run when a crash has recently occurred and
we should avoid
On 2010/12/13 19:27:57, petkov wrote:
> briefly document the format somewhere -- either her, or in the header file? a
> one-liner will do.

Done.

http://codereview.chromium.org/5814001/diff/1/crash_reporter_logs.conf#newcode4
crash_reporter_logs.conf:4: update_engine:tail -c 50000
/var/log/update_engine.log
On 2010/12/13 19:27:57, petkov wrote:
> since update engine starts a new log every time it's started, can we do
> something similar to what we do for chrome:system to collect data from the
last
> few logs? I think the command below will include the last 5 logs, at most 50KB
> total.
> 
> cat $(ls -1tr /var/log/update_engine | tail -5 | sed
> s.^./var/log/update_engine/.) | tail -c 50000

Done.

http://codereview.chromium.org/5814001/diff/1/user_collector.cc
File user_collector.cc (right):

http://codereview.chromium.org/5814001/diff/1/user_collector.cc#newcode34
user_collector.cc:34: // Set core_pipe_limit to 1 so that any crashes occurring
while crash_reporter is
On 2010/12/13 19:27:57, petkov wrote:
> 80 chars

Done.

http://codereview.chromium.org/5814001/diff/1/user_collector.cc#newcode351
user_collector.cc:351: std::string log_contents;
On 2010/12/13 19:27:57, petkov wrote:
> unused? why are you not compiling with -Wall -Werror anyway? :)
> 

I am compiling with them.  Guessing they don't catch locals that are
constructed.  Removed, anyway.

[petkov, 2010-12-14 18:58:58]

LGTM

http://codereview.chromium.org/5814001/diff/7001/user_collector.cc
File user_collector.cc (right):

http://codereview.chromium.org/5814001/diff/7001/user_collector.cc#newcode36
user_collector.cc:36: static const char kCorePipeLimit[] = "4";
the CL description says more than 2 crashes.