Add PDBs to .isolate files so that tests show stack traces.

Add PDBs to .isolate files so that tests show stack traces.

BUG=405074
R=scottmg@chromium.org
TBR=brettw, davidben

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=290612

[scottmg, 2014-08-19 17:00:02]

Why are these in _untracked?

[jam, 2014-08-19 17:52:35]

On 2014/08/19 17:00:02, scottmg wrote:
> Why are these in _untracked?

this is historical because originally these files were created using a system
that traced the files. tracked was for the files that the tracing system knew
about. untracked where files that it missed or that it couldn't know about so
they were listed in a separate section so that they're not overwritten later.

[scottmg, 2014-08-19 18:00:44]

On 2014/08/19 17:52:35, jam wrote:
> On 2014/08/19 17:00:02, scottmg wrote:
> > Why are these in _untracked?
> 
> this is historical because originally these files were created using a system
> that traced the files. tracked was for the files that the tracing system knew
> about. untracked where files that it missed or that it couldn't know about so
> they were listed in a separate section so that they're not overwritten later.

OK, so we'll get rid of that at some point? Or we're still using the tool?

LGTM

[jam, 2014-08-19 18:03:49]

On 2014/08/19 18:00:44, scottmg wrote:
> On 2014/08/19 17:52:35, jam wrote:
> > On 2014/08/19 17:00:02, scottmg wrote:
> > > Why are these in _untracked?
> > 
> > this is historical because originally these files were created using a
system
> > that traced the files. tracked was for the files that the tracing system
knew
> > about. untracked where files that it missed or that it couldn't know about
so
> > they were listed in a separate section so that they're not overwritten
later.
> 
> OK, so we'll get rid of that at some point? Or we're still using the tool?

I imagine we would; I'll defer to Marc-Antoine on if/when we can get rid of
these separate sections.

[jam, 2014-08-19 18:05:39]

TBR brettw for base, davidben for net

[davidben, 2014-08-19 21:54:27]

net lgtm

[M-A Ruel, 2014-08-25 16:06:26]

They should be _tracked and keyed on fastbuild==0 or fastbuild==1. See as an
example:
https://codereview.chromium.org/448773002/diff/40001/chrome/telemetry_gpu_tes...

[jam, 2014-08-25 19:03:42]

Thanks, two questions:

-why _tracked? wasn't that for stuff that the tracing system noticed was used?
in this case, that wouldn't be automatically used unless a crash occurred.
  -also why do we still have tracked and untracked, should we merge them now?
-why fastbuild==0 or fastbuild==1? if fastbuild==2, that signifies a
compile-only build, and that shouldn't be using isolate by definition since
there are no tests

On 2014/08/25 16:06:26, M-A Ruel wrote:
> They should be _tracked and keyed on fastbuild==0 or fastbuild==1. See as an
> example:
>
https://codereview.chromium.org/448773002/diff/40001/chrome/telemetry_gpu_tes...

[M-A Ruel, 2014-08-25 20:07:05]

On 2014/08/25 19:03:42, jam wrote:
> Thanks, two questions:
> 
> -why _tracked? wasn't that for stuff that the tracing system noticed was used?

No, it's for things that are real files. _untracked is what couldn't be used by
ninja for dependency graph, like directories or symlinks.

> in this case, that wouldn't be automatically used unless a crash occurred.
>   -also why do we still have tracked and untracked, should we merge them now?

I agree the point is moot and it's likely something we should now get rid of.

> -why fastbuild==0 or fastbuild==1? if fastbuild==2, that signifies a
> compile-only build, and that shouldn't be using isolate by definition since
> there are no tests

fastbuild=2 means not PDB.
fastbuild=1 means link-time PDB.
fastbuild=0 means normal compile-time PDB

So only with fastbuild=2 there is no PDB to isolate. But isolate conditions are
inclusive, not exclusive, so you have to do fastbuild==0 or fastbuild==1.

[jam, 2014-08-25 20:12:51]

On 2014/08/25 20:07:05, M-A Ruel wrote:
> On 2014/08/25 19:03:42, jam wrote:
> > Thanks, two questions:
> > 
> > -why _tracked? wasn't that for stuff that the tracing system noticed was
used?
> 
> No, it's for things that are real files. _untracked is what couldn't be used
by
> ninja for dependency graph, like directories or symlinks.
> 
> > in this case, that wouldn't be automatically used unless a crash occurred.
> >   -also why do we still have tracked and untracked, should we merge them
now?
> 
> I agree the point is moot and it's likely something we should now get rid of.
> 
> > -why fastbuild==0 or fastbuild==1? if fastbuild==2, that signifies a
> > compile-only build, and that shouldn't be using isolate by definition since
> > there are no tests
> 
> fastbuild=2 means not PDB.
> fastbuild=1 means link-time PDB.
> fastbuild=0 means normal compile-time PDB
> 
> So only with fastbuild=2 there is no PDB to isolate.

Right, but fastbuild==2 is only used on compile-only builders. That means these
isolate files shouldn't be used at all.

> But isolate conditions are
> inclusive, not exclusive, so you have to do fastbuild==0 or fastbuild==1.

[M-A Ruel, 2014-08-25 20:16:37]

On 2014/08/25 20:12:51, jam wrote:
> On 2014/08/25 20:07:05, M-A Ruel wrote:
> > On 2014/08/25 19:03:42, jam wrote:
> > > Thanks, two questions:
> > > 
> > > -why _tracked? wasn't that for stuff that the tracing system noticed was
> used?
> > 
> > No, it's for things that are real files. _untracked is what couldn't be used
> by
> > ninja for dependency graph, like directories or symlinks.
> > 
> > > in this case, that wouldn't be automatically used unless a crash occurred.
> > >   -also why do we still have tracked and untracked, should we merge them
> now?
> > 
> > I agree the point is moot and it's likely something we should now get rid
of.
> > 
> > > -why fastbuild==0 or fastbuild==1? if fastbuild==2, that signifies a
> > > compile-only build, and that shouldn't be using isolate by definition
since
> > > there are no tests
> > 
> > fastbuild=2 means not PDB.
> > fastbuild=1 means link-time PDB.
> > fastbuild=0 means normal compile-time PDB
> > 
> > So only with fastbuild=2 there is no PDB to isolate.
> 
> Right, but fastbuild==2 is only used on compile-only builders. That means
these
> isolate files shouldn't be used at all.

Yes but compile-only builders will still fail if the *_run targets are "built",
which really mean generating the .isolated file in this case. So generally
speaking, it's highly preferable to have this right.

[jam, 2014-08-25 20:29:06]

On 2014/08/25 20:16:37, M-A Ruel wrote:
> On 2014/08/25 20:12:51, jam wrote:
> > On 2014/08/25 20:07:05, M-A Ruel wrote:
> > > On 2014/08/25 19:03:42, jam wrote:
> > > > Thanks, two questions:
> > > > 
> > > > -why _tracked? wasn't that for stuff that the tracing system noticed was
> > used?
> > > 
> > > No, it's for things that are real files. _untracked is what couldn't be
used
> > by
> > > ninja for dependency graph, like directories or symlinks.
> > > 
> > > > in this case, that wouldn't be automatically used unless a crash
occurred.
> > > >   -also why do we still have tracked and untracked, should we merge them
> > now?
> > > 
> > > I agree the point is moot and it's likely something we should now get rid
> of.
> > > 
> > > > -why fastbuild==0 or fastbuild==1? if fastbuild==2, that signifies a
> > > > compile-only build, and that shouldn't be using isolate by definition
> since
> > > > there are no tests
> > > 
> > > fastbuild=2 means not PDB.
> > > fastbuild=1 means link-time PDB.
> > > fastbuild=0 means normal compile-time PDB
> > > 
> > > So only with fastbuild=2 there is no PDB to isolate.
> > 
> > Right, but fastbuild==2 is only used on compile-only builders. That means
> these
> > isolate files shouldn't be used at all.
> 
> Yes but compile-only builders will still fail if the *_run targets are
"built",
> which really mean generating the .isolated file in this case. So generally
> speaking, it's highly preferable to have this right.

Why would we run the _run steps for compile only steps? this slows down the
compile for binaries that won't be used.

[M-A Ruel, 2014-08-25 20:34:40]

On 2014/08/25 20:29:06, jam wrote:
> Why would we run the _run steps for compile only steps? this slows down the
> compile for binaries that won't be used.

To ensure that the .isolate is valid on that configuration?

[jam, 2014-08-25 20:46:15]

On 2014/08/25 20:34:40, M-A Ruel wrote:
> On 2014/08/25 20:29:06, jam wrote:
> > Why would we run the _run steps for compile only steps? this slows down the
> > compile for binaries that won't be used.
> 
> To ensure that the .isolate is valid on that configuration?

What would be the need to ensure that an isolate, which is used for distributing
tests, is valid on a system with no tests? even ignoring the time it takes to
upload this data to the isolate servers, this is extra work that developers have
to maintain with no benefit.