Reland Combined test runner.

Reland Combined test runner.

This reverts commit 7a4374583efc0c41c826122aa26c1198c8d5cc37.

Original Commit Message:

This CL revives the old CL to combine the test runners [1] which was reverted
due to failing font_size tests. I've deleted the font_size.pdf as it is not
needed and for testing. Pixel tests are either .in or .pdf files, not both.

Original description:
This CL takes the three test runners (corpus, javascript, pixel) and combines
the code into a single test_runner file. Each of the individual runners still
exists and calls the test runner with their data directory.

With this change, the pixel and javascript test will now run in parallel if
multiple processors are available.

1-https://codereview.chromium.org/1430623006/

Committed: https://pdfium.googlesource.com/pdfium/+/9cec54ab0a5461b3075c585f8f233dbfd06c2cbd

[dsinclair, 2016-04-13 13:38:54]

dsinclair@chromium.org changed reviewers:
+ ochang@chromium.org, tsepez@chromium.org

[dsinclair, 2016-04-13 13:38:54]

PTAL.

Looking at the failed run DrMemory passed for Corpus tests and failed for Pixel
and JavaScript tests. The difference between those runs is DrMemory does a -j 1
for corpus tests specifically. This would also explain why all the errors were
Python stack traces.

I've forced the -j count to be 1 in PS2 in order to make DrMemory happy. This
will slow down corpus tests at the moment unless -j xx is passed. Once DrMemory
is updated to pass -j 1 to pixel and javascript tests I'll remove the 1 and go
back to multiprocessing.cpu_count().

[Tom Sepez, 2016-04-13 17:23:26]

On 2016/04/13 13:38:54, dsinclair wrote:
> PTAL.
> 
> Looking at the failed run DrMemory passed for Corpus tests and failed for
Pixel
> and JavaScript tests. The difference between those runs is DrMemory does a -j
1
> for corpus tests specifically. This would also explain why all the errors were
> Python stack traces.
> 
> I've forced the -j count to be 1 in PS2 in order to make DrMemory happy. This
> will slow down corpus tests at the moment unless -j xx is passed. Once
DrMemory
> is updated to pass -j 1 to pixel and javascript tests I'll remove the 1 and go
> back to multiprocessing.cpu_count().

Does this prevent parallel corpus tests on the bots?  I'm not willing to give
that up.

[dsinclair, 2016-04-13 17:26:17]

On 2016/04/13 17:23:26, Tom Sepez wrote:
> On 2016/04/13 13:38:54, dsinclair wrote:
> > PTAL.
> > 
> > Looking at the failed run DrMemory passed for Corpus tests and failed for
> Pixel
> > and JavaScript tests. The difference between those runs is DrMemory does a
-j
> 1
> > for corpus tests specifically. This would also explain why all the errors
were
> > Python stack traces.
> > 
> > I've forced the -j count to be 1 in PS2 in order to make DrMemory happy.
This
> > will slow down corpus tests at the moment unless -j xx is passed. Once
> DrMemory
> > is updated to pass -j 1 to pixel and javascript tests I'll remove the 1 and
go
> > back to multiprocessing.cpu_count().
> 
> Does this prevent parallel corpus tests on the bots?  I'm not willing to give
> that up.


I don't plan on giving up parallel tests. Once we update the bots to pass the
correct -j 1 to Dr Memory this will change back to the parallel version. The
other option is, we update the pixel and javascript tests to accept -j and
ignore it. Update Dr Memory to pass -j 1 and then land this CL without the 1
hack.

[dsinclair, 2016-04-13 17:42:35]

Description was changed from

==========
Reland Combined test runner.

This reverts commit 7a4374583efc0c41c826122aa26c1198c8d5cc37.

Original Commit Message:

This CL revives the old CL to combine the test runners [1] which was reverted
due to failing font_size tests. I've deleted the font_size.pdf as it is not
needed and for testing. Pixel tests are either .in or .pdf files, not both.

Original description:
This CL takes the three test runners (corpus, javascript, pixel) and combines
the code into a single test_runner file. Each of the individual runners still
exists and calls the test runner with their data directory.

With this change, the pixel and javascript test will now run in parallel if
multiple processors are available.

1-https://codereview.chromium.org/1430623006/
==========

to

==========
Reland Combined test runner.

[DO NOT COMMIT BEFORE UPDATING DR MEMORY]

This reverts commit 7a4374583efc0c41c826122aa26c1198c8d5cc37.

Original Commit Message:

This CL revives the old CL to combine the test runners [1] which was reverted
due to failing font_size tests. I've deleted the font_size.pdf as it is not
needed and for testing. Pixel tests are either .in or .pdf files, not both.

Original description:
This CL takes the three test runners (corpus, javascript, pixel) and combines
the code into a single test_runner file. Each of the individual runners still
exists and calls the test runner with their data directory.

With this change, the pixel and javascript test will now run in parallel if
multiple processors are available.

1-https://codereview.chromium.org/1430623006/
==========

[dsinclair, 2016-04-13 17:43:02]

On 2016/04/13 17:26:17, dsinclair wrote:
> On 2016/04/13 17:23:26, Tom Sepez wrote:
> > On 2016/04/13 13:38:54, dsinclair wrote:
> > > PTAL.
> > > 
> > > Looking at the failed run DrMemory passed for Corpus tests and failed for
> > Pixel
> > > and JavaScript tests. The difference between those runs is DrMemory does a
> -j
> > 1
> > > for corpus tests specifically. This would also explain why all the errors
> were
> > > Python stack traces.
> > > 
> > > I've forced the -j count to be 1 in PS2 in order to make DrMemory happy.
> This
> > > will slow down corpus tests at the moment unless -j xx is passed. Once
> > DrMemory
> > > is updated to pass -j 1 to pixel and javascript tests I'll remove the 1
and
> go
> > > back to multiprocessing.cpu_count().
> > 
> > Does this prevent parallel corpus tests on the bots?  I'm not willing to
give
> > that up.
> 
> 
> I don't plan on giving up parallel tests. Once we update the bots to pass the
> correct -j 1 to Dr Memory this will change back to the parallel version. The
> other option is, we update the pixel and javascript tests to accept -j and
> ignore it. Update Dr Memory to pass -j 1 and then land this CL without the 1
> hack.


Uploaded https://codereview.chromium.org/1882143005/ to add -j to pixel and
javascript. We can commit this after that lands and Dr Memory is updated.

[dsinclair, 2016-04-13 19:38:27]

Description was changed from

==========
Reland Combined test runner.

[DO NOT COMMIT BEFORE UPDATING DR MEMORY]

This reverts commit 7a4374583efc0c41c826122aa26c1198c8d5cc37.

Original Commit Message:

This CL revives the old CL to combine the test runners [1] which was reverted
due to failing font_size tests. I've deleted the font_size.pdf as it is not
needed and for testing. Pixel tests are either .in or .pdf files, not both.

Original description:
This CL takes the three test runners (corpus, javascript, pixel) and combines
the code into a single test_runner file. Each of the individual runners still
exists and calls the test runner with their data directory.

With this change, the pixel and javascript test will now run in parallel if
multiple processors are available.

1-https://codereview.chromium.org/1430623006/
==========

to

==========
Reland Combined test runner.

This reverts commit 7a4374583efc0c41c826122aa26c1198c8d5cc37.

Original Commit Message:

This CL revives the old CL to combine the test runners [1] which was reverted
due to failing font_size tests. I've deleted the font_size.pdf as it is not
needed and for testing. Pixel tests are either .in or .pdf files, not both.

Original description:
This CL takes the three test runners (corpus, javascript, pixel) and combines
the code into a single test_runner file. Each of the individual runners still
exists and calls the test runner with their data directory.

With this change, the pixel and javascript test will now run in parallel if
multiple processors are available.

1-https://codereview.chromium.org/1430623006/
==========

[dsinclair, 2016-04-13 19:39:06]

PTAL. The DrMemory change has landed to set -j1 which should fix the failure.

[Tom Sepez, 2016-04-13 20:04:16]

rs lgtm

[dsinclair, 2016-04-13 20:04:43]

The CQ bit was checked by dsinclair@chromium.org

[commit-bot: I haz the power, 2016-04-13 20:04:45]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1886753002/40001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1886753002/40001

[commit-bot: I haz the power, 2016-04-13 20:05:54]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-04-13 20:05:55]

Try jobs failed on following builders:
  win_xfa on master.tryserver.client.pdfium (JOB_FAILED,
https://build.chromium.org/p/tryserver.client.pdfium/builders/win_xfa/builds/232)

[dsinclair, 2016-04-13 20:09:41]

The CQ bit was checked by dsinclair@chromium.org

[dsinclair, 2016-04-13 20:09:41]

The patchset sent to the CQ was uploaded after l-g-t-m from tsepez@chromium.org
Link to the patchset: https://codereview.chromium.org/1886753002/#ps60001
(title: "Rebase to master")

[commit-bot: I haz the power, 2016-04-13 20:09:44]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1886753002/60001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1886753002/60001

[commit-bot: I haz the power, 2016-04-13 20:36:48]

Description was changed from

==========
Reland Combined test runner.

This reverts commit 7a4374583efc0c41c826122aa26c1198c8d5cc37.

Original Commit Message:

This CL revives the old CL to combine the test runners [1] which was reverted
due to failing font_size tests. I've deleted the font_size.pdf as it is not
needed and for testing. Pixel tests are either .in or .pdf files, not both.

Original description:
This CL takes the three test runners (corpus, javascript, pixel) and combines
the code into a single test_runner file. Each of the individual runners still
exists and calls the test runner with their data directory.

With this change, the pixel and javascript test will now run in parallel if
multiple processors are available.

1-https://codereview.chromium.org/1430623006/
==========

to

==========
Reland Combined test runner.

This reverts commit 7a4374583efc0c41c826122aa26c1198c8d5cc37.

Original Commit Message:

This CL revives the old CL to combine the test runners [1] which was reverted
due to failing font_size tests. I've deleted the font_size.pdf as it is not
needed and for testing. Pixel tests are either .in or .pdf files, not both.

Original description:
This CL takes the three test runners (corpus, javascript, pixel) and combines
the code into a single test_runner file. Each of the individual runners still
exists and calls the test runner with their data directory.

With this change, the pixel and javascript test will now run in parallel if
multiple processors are available.

1-https://codereview.chromium.org/1430623006/

Committed:
https://pdfium.googlesource.com/pdfium/+/9cec54ab0a5461b3075c585f8f233dbfd06c...
==========

[commit-bot: I haz the power, 2016-04-13 20:36:49]

Committed patchset #4 (id:60001) as
https://pdfium.googlesource.com/pdfium/+/9cec54ab0a5461b3075c585f8f233dbfd06c...

[dsinclair, 2016-04-14 02:13:11]

A revert of this CL (patchset #4 id:60001) has been created in
https://codereview.chromium.org/1890703002/ by dsinclair@chromium.org.

The reason for reverting is: Dr. Memory still broken:
https://build.chromium.org/p/client.pdfium/builders/drm_win_xfa/builds/330/st....