Re-Reland Combined test runner.

Re-Reland Combined test runner.

Original Commit Message:

This CL revives the old CL to combine the test runners [1] which was reverted
due to failing font_size tests. I've deleted the font_size.pdf as it is not
needed and for testing. Pixel tests are either .in or .pdf files, not both.

Original description:
This CL takes the three test runners (corpus, javascript, pixel) and combines
the code into a single test_runner file. Each of the individual runners still
exists and calls the test runner with their data directory.

With this change, the pixel and javascript test will now run in parallel if
multiple processors are available.

1-https://codereview.chromium.org/1430623006/

BUG=

Committed: https://pdfium.googlesource.com/pdfium/+/84b26ec6ba8193235ce0df817dec5c0f1da29841

[dsinclair, 2016-04-14 13:21:00]

dsinclair@chromium.org changed reviewers:
+ ochang@chromium.org

[dsinclair, 2016-04-14 13:21:00]

I don't know why this breaks Dr.Memory. It's just the pixel and Javascript
tests, Corpus seems to be fine. Is there something that suppresses for corpus
that isn't kicking in for the other two types of tests?

You can see a run at: 
https://build.chromium.org/p/client.pdfium/builders/drm_win_xfa/builds/330/

The errors all seem to come from python?

~Dr.M~~ Error #15: UNINITIALIZED READ: reading register edx
~~Dr.M~~ # 0 python27.dll!PyObject_Free              +0x18     (0x1e0ad138
<python27.dll+0xad138>)
~~Dr.M~~ # 1 python27.dll!PyCode_NewEmpty            +0x4b8    (0x1e089df9
<python27.dll+0x89df9>)
~~Dr.M~~ # 2 python27.dll!PyFunction_SetClosure      +0x59a    (0x1e09d45b
<python27.dll+0x9d45b>)
~~Dr.M~~ # 3 python27.dll!PyDict_MaybeUntrack        +0x1a8    (0x1e08e309
<python27.dll+0x8e309>)
~~Dr.M~~ # 4 python27.dll!PyDict_MaybeUntrack        +0x25d    (0x1e08e3be
<python27.dll+0x8e3be>)

[Oliver Chang, 2016-04-14 15:49:06]

ochang@chromium.org changed reviewers:
+ zhaoqin@chromium.org

[Oliver Chang, 2016-04-14 15:49:06]

+zhaoqin. Any ideas why this is happening?

[zhaoqin1, 2016-04-15 15:16:07]

On 2016/04/14 15:49:06, Oliver Chang wrote:
> +zhaoqin. Any ideas why this is happening?

On 2016/04/14 15:49:06, Oliver Chang wrote:
> +zhaoqin. Any ideas why this is happening?

From the error report, it looks like python is running under Dr. Memory.
However, it should have been excluded, and I can see the command that exclude
python.
https://build.chromium.org/p/client.pdfium/builders/drm_win_xfa/builds/330/st...
...
Extracting  bin

Everything is Ok
16:24:58 pdfium_tests.py [INFO] running test pdfium_javascript
16:24:58 valgrind_test.py [INFO] starting execution...
16:24:58 valgrind_test.py [INFO] export CHROME_ALLOCATOR=WINHEAP
16:24:58 valgrind_test.py [INFO] export JSIMD_FORCEMMX=1
16:24:58 valgrind_test.py [INFO] Excluding python children
16:24:58 common.py [INFO] running
E:\b\build\slave\drm_win_xfa\build\pdfium\tools\drmemory\unpacked\bin\drconfig.exe
-quiet -reg python.exe -norun, timeout 100000 sec

In the old code, Dr. Memory directly run with pdfium_test:
  # add Dr. Memory wrapper if exist
  cmd_to_run = common.DrMemoryWrapper(drmem_wrapper, input_root)	
  cmd_to_run.extend([pdfium_test_path, pdf_path])
In the new code, it seems Dr. Memory will run the fixup python script first:
  # Add Dr. Memory wrapper if exists.
  cmd = common.DrMemoryWrapper(self.drmem_wrapper, input_root)
  cmd.extend([sys.executable, self.fixup_path,
              '--output-dir=' + self.working_dir, input_path])

[zhaoqin1, 2016-04-15 15:16:25]

https://codereview.chromium.org/1888673003/diff/1/testing/tools/run_javascrip...
File testing/tools/run_javascript_tests.py (left):

https://codereview.chromium.org/1888673003/diff/1/testing/tools/run_javascrip...
testing/tools/run_javascript_tests.py:35: cmd_to_run.extend([pdfium_test_path,
pdf_path])
we run DrM with the pdfium test directly.

https://codereview.chromium.org/1888673003/diff/1/testing/tools/test_runner.py
File testing/tools/test_runner.py (right):

https://codereview.chromium.org/1888673003/diff/1/testing/tools/test_runner.p...
testing/tools/test_runner.py:103: '--output-dir=' + self.working_dir,
input_path])
it seems the fixup script is extend here, which could be the cause?

[dsinclair, 2016-04-18 13:53:27]

On 2016/04/15 15:16:25, zhaoqin1 wrote:
>
https://codereview.chromium.org/1888673003/diff/1/testing/tools/run_javascrip...
> File testing/tools/run_javascript_tests.py (left):
> 
>
https://codereview.chromium.org/1888673003/diff/1/testing/tools/run_javascrip...
> testing/tools/run_javascript_tests.py:35: cmd_to_run.extend([pdfium_test_path,
> pdf_path])
> we run DrM with the pdfium test directly.
> 
> https://codereview.chromium.org/1888673003/diff/1/testing/tools/test_runner.py
> File testing/tools/test_runner.py (right):
> 
>
https://codereview.chromium.org/1888673003/diff/1/testing/tools/test_runner.p...
> testing/tools/test_runner.py:103: '--output-dir=' + self.working_dir,
> input_path])
> it seems the fixup script is extend here, which could be the cause?


Except this is working for Corpus running through this same script. All three
tests, pixel, javascript and corpus, all run with this script. Only pixel and
javascript are failing. If the script was wrong, wouldn't corpus also be
failing?

[zhaoqin1, 2016-04-18 15:25:06]

On 2016/04/18 13:53:27, dsinclair wrote:
> On 2016/04/15 15:16:25, zhaoqin1 wrote:
> >
>
https://codereview.chromium.org/1888673003/diff/1/testing/tools/run_javascrip...
> > File testing/tools/run_javascript_tests.py (left):
> > 
> >
>
https://codereview.chromium.org/1888673003/diff/1/testing/tools/run_javascrip...
> > testing/tools/run_javascript_tests.py:35:
cmd_to_run.extend([pdfium_test_path,
> > pdf_path])
> > we run DrM with the pdfium test directly.
> > 
> >
https://codereview.chromium.org/1888673003/diff/1/testing/tools/test_runner.py
> > File testing/tools/test_runner.py (right):
> > 
> >
>
https://codereview.chromium.org/1888673003/diff/1/testing/tools/test_runner.p...
> > testing/tools/test_runner.py:103: '--output-dir=' + self.working_dir,
> > input_path])
> > it seems the fixup script is extend here, which could be the cause?
> 
> 
> Except this is working for Corpus running through this same script. All three
> tests, pixel, javascript and corpus, all run with this script. Only pixel and
> javascript are failing. If the script was wrong, wouldn't corpus also be
> failing?

The original run_corpus_tests.py does not run fixup script, maybe the new
test_runner.py skipped fixup script somehow too?
e.g., self.fixup_path = finder.ScriptPath('fixup_pdf_template.py') get NULL or
something?

[dsinclair, 2016-04-18 17:03:31]

Patchset #2 (id:20001) has been deleted

[dsinclair, 2016-04-18 17:04:17]

PTAL.

Moved the fixup call outside of the Dr. Memory call.

[dsinclair, 2016-04-19 13:24:17]

dsinclair@chromium.org changed reviewers:
+ tsepez@chromium.org

[dsinclair, 2016-04-19 13:24:17]

ping.

Tom, can you take a quick peek to make sure I merged your change correctly?

[Tom Sepez, 2016-04-19 17:43:37]

lgtm

https://codereview.chromium.org/1888673003/diff/60001/testing/tools/test_runn...
File testing/tools/test_runner.py (right):

https://codereview.chromium.org/1888673003/diff/60001/testing/tools/test_runn...
testing/tools/test_runner.py:171: '--show-config'])
nit: alignment with opening [

[dsinclair, 2016-04-19 20:08:02]

https://codereview.chromium.org/1888673003/diff/60001/testing/tools/test_runn...
File testing/tools/test_runner.py (right):

https://codereview.chromium.org/1888673003/diff/60001/testing/tools/test_runn...
testing/tools/test_runner.py:171: '--show-config'])
On 2016/04/19 17:43:37, Tom Sepez wrote:
> nit: alignment with opening [

Done.

[dsinclair, 2016-04-20 12:54:01]

The CQ bit was checked by dsinclair@chromium.org

[dsinclair, 2016-04-20 12:54:02]

The patchset sent to the CQ was uploaded after l-g-t-m from tsepez@chromium.org
Link to the patchset: https://codereview.chromium.org/1888673003/#ps80001
(title: " ")

[commit-bot: I haz the power, 2016-04-20 12:54:03]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1888673003/80001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1888673003/80001

[commit-bot: I haz the power, 2016-04-20 12:54:18]

Description was changed from

==========
Re-Reland Combined test runner.

Original Commit Message:

This CL revives the old CL to combine the test runners [1] which was reverted
due to failing font_size tests. I've deleted the font_size.pdf as it is not
needed and for testing. Pixel tests are either .in or .pdf files, not both.

Original description:
This CL takes the three test runners (corpus, javascript, pixel) and combines
the code into a single test_runner file. Each of the individual runners still
exists and calls the test runner with their data directory.

With this change, the pixel and javascript test will now run in parallel if
multiple processors are available.

1-https://codereview.chromium.org/1430623006/

BUG=
==========

to

==========
Re-Reland Combined test runner.

Original Commit Message:

This CL revives the old CL to combine the test runners [1] which was reverted
due to failing font_size tests. I've deleted the font_size.pdf as it is not
needed and for testing. Pixel tests are either .in or .pdf files, not both.

Original description:
This CL takes the three test runners (corpus, javascript, pixel) and combines
the code into a single test_runner file. Each of the individual runners still
exists and calls the test runner with their data directory.

With this change, the pixel and javascript test will now run in parallel if
multiple processors are available.

1-https://codereview.chromium.org/1430623006/

BUG=

Committed:
https://pdfium.googlesource.com/pdfium/+/84b26ec6ba8193235ce0df817dec5c0f1da2...
==========

[commit-bot: I haz the power, 2016-04-20 12:54:19]

Committed patchset #4 (id:80001) as
https://pdfium.googlesource.com/pdfium/+/84b26ec6ba8193235ce0df817dec5c0f1da2...

[dsinclair, 2016-04-20 14:05:18]

A revert of this CL (patchset #4 id:80001) has been created in
https://codereview.chromium.org/1898353004/ by dsinclair@chromium.org.

The reason for reverting is: Dr.Memory code is wrong......