Allow running individual tests.

Allow running individual tests.

This CL adds the ability to run a given test from the corpus, javascript and
pixel test runners. The filename to provide is relative to the testing
directory in question.

Because the directories for javascript and pixel are flat you just provide the filename (it will rewrite the .pdf to .in if .pdf is provided). For corpus tests you have to provide the path from the corpus directory.


Development/pdfium/pdfium % ./testing/tools/run_javascript_tests.py apply.pdf
Rendering PDF file /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/javascript/apply.pdf.
Non-linearized path...
Rendered 1 pages.
Skipped 0 bad pages.


Development/pdfium/pdfium % ./testing/tools/run_pixel_tests.py bug_524043_1.pdf
Rendering PDF file /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/pixel/bug_524043_1.pdf.
Linearized path...
Rendered 1 pages.
Skipped 0 bad pages.
Checking /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/pixel/bug_524043_1.pdf.0.png
diff: 0.00% passed


Development/pdfium/pdfium % ./testing/tools/run_corpus_tests.py third_party/tcpdf/example_065.pdf
Rendering PDF file /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/corpus/example_065.pdf.
Non-linearized path...
Rendered 1 pages.
Skipped 0 bad pages.
Checking /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/corpus/example_065.pdf.0.png
diff: 0.14% failed
FAILURE: example_065.pdf; Command '['/Development/pdfium/pdfium/out/Debug/pdfium_diff', '/Development/pdfium/pdfium/testing/corpus/third_party/tcpdf/example_065_expected.pdf.0.png', '/Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/corpus/example_065.pdf.0.png']' returned non-zero exit status 1


Summary of Failures:
/Development/pdfium/pdfium/testing/corpus/third_party/tcpdf/example_065.pdf

R=thestig@chromium.org

Committed: https://pdfium.googlesource.com/pdfium/+/da6254ff78553773b7bb637ee1e3f5111c083c74

[dsinclair, 2015-10-28 19:37:35]

Description was changed from

==========
Allow running individual tests.

This CL adds the ability to run a given test from the corpus, javascript and
pixel test runners. The filename to provide is relative to the testing
directory in question.
==========

to

==========
Allow running individual tests.

This CL adds the ability to run a given test from the corpus, javascript and
pixel test runners. The filename to provide is relative to the testing
directory in question.

Because the directories for javascript and pixel are flat you just provide the
filename (it will rewrite the .pdf to .in if .pdf is provided). For corpus tests
you have to provide the path from the corpus directory.


Development/pdfium/pdfium % ./testing/tools/run_javascript_tests.py apply.pdf
Rendering PDF file
/Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/javascript/apply.pdf.
Non-linearized path...
Rendered 1 pages.
Skipped 0 bad pages.


Development/pdfium/pdfium % ./testing/tools/run_pixel_tests.py bug_524043_1.pdf
Rendering PDF file
/Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/pixel/bug_524043_1.pdf.
Linearized path...
Rendered 1 pages.
Skipped 0 bad pages.
Checking
/Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/pixel/bug_524043_1.pdf.0.png
diff: 0.00% passed


Development/pdfium/pdfium % ./testing/tools/run_corpus_tests.py
third_party/tcpdf/example_065.pdf
Rendering PDF file
/Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/corpus/example_065.pdf.
Non-linearized path...
Rendered 1 pages.
Skipped 0 bad pages.
Checking
/Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/corpus/example_065.pdf.0.png
diff: 0.14% failed
FAILURE: example_065.pdf; Command
'['/Development/pdfium/pdfium/out/Debug/pdfium_diff',
'/Development/pdfium/pdfium/testing/corpus/third_party/tcpdf/example_065_expected.pdf.0.png',
'/Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/corpus/example_065.pdf.0.png']'
returned non-zero exit status 1


Summary of Failures:
/Development/pdfium/pdfium/testing/corpus/third_party/tcpdf/example_065.pdf
==========

[dsinclair, 2015-10-28 19:39:01]

dsinclair@chromium.org changed reviewers:
+ thestig@chromium.org, tsepez@chromium.org

[dsinclair, 2015-10-28 19:39:02]

PTAL.

I wanted to be able to run individual tests without having to run the whole
corpus. There is some cleanup that could be done, javascript and pixel look like
they're almost copies of each other, corpus is different. That seemed like a
larger CL and wanted to try to land this first.

Note, I don't know python, so please give it a good read ....

[Tom Sepez, 2015-10-28 20:25:49]

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_corpus_te...
File testing/tools/run_corpus_tests.py (right):

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_corpus_te...
testing/tools/run_corpus_tests.py:108: if len(args) != 0:
nit: can just write if len(args):

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_corpus_te...
testing/tools/run_corpus_tests.py:111: if os.path.isfile(input_path):
This is to filter out directories, you shouldn't be passing a directory as an
arg, so you'd want to error out if this isn't the case.

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_corpus_te...
testing/tools/run_corpus_tests.py:113: os.path.dirname(input_path)))
nit: this should line up under the above.

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_javascrip...
File testing/tools/run_javascript_tests.py (right):

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_javascrip...
testing/tools/run_javascript_tests.py:62: input_files.append(file_name)
nit: assignment not needed, combine with previous line.

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_javascrip...
testing/tools/run_javascript_tests.py:64: if len(input_files) == 0:
Why not do the same if len(args): ... else: as in the other file?

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_pixel_tes...
File testing/tools/run_pixel_tests.py (right):

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_pixel_tes...
testing/tools/run_pixel_tests.py:69: input_files = []
ditto

[dsinclair, 2015-10-28 20:56:46]

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_corpus_te...
File testing/tools/run_corpus_tests.py (right):

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_corpus_te...
testing/tools/run_corpus_tests.py:108: if len(args) != 0:
On 2015/10/28 at 20:25:48, Tom Sepez wrote:
> nit: can just write if len(args):

Done.

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_corpus_te...
testing/tools/run_corpus_tests.py:111: if os.path.isfile(input_path):
On 2015/10/28 at 20:25:48, Tom Sepez wrote:
> This is to filter out directories, you shouldn't be passing a directory as an
arg, so you'd want to error out if this isn't the case.

Done.

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_corpus_te...
testing/tools/run_corpus_tests.py:113: os.path.dirname(input_path)))
On 2015/10/28 at 20:25:48, Tom Sepez wrote:
> nit: this should line up under the above.

Like this?

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_javascrip...
File testing/tools/run_javascript_tests.py (right):

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_javascrip...
testing/tools/run_javascript_tests.py:62: input_files.append(file_name)
On 2015/10/28 at 20:25:48, Tom Sepez wrote:
> nit: assignment not needed, combine with previous line.

Done.

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_javascrip...
testing/tools/run_javascript_tests.py:64: if len(input_files) == 0:
On 2015/10/28 at 20:25:48, Tom Sepez wrote:
> Why not do the same if len(args): ... else: as in the other file?

Done.

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_pixel_tes...
File testing/tools/run_pixel_tests.py (right):

https://codereview.chromium.org/1407913005/diff/1/testing/tools/run_pixel_tes...
testing/tools/run_pixel_tests.py:69: input_files = []
On 2015/10/28 at 20:25:48, Tom Sepez wrote:
> ditto

Done.

[Tom Sepez, 2015-10-28 20:58:16]

Looks good but get Lei's (stronger python) opinion.

[Lei Zhang, 2015-10-28 21:02:41]

lgtm

I had started commenting, but noticed Tom was already finished.

[dsinclair, 2015-10-28 21:06:08]

Committed patchset #2 (id:20001) manually as
da6254ff78553773b7bb637ee1e3f5111c083c74 (presubmit successful).