Refactor make-file-arrays.py to ease reuse

Refactor make-file-arrays.py to ease reuse

Per: "Retiring build-time execution of Perl"
https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/fnWX8n6JrTg
...we should be able to replace these 2 Perl files:
xxd.pl
make-css-file-arrays.pl
...with a generalization of make-file-arrays.py.

As a preliminary step, this CL refactors make-file-arrays.py so it's easier
to reuse or modify.

BUG=347863
Committed: https://src.chromium.org/viewvc/blink?view=rev&revision=160549

[Nils Barth (inactive), 2013-10-25 03:11:55]

Hi Adam,
A preliminary refactoring CL; how does it look?

[abarth-chromium, 2013-10-25 04:04:02]

LGTM

https://codereview.chromium.org/43083002/diff/1/Source/web/scripts/make-file-...
File Source/web/scripts/make-file-arrays.py (right):

https://codereview.chromium.org/43083002/diff/1/Source/web/scripts/make-file-...
Source/web/scripts/make-file-arrays.py:88: header_file.write('namespace WebKit
{\n')
It's still hard-coded to use the WebKit namespace.  The other scripts are used
in WebCore, so we'll likely need to parameterize the script.

Also, should we be using jinja2 templates instead of manually building these
strings?

[tkent, 2013-10-25 04:06:06]

https://codereview.chromium.org/43083002/diff/1/Source/web/scripts/make-file-...
File Source/web/scripts/make-file-arrays.py (right):

https://codereview.chromium.org/43083002/diff/1/Source/web/scripts/make-file-...
Source/web/scripts/make-file-arrays.py:56: single_line_comment =
re.compile(r'//.*$', re.MULTILINE)
This pattern will break XMLView.js.

http://src.chromium.org/viewvc/blink/trunk/Source/core/xml/XMLViewer.js?revis...

[abarth-chromium, 2013-10-25 04:09:35]

https://codereview.chromium.org/43083002/diff/1/Source/web/scripts/make-file-...
File Source/web/scripts/make-file-arrays.py (right):

https://codereview.chromium.org/43083002/diff/1/Source/web/scripts/make-file-...
Source/web/scripts/make-file-arrays.py:56: single_line_comment =
re.compile(r'//.*$', re.MULTILINE)
On 2013/10/25 04:06:07, tkent wrote:
> This pattern will break XMLView.js.
> 
>
http://src.chromium.org/viewvc/blink/trunk/Source/core/xml/XMLViewer.js?revis...

That's a good point.  Currently XMLViewer.js goes through xxd rather than this
file.  Maybe we should use an off-the-shelf JS minifier instead of trying to
write our own?

[abarth-chromium, 2013-10-25 04:11:52]

I bet we already have one checked into third_party somewhere.

[Nils Barth (inactive), 2013-10-25 05:03:57]

Thanks for comments; some revisions to address issues.

https://codereview.chromium.org/43083002/diff/1/Source/web/scripts/make-file-...
File Source/web/scripts/make-file-arrays.py (right):

https://codereview.chromium.org/43083002/diff/1/Source/web/scripts/make-file-...
Source/web/scripts/make-file-arrays.py:56: single_line_comment =
re.compile(r'//.*$', re.MULTILINE)
On 2013/10/25 04:06:07, tkent wrote:
> This pattern will break XMLView.js.
> 
>
http://src.chromium.org/viewvc/blink/trunk/Source/core/xml/XMLViewer.js?revis...

Rather, XMLViewer.js.
Good point, thanks!
I've added another pattern to avoid matching '://'.

https://codereview.chromium.org/43083002/diff/1/Source/web/scripts/make-file-...
Source/web/scripts/make-file-arrays.py:56: single_line_comment =
re.compile(r'//.*$', re.MULTILINE)
On 2013/10/25 04:09:35, abarth wrote:
> Maybe we should use an off-the-shelf JS minifier instead of trying to
> write our own?

That would be much better; using regexes for parsing is really
error-prone, with comments being a serious problem.

https://codereview.chromium.org/43083002/diff/1/Source/web/scripts/make-file-...
Source/web/scripts/make-file-arrays.py:88: header_file.write('namespace WebKit
{\n')
On 2013/10/25 04:04:02, abarth wrote:
> It's still hard-coded to use the WebKit namespace.  The other scripts are used
> in WebCore, so we'll likely need to parameterize the script.

Got it, made |namespace| into a parameter.

> Also, should we be using jinja2 templates instead of manually building these
> strings?

If we want to keep generating these files, yes, definitely Jinja2 would be
better!
(I understood you were suggesting we might want to simply replace
these scripts with something else?)

[commit-bot: I haz the power, 2013-10-25 05:04:08]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/nbarth@chromium.org/43083002/90001

[commit-bot: I haz the power, 2013-10-25 06:26:52]

Change committed as 160549

[abarth-chromium, 2013-10-25 15:48:58]

On 2013/10/25 05:03:57, Nils Barth wrote:
> I understood you were suggesting we might want to simply replace
> these scripts with something else?

Chromium has a resource system based on GRD files.  They already know how to do
some amount of minification.  It might be worth investigating whether we should
use that system instead of embedding these resources into the binary.

[Nils Barth (inactive), 2013-10-29 07:28:00]

On 2013/10/25 15:48:58, abarth wrote:
> On 2013/10/25 05:03:57, Nils Barth wrote:
> > I understood you were suggesting we might want to simply replace
> > these scripts with something else?
> 
> Chromium has a resource system based on GRD files.  They already know how to
do
> some amount of minification.  It might be worth investigating whether we
should
> use that system instead of embedding these resources into the binary.

Got it; opened a bug to that effect so we remember it.
(No time to look into currently, but FWIW.)

Replace embedding resources by ad hoc Python with GRD resources	
https://code.google.com/p/chromium/issues/detail?id=312586