OLD | NEW |
| (Empty) |
1 #!/usr/bin/env python | |
2 | |
3 # Copyright (c) 2007, Google Inc. | |
4 # All rights reserved. | |
5 # | |
6 # Redistribution and use in source and binary forms, with or without | |
7 # modification, are permitted provided that the following conditions are | |
8 # met: | |
9 # | |
10 # * Redistributions of source code must retain the above copyright | |
11 # notice, this list of conditions and the following disclaimer. | |
12 # * Redistributions in binary form must reproduce the above | |
13 # copyright notice, this list of conditions and the following disclaimer | |
14 # in the documentation and/or other materials provided with the | |
15 # distribution. | |
16 # * Neither the name of Google Inc. nor the names of its | |
17 # contributors may be used to endorse or promote products derived from | |
18 # this software without specific prior written permission. | |
19 # | |
20 # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | |
21 # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | |
22 # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR | |
23 # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT | |
24 # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, | |
25 # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT | |
26 # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, | |
27 # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY | |
28 # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | |
29 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | |
30 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | |
31 # | |
32 # --- | |
33 # Author: Chad Lester | |
34 # Design and style contributions by: | |
35 # Amit Patel, Bogdan Cocosel, Daniel Dulitz, Eric Tiedemann, | |
36 # Eric Veach, Laurence Gonsalves, Matthew Springer | |
37 # Code reorganized a bit by Craig Silverstein | |
38 | |
39 """This module is used to define and parse command line flags. | |
40 | |
41 This module defines a *distributed* flag-definition policy: rather than | |
42 an application having to define all flags in or near main(), each python | |
43 module defines flags that are useful to it. When one python module | |
44 imports another, it gains access to the other's flags. (This is | |
45 implemented by having all modules share a common, global registry object | |
46 containing all the flag information.) | |
47 | |
48 Flags are defined through the use of one of the DEFINE_xxx functions. | |
49 The specific function used determines how the flag is parsed, checked, | |
50 and optionally type-converted, when it's seen on the command line. | |
51 | |
52 | |
53 IMPLEMENTATION: DEFINE_* creates a 'Flag' object and registers it with a | |
54 'FlagValues' object (typically the global FlagValues FLAGS, defined | |
55 here). The 'FlagValues' object can scan the command line arguments and | |
56 pass flag arguments to the corresponding 'Flag' objects for | |
57 value-checking and type conversion. The converted flag values are | |
58 available as attributes of the 'FlagValues' object. | |
59 | |
60 Code can access the flag through a FlagValues object, for instance | |
61 gflags.FLAGS.myflag. Typically, the __main__ module passes the | |
62 command line arguments to gflags.FLAGS for parsing. | |
63 | |
64 At bottom, this module calls getopt(), so getopt functionality is | |
65 supported, including short- and long-style flags, and the use of -- to | |
66 terminate flags. | |
67 | |
68 Methods defined by the flag module will throw 'FlagsError' exceptions. | |
69 The exception argument will be a human-readable string. | |
70 | |
71 | |
72 FLAG TYPES: This is a list of the DEFINE_*'s that you can do. All flags | |
73 take a name, default value, help-string, and optional 'short' name | |
74 (one-letter name). Some flags have other arguments, which are described | |
75 with the flag. | |
76 | |
77 DEFINE_string: takes any input, and interprets it as a string. | |
78 | |
79 DEFINE_bool or | |
80 DEFINE_boolean: typically does not take an argument: say --myflag to | |
81 set FLAGS.myflag to true, or --nomyflag to set | |
82 FLAGS.myflag to false. Alternately, you can say | |
83 --myflag=true or --myflag=t or --myflag=1 or | |
84 --myflag=false or --myflag=f or --myflag=0 | |
85 | |
86 DEFINE_float: takes an input and interprets it as a floating point | |
87 number. Takes optional args lower_bound and upper_bound; | |
88 if the number specified on the command line is out of | |
89 range, it will raise a FlagError. | |
90 | |
91 DEFINE_integer: takes an input and interprets it as an integer. Takes | |
92 optional args lower_bound and upper_bound as for floats. | |
93 | |
94 DEFINE_enum: takes a list of strings which represents legal values. If | |
95 the command-line value is not in this list, raise a flag | |
96 error. Otherwise, assign to FLAGS.flag as a string. | |
97 | |
98 DEFINE_list: Takes a comma-separated list of strings on the commandline. | |
99 Stores them in a python list object. | |
100 | |
101 DEFINE_spaceseplist: Takes a space-separated list of strings on the | |
102 commandline. Stores them in a python list object. | |
103 Example: --myspacesepflag "foo bar baz" | |
104 | |
105 DEFINE_multistring: The same as DEFINE_string, except the flag can be | |
106 specified more than once on the commandline. The | |
107 result is a python list object (list of strings), | |
108 even if the flag is only on the command line once. | |
109 | |
110 DEFINE_multi_int: The same as DEFINE_integer, except the flag can be | |
111 specified more than once on the commandline. The | |
112 result is a python list object (list of ints), even if | |
113 the flag is only on the command line once. | |
114 | |
115 | |
116 SPECIAL FLAGS: There are a few flags that have special meaning: | |
117 --help prints a list of all the flags in a human-readable fashion | |
118 --helpshort prints a list of all key flags (see below). | |
119 --helpxml prints a list of all flags, in XML format. DO NOT parse | |
120 the output of --help and --helpshort. Instead, parse | |
121 the output of --helpxml. As we add new flags, we may | |
122 add new XML elements. Hence, make sure your parser | |
123 does not crash when it encounters new XML elements. | |
124 --flagfile=foo read flags from foo. | |
125 --undefok=f1,f2 ignore unrecognized option errors for f1,f2. | |
126 For boolean flags, you should use --undefok=boolflag, and | |
127 --boolflag and --noboolflag will be accepted. Do not use | |
128 --undefok=noboolflag. | |
129 -- as in getopt(), terminates flag-processing | |
130 | |
131 | |
132 NOTE ON --flagfile: | |
133 | |
134 Flags may be loaded from text files in addition to being specified on | |
135 the commandline. | |
136 | |
137 Any flags you don't feel like typing, throw them in a file, one flag per | |
138 line, for instance: | |
139 --myflag=myvalue | |
140 --nomyboolean_flag | |
141 You then specify your file with the special flag '--flagfile=somefile'. | |
142 You CAN recursively nest flagfile= tokens OR use multiple files on the | |
143 command line. Lines beginning with a single hash '#' or a double slash | |
144 '//' are comments in your flagfile. | |
145 | |
146 Any flagfile=<file> will be interpreted as having a relative path from | |
147 the current working directory rather than from the place the file was | |
148 included from: | |
149 myPythonScript.py --flagfile=config/somefile.cfg | |
150 | |
151 If somefile.cfg includes further --flagfile= directives, these will be | |
152 referenced relative to the original CWD, not from the directory the | |
153 including flagfile was found in! | |
154 | |
155 The caveat applies to people who are including a series of nested files | |
156 in a different dir than they are executing out of. Relative path names | |
157 are always from CWD, not from the directory of the parent include | |
158 flagfile. We do now support '~' expanded directory names. | |
159 | |
160 Absolute path names ALWAYS work! | |
161 | |
162 | |
163 EXAMPLE USAGE: | |
164 | |
165 import gflags | |
166 FLAGS = gflags.FLAGS | |
167 | |
168 # Flag names are globally defined! So in general, we need to be | |
169 # careful to pick names that are unlikely to be used by other libraries. | |
170 # If there is a conflict, we'll get an error at import time. | |
171 gflags.DEFINE_string('name', 'Mr. President', 'your name') | |
172 gflags.DEFINE_integer('age', None, 'your age in years', lower_bound=0) | |
173 gflags.DEFINE_boolean('debug', False, 'produces debugging output') | |
174 gflags.DEFINE_enum('gender', 'male', ['male', 'female'], 'your gender') | |
175 | |
176 def main(argv): | |
177 try: | |
178 argv = FLAGS(argv) # parse flags | |
179 except gflags.FlagsError, e: | |
180 print '%s\\nUsage: %s ARGS\\n%s' % (e, sys.argv[0], FLAGS) | |
181 sys.exit(1) | |
182 if FLAGS.debug: print 'non-flag arguments:', argv | |
183 print 'Happy Birthday', FLAGS.name | |
184 if FLAGS.age is not None: | |
185 print 'You are a %s, who is %d years old' % (FLAGS.gender, FLAGS.age) | |
186 | |
187 if __name__ == '__main__': | |
188 main(sys.argv) | |
189 | |
190 | |
191 KEY FLAGS: | |
192 | |
193 As we already explained, each module gains access to all flags defined | |
194 by all the other modules it transitively imports. In the case of | |
195 non-trivial scripts, this means a lot of flags ... For documentation | |
196 purposes, it is good to identify the flags that are key (i.e., really | |
197 important) to a module. Clearly, the concept of "key flag" is a | |
198 subjective one. When trying to determine whether a flag is key to a | |
199 module or not, assume that you are trying to explain your module to a | |
200 potential user: which flags would you really like to mention first? | |
201 | |
202 We'll describe shortly how to declare which flags are key to a module. | |
203 For the moment, assume we know the set of key flags for each module. | |
204 Then, if you use the app.py module, you can use the --helpshort flag to | |
205 print only the help for the flags that are key to the main module, in a | |
206 human-readable format. | |
207 | |
208 NOTE: If you need to parse the flag help, do NOT use the output of | |
209 --help / --helpshort. That output is meant for human consumption, and | |
210 may be changed in the future. Instead, use --helpxml; flags that are | |
211 key for the main module are marked there with a <key>yes</key> element. | |
212 | |
213 The set of key flags for a module M is composed of: | |
214 | |
215 1. Flags defined by module M by calling a DEFINE_* function. | |
216 | |
217 2. Flags that module M explictly declares as key by using the function | |
218 | |
219 DECLARE_key_flag(<flag_name>) | |
220 | |
221 3. Key flags of other modules that M specifies by using the function | |
222 | |
223 ADOPT_module_key_flags(<other_module>) | |
224 | |
225 This is a "bulk" declaration of key flags: each flag that is key for | |
226 <other_module> becomes key for the current module too. | |
227 | |
228 Notice that if you do not use the functions described at points 2 and 3 | |
229 above, then --helpshort prints information only about the flags defined | |
230 by the main module of our script. In many cases, this behavior is good | |
231 enough. But if you move part of the main module code (together with the | |
232 related flags) into a different module, then it is nice to use | |
233 DECLARE_key_flag / ADOPT_module_key_flags and make sure --helpshort | |
234 lists all relevant flags (otherwise, your code refactoring may confuse | |
235 your users). | |
236 | |
237 Note: each of DECLARE_key_flag / ADOPT_module_key_flags has its own | |
238 pluses and minuses: DECLARE_key_flag is more targeted and may lead a | |
239 more focused --helpshort documentation. ADOPT_module_key_flags is good | |
240 for cases when an entire module is considered key to the current script. | |
241 Also, it does not require updates to client scripts when a new flag is | |
242 added to the module. | |
243 | |
244 | |
245 EXAMPLE USAGE 2 (WITH KEY FLAGS): | |
246 | |
247 Consider an application that contains the following three files (two | |
248 auxiliary modules and a main module): | |
249 | |
250 File libfoo.py: | |
251 | |
252 import gflags | |
253 | |
254 gflags.DEFINE_integer('num_replicas', 3, 'Number of replicas to start') | |
255 gflags.DEFINE_boolean('rpc2', True, 'Turn on the usage of RPC2.') | |
256 | |
257 ... some code ... | |
258 | |
259 File libbar.py: | |
260 | |
261 import gflags | |
262 | |
263 gflags.DEFINE_string('bar_gfs_path', '/gfs/path', | |
264 'Path to the GFS files for libbar.') | |
265 gflags.DEFINE_string('email_for_bar_errors', 'bar-team@google.com', | |
266 'Email address for bug reports about module libbar.') | |
267 gflags.DEFINE_boolean('bar_risky_hack', False, | |
268 'Turn on an experimental and buggy optimization.') | |
269 | |
270 ... some code ... | |
271 | |
272 File myscript.py: | |
273 | |
274 import gflags | |
275 import libfoo | |
276 import libbar | |
277 | |
278 gflags.DEFINE_integer('num_iterations', 0, 'Number of iterations.') | |
279 | |
280 # Declare that all flags that are key for libfoo are | |
281 # key for this module too. | |
282 gflags.ADOPT_module_key_flags(libfoo) | |
283 | |
284 # Declare that the flag --bar_gfs_path (defined in libbar) is key | |
285 # for this module. | |
286 gflags.DECLARE_key_flag('bar_gfs_path') | |
287 | |
288 ... some code ... | |
289 | |
290 When myscript is invoked with the flag --helpshort, the resulted help | |
291 message lists information about all the key flags for myscript: | |
292 --num_iterations, --num_replicas, --rpc2, and --bar_gfs_path (in | |
293 addition to the special flags --help and --helpshort). | |
294 | |
295 Of course, myscript uses all the flags declared by it (in this case, | |
296 just --num_replicas) or by any of the modules it transitively imports | |
297 (e.g., the modules libfoo, libbar). E.g., it can access the value of | |
298 FLAGS.bar_risky_hack, even if --bar_risky_hack is not declared as a key | |
299 flag for myscript. | |
300 """ | |
301 | |
302 import cgi | |
303 import getopt | |
304 import os | |
305 import re | |
306 import string | |
307 import sys | |
308 | |
309 # Are we running at least python 2.2? | |
310 try: | |
311 if tuple(sys.version_info[:3]) < (2,2,0): | |
312 raise NotImplementedError("requires python 2.2.0 or later") | |
313 except AttributeError: # a very old python, that lacks sys.version_info | |
314 raise NotImplementedError("requires python 2.2.0 or later") | |
315 | |
316 # If we're not running at least python 2.2.1, define True, False, and bool. | |
317 # Thanks, Guido, for the code. | |
318 try: | |
319 True, False, bool | |
320 except NameError: | |
321 False = 0 | |
322 True = 1 | |
323 def bool(x): | |
324 if x: | |
325 return True | |
326 else: | |
327 return False | |
328 | |
329 # Are we running under pychecker? | |
330 _RUNNING_PYCHECKER = 'pychecker.python' in sys.modules | |
331 | |
332 | |
333 def _GetCallingModule(): | |
334 """Returns the name of the module that's calling into this module. | |
335 | |
336 We generally use this function to get the name of the module calling a | |
337 DEFINE_foo... function. | |
338 """ | |
339 # Walk down the stack to find the first globals dict that's not ours. | |
340 for depth in range(1, sys.getrecursionlimit()): | |
341 if not sys._getframe(depth).f_globals is globals(): | |
342 module_name = __GetModuleName(sys._getframe(depth).f_globals) | |
343 if module_name is not None: | |
344 return module_name | |
345 raise AssertionError("No module was found") | |
346 | |
347 | |
348 # module exceptions: | |
349 class FlagsError(Exception): | |
350 """The base class for all flags errors.""" | |
351 pass | |
352 | |
353 | |
354 class DuplicateFlag(FlagsError): | |
355 """Raised if there is a flag naming conflict.""" | |
356 pass | |
357 | |
358 | |
359 # A DuplicateFlagError conveys more information than a | |
360 # DuplicateFlag. Since there are external modules that create | |
361 # DuplicateFlags, the interface to DuplicateFlag shouldn't change. | |
362 class DuplicateFlagError(DuplicateFlag): | |
363 | |
364 def __init__(self, flagname, flag_values): | |
365 self.flagname = flagname | |
366 message = "The flag '%s' is defined twice." % self.flagname | |
367 flags_by_module = flag_values.FlagsByModuleDict() | |
368 for module in flags_by_module: | |
369 for flag in flags_by_module[module]: | |
370 if flag.name == flagname or flag.short_name == flagname: | |
371 message = message + " First from " + module + "," | |
372 break | |
373 message = message + " Second from " + _GetCallingModule() | |
374 DuplicateFlag.__init__(self, message) | |
375 | |
376 | |
377 class IllegalFlagValue(FlagsError): | |
378 """The flag command line argument is illegal.""" | |
379 pass | |
380 | |
381 | |
382 class UnrecognizedFlag(FlagsError): | |
383 """Raised if a flag is unrecognized.""" | |
384 pass | |
385 | |
386 | |
387 # An UnrecognizedFlagError conveys more information than an | |
388 # UnrecognizedFlag. Since there are external modules that create | |
389 # DuplicateFlags, the interface to DuplicateFlag shouldn't change. | |
390 class UnrecognizedFlagError(UnrecognizedFlag): | |
391 def __init__(self, flagname): | |
392 self.flagname = flagname | |
393 UnrecognizedFlag.__init__( | |
394 self, "Unknown command line flag '%s'" % flagname) | |
395 | |
396 | |
397 # Global variable used by expvar | |
398 _exported_flags = {} | |
399 _help_width = 80 # width of help output | |
400 | |
401 | |
402 def GetHelpWidth(): | |
403 """Returns: an integer, the width of help lines that is used in TextWrap.""" | |
404 return _help_width | |
405 | |
406 | |
407 def CutCommonSpacePrefix(text): | |
408 """Removes a common space prefix from the lines of a multiline text. | |
409 | |
410 If the first line does not start with a space, it is left as it is and | |
411 only in the remaining lines a common space prefix is being searched | |
412 for. That means the first line will stay untouched. This is especially | |
413 useful to turn doc strings into help texts. This is because some | |
414 people prefer to have the doc comment start already after the | |
415 apostrophy and then align the following lines while others have the | |
416 apostrophies on a seperately line. | |
417 | |
418 The function also drops trailing empty lines and ignores empty lines | |
419 following the initial content line while calculating the initial | |
420 common whitespace. | |
421 | |
422 Args: | |
423 text: text to work on | |
424 | |
425 Returns: | |
426 the resulting text | |
427 """ | |
428 text_lines = text.splitlines() | |
429 # Drop trailing empty lines | |
430 while text_lines and not text_lines[-1]: | |
431 text_lines = text_lines[:-1] | |
432 if text_lines: | |
433 # We got some content, is the first line starting with a space? | |
434 if text_lines[0] and text_lines[0][0].isspace(): | |
435 text_first_line = [] | |
436 else: | |
437 text_first_line = [text_lines.pop(0)] | |
438 # Calculate length of common leading whitesppace (only over content lines) | |
439 common_prefix = os.path.commonprefix([line for line in text_lines if line]) | |
440 space_prefix_len = len(common_prefix) - len(common_prefix.lstrip()) | |
441 # If we have a common space prefix, drop it from all lines | |
442 if space_prefix_len: | |
443 for index in xrange(len(text_lines)): | |
444 if text_lines[index]: | |
445 text_lines[index] = text_lines[index][space_prefix_len:] | |
446 return '\n'.join(text_first_line + text_lines) | |
447 return '' | |
448 | |
449 | |
450 def TextWrap(text, length=None, indent='', firstline_indent=None, tabs=' '): | |
451 """Wraps a given text to a maximum line length and returns it. | |
452 | |
453 We turn lines that only contain whitespaces into empty lines. We keep | |
454 new lines and tabs (e.g., we do not treat tabs as spaces). | |
455 | |
456 Args: | |
457 text: text to wrap | |
458 length: maximum length of a line, includes indentation | |
459 if this is None then use GetHelpWidth() | |
460 indent: indent for all but first line | |
461 firstline_indent: indent for first line; if None, fall back to indent | |
462 tabs: replacement for tabs | |
463 | |
464 Returns: | |
465 wrapped text | |
466 | |
467 Raises: | |
468 FlagsError: if indent not shorter than length | |
469 FlagsError: if firstline_indent not shorter than length | |
470 """ | |
471 # Get defaults where callee used None | |
472 if length is None: | |
473 length = GetHelpWidth() | |
474 if indent is None: | |
475 indent = '' | |
476 if len(indent) >= length: | |
477 raise FlagsError('Indent must be shorter than length') | |
478 # In line we will be holding the current line which is to be started | |
479 # with indent (or firstline_indent if available) and then appended | |
480 # with words. | |
481 if firstline_indent is None: | |
482 firstline_indent = '' | |
483 line = indent | |
484 else: | |
485 line = firstline_indent | |
486 if len(firstline_indent) >= length: | |
487 raise FlagsError('First iline indent must be shorter than length') | |
488 | |
489 # If the callee does not care about tabs we simply convert them to | |
490 # spaces If callee wanted tabs to be single space then we do that | |
491 # already here. | |
492 if not tabs or tabs == ' ': | |
493 text = text.replace('\t', ' ') | |
494 else: | |
495 tabs_are_whitespace = not tabs.strip() | |
496 | |
497 line_regex = re.compile('([ ]*)(\t*)([^ \t]+)', re.MULTILINE) | |
498 | |
499 # Split the text into lines and the lines with the regex above. The | |
500 # resulting lines are collected in result[]. For each split we get the | |
501 # spaces, the tabs and the next non white space (e.g. next word). | |
502 result = [] | |
503 for text_line in text.splitlines(): | |
504 # Store result length so we can find out whether processing the next | |
505 # line gave any new content | |
506 old_result_len = len(result) | |
507 # Process next line with line_regex. For optimization we do an rstrip(). | |
508 # - process tabs (changes either line or word, see below) | |
509 # - process word (first try to squeeze on line, then wrap or force wrap) | |
510 # Spaces found on the line are ignored, they get added while wrapping as | |
511 # needed. | |
512 for spaces, current_tabs, word in line_regex.findall(text_line.rstrip()): | |
513 # If tabs weren't converted to spaces, handle them now | |
514 if current_tabs: | |
515 # If the last thing we added was a space anyway then drop | |
516 # it. But let's not get rid of the indentation. | |
517 if (((result and line != indent) or | |
518 (not result and line != firstline_indent)) and line[-1] == ' '): | |
519 line = line[:-1] | |
520 # Add the tabs, if that means adding whitespace, just add it at | |
521 # the line, the rstrip() code while shorten the line down if | |
522 # necessary | |
523 if tabs_are_whitespace: | |
524 line += tabs * len(current_tabs) | |
525 else: | |
526 # if not all tab replacement is whitespace we prepend it to the word | |
527 word = tabs * len(current_tabs) + word | |
528 # Handle the case where word cannot be squeezed onto current last line | |
529 if len(line) + len(word) > length and len(indent) + len(word) <= length: | |
530 result.append(line.rstrip()) | |
531 line = indent + word | |
532 word = '' | |
533 # No space left on line or can we append a space? | |
534 if len(line) + 1 >= length: | |
535 result.append(line.rstrip()) | |
536 line = indent | |
537 else: | |
538 line += ' ' | |
539 # Add word and shorten it up to allowed line length. Restart next | |
540 # line with indent and repeat, or add a space if we're done (word | |
541 # finished) This deals with words that caanot fit on one line | |
542 # (e.g. indent + word longer than allowed line length). | |
543 while len(line) + len(word) >= length: | |
544 line += word | |
545 result.append(line[:length]) | |
546 word = line[length:] | |
547 line = indent | |
548 # Default case, simply append the word and a space | |
549 if word: | |
550 line += word + ' ' | |
551 # End of input line. If we have content we finish the line. If the | |
552 # current line is just the indent but we had content in during this | |
553 # original line then we need to add an emoty line. | |
554 if (result and line != indent) or (not result and line != firstline_indent): | |
555 result.append(line.rstrip()) | |
556 elif len(result) == old_result_len: | |
557 result.append('') | |
558 line = indent | |
559 | |
560 return '\n'.join(result) | |
561 | |
562 | |
563 def DocToHelp(doc): | |
564 """Takes a __doc__ string and reformats it as help.""" | |
565 | |
566 # Get rid of starting and ending white space. Using lstrip() or even | |
567 # strip() could drop more than maximum of first line and right space | |
568 # of last line. | |
569 doc = doc.strip() | |
570 | |
571 # Get rid of all empty lines | |
572 whitespace_only_line = re.compile('^[ \t]+$', re.M) | |
573 doc = whitespace_only_line.sub('', doc) | |
574 | |
575 # Cut out common space at line beginnings | |
576 doc = CutCommonSpacePrefix(doc) | |
577 | |
578 # Just like this module's comment, comments tend to be aligned somehow. | |
579 # In other words they all start with the same amount of white space | |
580 # 1) keep double new lines | |
581 # 2) keep ws after new lines if not empty line | |
582 # 3) all other new lines shall be changed to a space | |
583 # Solution: Match new lines between non white space and replace with space. | |
584 doc = re.sub('(?<=\S)\n(?=\S)', ' ', doc, re.M) | |
585 | |
586 return doc | |
587 | |
588 | |
589 def __GetModuleName(globals_dict): | |
590 """Given a globals dict, returns the name of the module that defines it. | |
591 | |
592 Args: | |
593 globals_dict: A dictionary that should correspond to an environment | |
594 providing the values of the globals. | |
595 | |
596 Returns: | |
597 A string (the name of the module) or None (if the module could not | |
598 be identified. | |
599 """ | |
600 for name, module in sys.modules.iteritems(): | |
601 if getattr(module, '__dict__', None) is globals_dict: | |
602 if name == '__main__': | |
603 return sys.argv[0] | |
604 return name | |
605 return None | |
606 | |
607 | |
608 def _GetMainModule(): | |
609 """Returns the name of the module from which execution started.""" | |
610 for depth in range(1, sys.getrecursionlimit()): | |
611 try: | |
612 globals_of_main = sys._getframe(depth).f_globals | |
613 except ValueError: | |
614 return __GetModuleName(globals_of_main) | |
615 raise AssertionError("No module was found") | |
616 | |
617 | |
618 class FlagValues: | |
619 """Registry of 'Flag' objects. | |
620 | |
621 A 'FlagValues' can then scan command line arguments, passing flag | |
622 arguments through to the 'Flag' objects that it owns. It also | |
623 provides easy access to the flag values. Typically only one | |
624 'FlagValues' object is needed by an application: gflags.FLAGS | |
625 | |
626 This class is heavily overloaded: | |
627 | |
628 'Flag' objects are registered via __setitem__: | |
629 FLAGS['longname'] = x # register a new flag | |
630 | |
631 The .value attribute of the registered 'Flag' objects can be accessed | |
632 as attributes of this 'FlagValues' object, through __getattr__. Both | |
633 the long and short name of the original 'Flag' objects can be used to | |
634 access its value: | |
635 FLAGS.longname # parsed flag value | |
636 FLAGS.x # parsed flag value (short name) | |
637 | |
638 Command line arguments are scanned and passed to the registered 'Flag' | |
639 objects through the __call__ method. Unparsed arguments, including | |
640 argv[0] (e.g. the program name) are returned. | |
641 argv = FLAGS(sys.argv) # scan command line arguments | |
642 | |
643 The original registered Flag objects can be retrieved through the use | |
644 of the dictionary-like operator, __getitem__: | |
645 x = FLAGS['longname'] # access the registered Flag object | |
646 | |
647 The str() operator of a 'FlagValues' object provides help for all of | |
648 the registered 'Flag' objects. | |
649 """ | |
650 | |
651 def __init__(self): | |
652 # Since everything in this class is so heavily overloaded, the only | |
653 # way of defining and using fields is to access __dict__ directly. | |
654 | |
655 # Dictionary: flag name (string) -> Flag object. | |
656 self.__dict__['__flags'] = {} | |
657 # Dictionary: module name (string) -> list of Flag objects that are defined | |
658 # by that module. | |
659 self.__dict__['__flags_by_module'] = {} | |
660 # Dictionary: module name (string) -> list of Flag objects that are | |
661 # key for that module. | |
662 self.__dict__['__key_flags_by_module'] = {} | |
663 | |
664 def FlagDict(self): | |
665 return self.__dict__['__flags'] | |
666 | |
667 def FlagsByModuleDict(self): | |
668 """Returns the dictionary of module_name -> list of defined flags. | |
669 | |
670 Returns: | |
671 A dictionary. Its keys are module names (strings). Its values | |
672 are lists of Flag objects. | |
673 """ | |
674 return self.__dict__['__flags_by_module'] | |
675 | |
676 def KeyFlagsByModuleDict(self): | |
677 """Returns the dictionary of module_name -> list of key flags. | |
678 | |
679 Returns: | |
680 A dictionary. Its keys are module names (strings). Its values | |
681 are lists of Flag objects. | |
682 """ | |
683 return self.__dict__['__key_flags_by_module'] | |
684 | |
685 def _RegisterFlagByModule(self, module_name, flag): | |
686 """Records the module that defines a specific flag. | |
687 | |
688 We keep track of which flag is defined by which module so that we | |
689 can later sort the flags by module. | |
690 | |
691 Args: | |
692 module_name: A string, the name of a Python module. | |
693 flag: A Flag object, a flag that is key to the module. | |
694 """ | |
695 flags_by_module = self.FlagsByModuleDict() | |
696 flags_by_module.setdefault(module_name, []).append(flag) | |
697 | |
698 def _RegisterKeyFlagForModule(self, module_name, flag): | |
699 """Specifies that a flag is a key flag for a module. | |
700 | |
701 Args: | |
702 module_name: A string, the name of a Python module. | |
703 flag: A Flag object, a flag that is key to the module. | |
704 """ | |
705 key_flags_by_module = self.KeyFlagsByModuleDict() | |
706 # The list of key flags for the module named module_name. | |
707 key_flags = key_flags_by_module.setdefault(module_name, []) | |
708 # Add flag, but avoid duplicates. | |
709 if flag not in key_flags: | |
710 key_flags.append(flag) | |
711 | |
712 def _GetFlagsDefinedByModule(self, module): | |
713 """Returns the list of flags defined by a module. | |
714 | |
715 Args: | |
716 module: A module object or a module name (a string). | |
717 | |
718 Returns: | |
719 A new list of Flag objects. Caller may update this list as he | |
720 wishes: none of those changes will affect the internals of this | |
721 FlagValue object. | |
722 """ | |
723 if not isinstance(module, str): | |
724 module = module.__name__ | |
725 | |
726 return list(self.FlagsByModuleDict().get(module, [])) | |
727 | |
728 def _GetKeyFlagsForModule(self, module): | |
729 """Returns the list of key flags for a module. | |
730 | |
731 Args: | |
732 module: A module object or a module name (a string) | |
733 | |
734 Returns: | |
735 A new list of Flag objects. Caller may update this list as he | |
736 wishes: none of those changes will affect the internals of this | |
737 FlagValue object. | |
738 """ | |
739 if not isinstance(module, str): | |
740 module = module.__name__ | |
741 | |
742 # Any flag is a key flag for the module that defined it. NOTE: | |
743 # key_flags is a fresh list: we can update it without affecting the | |
744 # internals of this FlagValues object. | |
745 key_flags = self._GetFlagsDefinedByModule(module) | |
746 | |
747 # Take into account flags explicitly declared as key for a module. | |
748 for flag in self.KeyFlagsByModuleDict().get(module, []): | |
749 if flag not in key_flags: | |
750 key_flags.append(flag) | |
751 return key_flags | |
752 | |
753 def AppendFlagValues(self, flag_values): | |
754 """Appends flags registered in another FlagValues instance. | |
755 | |
756 Args: | |
757 flag_values: registry to copy from | |
758 """ | |
759 for flag_name, flag in flag_values.FlagDict().iteritems(): | |
760 # Each flags with shortname appears here twice (once under its | |
761 # normal name, and again with its short name). To prevent | |
762 # problems (DuplicateFlagError) with double flag registration, we | |
763 # perform a check to make sure that the entry we're looking at is | |
764 # for its normal name. | |
765 if flag_name == flag.name: | |
766 self[flag_name] = flag | |
767 | |
768 def __setitem__(self, name, flag): | |
769 """Registers a new flag variable.""" | |
770 fl = self.FlagDict() | |
771 if not isinstance(flag, Flag): | |
772 raise IllegalFlagValue(flag) | |
773 if not isinstance(name, type("")): | |
774 raise FlagsError("Flag name must be a string") | |
775 if len(name) == 0: | |
776 raise FlagsError("Flag name cannot be empty") | |
777 # If running under pychecker, duplicate keys are likely to be | |
778 # defined. Disable check for duplicate keys when pycheck'ing. | |
779 if (fl.has_key(name) and not flag.allow_override and | |
780 not fl[name].allow_override and not _RUNNING_PYCHECKER): | |
781 raise DuplicateFlagError(name, self) | |
782 short_name = flag.short_name | |
783 if short_name is not None: | |
784 if (fl.has_key(short_name) and not flag.allow_override and | |
785 not fl[short_name].allow_override and not _RUNNING_PYCHECKER): | |
786 raise DuplicateFlagError(short_name, self) | |
787 fl[short_name] = flag | |
788 fl[name] = flag | |
789 global _exported_flags | |
790 _exported_flags[name] = flag | |
791 | |
792 def __getitem__(self, name): | |
793 """Retrieves the Flag object for the flag --name.""" | |
794 return self.FlagDict()[name] | |
795 | |
796 def __getattr__(self, name): | |
797 """Retrieves the 'value' attribute of the flag --name.""" | |
798 fl = self.FlagDict() | |
799 if not fl.has_key(name): | |
800 raise AttributeError(name) | |
801 return fl[name].value | |
802 | |
803 def __setattr__(self, name, value): | |
804 """Sets the 'value' attribute of the flag --name.""" | |
805 fl = self.FlagDict() | |
806 fl[name].value = value | |
807 return value | |
808 | |
809 def _FlagIsRegistered(self, flag_obj): | |
810 """Checks whether a Flag object is registered under some name. | |
811 | |
812 Note: this is non trivial: in addition to its normal name, a flag | |
813 may have a short name too. In self.FlagDict(), both the normal and | |
814 the short name are mapped to the same flag object. E.g., calling | |
815 only "del FLAGS.short_name" is not unregistering the corresponding | |
816 Flag object (it is still registered under the longer name). | |
817 | |
818 Args: | |
819 flag_obj: A Flag object. | |
820 | |
821 Returns: | |
822 A boolean: True iff flag_obj is registered under some name. | |
823 """ | |
824 flag_dict = self.FlagDict() | |
825 # Check whether flag_obj is registered under its long name. | |
826 name = flag_obj.name | |
827 if flag_dict.get(name, None) == flag_obj: | |
828 return True | |
829 # Check whether flag_obj is registered under its short name. | |
830 short_name = flag_obj.short_name | |
831 if (short_name is not None and | |
832 flag_dict.get(short_name, None) == flag_obj): | |
833 return True | |
834 # The flag cannot be registered under any other name, so we do not | |
835 # need to do a full search through the values of self.FlagDict(). | |
836 return False | |
837 | |
838 def __delattr__(self, flag_name): | |
839 """Deletes a previously-defined flag from a flag object. | |
840 | |
841 This method makes sure we can delete a flag by using | |
842 | |
843 del flag_values_object.<flag_name> | |
844 | |
845 E.g., | |
846 | |
847 flags.DEFINE_integer('foo', 1, 'Integer flag.') | |
848 del flags.FLAGS.foo | |
849 | |
850 Args: | |
851 flag_name: A string, the name of the flag to be deleted. | |
852 | |
853 Raises: | |
854 AttributeError: When there is no registered flag named flag_name. | |
855 """ | |
856 fl = self.FlagDict() | |
857 if flag_name not in fl: | |
858 raise AttributeError(flag_name) | |
859 | |
860 flag_obj = fl[flag_name] | |
861 del fl[flag_name] | |
862 | |
863 if not self._FlagIsRegistered(flag_obj): | |
864 # If the Flag object indicated by flag_name is no longer | |
865 # registered (please see the docstring of _FlagIsRegistered), then | |
866 # we delete the occurences of the flag object in all our internal | |
867 # dictionaries. | |
868 self.__RemoveFlagFromDictByModule(self.FlagsByModuleDict(), flag_obj) | |
869 self.__RemoveFlagFromDictByModule(self.KeyFlagsByModuleDict(), flag_obj) | |
870 | |
871 def __RemoveFlagFromDictByModule(self, flags_by_module_dict, flag_obj): | |
872 """Removes a flag object from a module -> list of flags dictionary. | |
873 | |
874 Args: | |
875 flags_by_module_dict: A dictionary that maps module names to lists of | |
876 flags. | |
877 flag_obj: A flag object. | |
878 """ | |
879 for unused_module, flags_in_module in flags_by_module_dict.iteritems(): | |
880 # while (as opposed to if) takes care of multiple occurences of a | |
881 # flag in the list for the same module. | |
882 while flag_obj in flags_in_module: | |
883 flags_in_module.remove(flag_obj) | |
884 | |
885 def SetDefault(self, name, value): | |
886 """Changes the default value of the named flag object.""" | |
887 fl = self.FlagDict() | |
888 if not fl.has_key(name): | |
889 raise AttributeError(name) | |
890 fl[name].SetDefault(value) | |
891 | |
892 def __contains__(self, name): | |
893 """Returns True if name is a value (flag) in the dict.""" | |
894 return name in self.FlagDict() | |
895 | |
896 has_key = __contains__ # a synonym for __contains__() | |
897 | |
898 def __iter__(self): | |
899 return self.FlagDict().iterkeys() | |
900 | |
901 def __call__(self, argv): | |
902 """Parses flags from argv; stores parsed flags into this FlagValues object. | |
903 | |
904 All unparsed arguments are returned. Flags are parsed using the GNU | |
905 Program Argument Syntax Conventions, using getopt: | |
906 | |
907 http://www.gnu.org/software/libc/manual/html_mono/libc.html#Getopt | |
908 | |
909 Args: | |
910 argv: argument list. Can be of any type that may be converted to a list. | |
911 | |
912 Returns: | |
913 The list of arguments not parsed as options, including argv[0] | |
914 | |
915 Raises: | |
916 FlagsError: on any parsing error | |
917 """ | |
918 # Support any sequence type that can be converted to a list | |
919 argv = list(argv) | |
920 | |
921 shortopts = "" | |
922 longopts = [] | |
923 | |
924 fl = self.FlagDict() | |
925 | |
926 # This pre parses the argv list for --flagfile=<> options. | |
927 argv = self.ReadFlagsFromFiles(argv) | |
928 | |
929 # Correct the argv to support the google style of passing boolean | |
930 # parameters. Boolean parameters may be passed by using --mybool, | |
931 # --nomybool, --mybool=(true|false|1|0). getopt does not support | |
932 # having options that may or may not have a parameter. We replace | |
933 # instances of the short form --mybool and --nomybool with their | |
934 # full forms: --mybool=(true|false). | |
935 original_argv = list(argv) # list() makes a copy | |
936 shortest_matches = None | |
937 for name, flag in fl.items(): | |
938 if not flag.boolean: | |
939 continue | |
940 if shortest_matches is None: | |
941 # Determine the smallest allowable prefix for all flag names | |
942 shortest_matches = self.ShortestUniquePrefixes(fl) | |
943 no_name = 'no' + name | |
944 prefix = shortest_matches[name] | |
945 no_prefix = shortest_matches[no_name] | |
946 | |
947 # Replace all occurences of this boolean with extended forms | |
948 for arg_idx in range(1, len(argv)): | |
949 arg = argv[arg_idx] | |
950 if arg.find('=') >= 0: continue | |
951 if arg.startswith('--'+prefix) and ('--'+name).startswith(arg): | |
952 argv[arg_idx] = ('--%s=true' % name) | |
953 elif arg.startswith('--'+no_prefix) and ('--'+no_name).startswith(arg): | |
954 argv[arg_idx] = ('--%s=false' % name) | |
955 | |
956 # Loop over all of the flags, building up the lists of short options | |
957 # and long options that will be passed to getopt. Short options are | |
958 # specified as a string of letters, each letter followed by a colon | |
959 # if it takes an argument. Long options are stored in an array of | |
960 # strings. Each string ends with an '=' if it takes an argument. | |
961 for name, flag in fl.items(): | |
962 longopts.append(name + "=") | |
963 if len(name) == 1: # one-letter option: allow short flag type also | |
964 shortopts += name | |
965 if not flag.boolean: | |
966 shortopts += ":" | |
967 | |
968 longopts.append('undefok=') | |
969 undefok_flags = [] | |
970 | |
971 # In case --undefok is specified, loop to pick up unrecognized | |
972 # options one by one. | |
973 unrecognized_opts = [] | |
974 args = argv[1:] | |
975 while True: | |
976 try: | |
977 optlist, unparsed_args = getopt.getopt(args, shortopts, longopts) | |
978 break | |
979 except getopt.GetoptError, e: | |
980 if not e.opt or e.opt in fl: | |
981 # Not an unrecognized option, reraise the exception as a FlagsError | |
982 raise FlagsError(e) | |
983 # Handle an unrecognized option. | |
984 unrecognized_opts.append(e.opt) | |
985 # Remove offender from args and try again | |
986 for arg_index in range(len(args)): | |
987 if ((args[arg_index] == '--' + e.opt) or | |
988 (args[arg_index] == '-' + e.opt) or | |
989 args[arg_index].startswith('--' + e.opt + '=')): | |
990 args = args[0:arg_index] + args[arg_index+1:] | |
991 break | |
992 else: | |
993 # We should have found the option, so we don't expect to get | |
994 # here. We could assert, but raising the original exception | |
995 # might work better. | |
996 raise FlagsError(e) | |
997 | |
998 for name, arg in optlist: | |
999 if name == '--undefok': | |
1000 flag_names = arg.split(',') | |
1001 undefok_flags.extend(flag_names) | |
1002 # For boolean flags, if --undefok=boolflag is specified, then we should | |
1003 # also accept --noboolflag, in addition to --boolflag. | |
1004 # Since we don't know the type of the undefok'd flag, this will affect | |
1005 # non-boolean flags as well. | |
1006 # NOTE: You shouldn't use --undefok=noboolflag, because then we will | |
1007 # accept --nonoboolflag here. We are choosing not to do the conversion | |
1008 # from noboolflag -> boolflag because of the ambiguity that flag names | |
1009 # can start with 'no'. | |
1010 undefok_flags.extend('no' + name for name in flag_names) | |
1011 continue | |
1012 if name.startswith('--'): | |
1013 # long option | |
1014 name = name[2:] | |
1015 short_option = 0 | |
1016 else: | |
1017 # short option | |
1018 name = name[1:] | |
1019 short_option = 1 | |
1020 if fl.has_key(name): | |
1021 flag = fl[name] | |
1022 if flag.boolean and short_option: arg = 1 | |
1023 flag.Parse(arg) | |
1024 | |
1025 # If there were unrecognized options, raise an exception unless | |
1026 # the options were named via --undefok. | |
1027 for opt in unrecognized_opts: | |
1028 if opt not in undefok_flags: | |
1029 raise UnrecognizedFlagError(opt) | |
1030 | |
1031 if unparsed_args: | |
1032 # unparsed_args becomes the first non-flag detected by getopt to | |
1033 # the end of argv. Because argv may have been modified above, | |
1034 # return original_argv for this region. | |
1035 return argv[:1] + original_argv[-len(unparsed_args):] | |
1036 else: | |
1037 return argv[:1] | |
1038 | |
1039 def Reset(self): | |
1040 """Resets the values to the point before FLAGS(argv) was called.""" | |
1041 for f in self.FlagDict().values(): | |
1042 f.Unparse() | |
1043 | |
1044 def RegisteredFlags(self): | |
1045 """Returns: a list of the names and short names of all registered flags.""" | |
1046 return self.FlagDict().keys() | |
1047 | |
1048 def FlagValuesDict(self): | |
1049 """Returns: a dictionary that maps flag names to flag values.""" | |
1050 flag_values = {} | |
1051 | |
1052 for flag_name in self.RegisteredFlags(): | |
1053 flag = self.FlagDict()[flag_name] | |
1054 flag_values[flag_name] = flag.value | |
1055 | |
1056 return flag_values | |
1057 | |
1058 def __str__(self): | |
1059 """Generates a help string for all known flags.""" | |
1060 return self.GetHelp() | |
1061 | |
1062 def GetHelp(self, prefix=''): | |
1063 """Generates a help string for all known flags.""" | |
1064 helplist = [] | |
1065 | |
1066 flags_by_module = self.FlagsByModuleDict() | |
1067 if flags_by_module: | |
1068 | |
1069 modules = flags_by_module.keys() | |
1070 modules.sort() | |
1071 | |
1072 # Print the help for the main module first, if possible. | |
1073 main_module = _GetMainModule() | |
1074 if main_module in modules: | |
1075 modules.remove(main_module) | |
1076 modules = [main_module] + modules | |
1077 | |
1078 for module in modules: | |
1079 self.__RenderOurModuleFlags(module, helplist) | |
1080 | |
1081 self.__RenderModuleFlags('gflags', | |
1082 _SPECIAL_FLAGS.FlagDict().values(), | |
1083 helplist) | |
1084 | |
1085 else: | |
1086 # Just print one long list of flags. | |
1087 self.__RenderFlagList( | |
1088 self.FlagDict().values() + _SPECIAL_FLAGS.FlagDict().values(), | |
1089 helplist, prefix) | |
1090 | |
1091 return '\n'.join(helplist) | |
1092 | |
1093 def __RenderModuleFlags(self, module, flags, output_lines, prefix=""): | |
1094 """Generates a help string for a given module.""" | |
1095 output_lines.append('\n%s%s:' % (prefix, module)) | |
1096 self.__RenderFlagList(flags, output_lines, prefix + " ") | |
1097 | |
1098 def __RenderOurModuleFlags(self, module, output_lines, prefix=""): | |
1099 """Generates a help string for a given module.""" | |
1100 flags = self._GetFlagsDefinedByModule(module) | |
1101 if flags: | |
1102 self.__RenderModuleFlags(module, flags, output_lines, prefix) | |
1103 | |
1104 def __RenderOurModuleKeyFlags(self, module, output_lines, prefix=""): | |
1105 """Generates a help string for the key flags of a given module. | |
1106 | |
1107 Args: | |
1108 module: A module object or a module name (a string). | |
1109 output_lines: A list of strings. The generated help message | |
1110 lines will be appended to this list. | |
1111 prefix: A string that is prepended to each generated help line. | |
1112 """ | |
1113 key_flags = self._GetKeyFlagsForModule(module) | |
1114 if key_flags: | |
1115 self.__RenderModuleFlags(module, key_flags, output_lines, prefix) | |
1116 | |
1117 def MainModuleHelp(self): | |
1118 """Returns: A string describing the key flags of the main module.""" | |
1119 helplist = [] | |
1120 self.__RenderOurModuleKeyFlags(_GetMainModule(), helplist) | |
1121 return '\n'.join(helplist) | |
1122 | |
1123 def __RenderFlagList(self, flaglist, output_lines, prefix=" "): | |
1124 fl = self.FlagDict() | |
1125 special_fl = _SPECIAL_FLAGS.FlagDict() | |
1126 flaglist = [(flag.name, flag) for flag in flaglist] | |
1127 flaglist.sort() | |
1128 flagset = {} | |
1129 for (name, flag) in flaglist: | |
1130 # It's possible this flag got deleted or overridden since being | |
1131 # registered in the per-module flaglist. Check now against the | |
1132 # canonical source of current flag information, the FlagDict. | |
1133 if fl.get(name, None) != flag and special_fl.get(name, None) != flag: | |
1134 # a different flag is using this name now | |
1135 continue | |
1136 # only print help once | |
1137 if flagset.has_key(flag): continue | |
1138 flagset[flag] = 1 | |
1139 flaghelp = "" | |
1140 if flag.short_name: flaghelp += "-%s," % flag.short_name | |
1141 if flag.boolean: | |
1142 flaghelp += "--[no]%s" % flag.name + ":" | |
1143 else: | |
1144 flaghelp += "--%s" % flag.name + ":" | |
1145 flaghelp += " " | |
1146 if flag.help: | |
1147 flaghelp += flag.help | |
1148 flaghelp = TextWrap(flaghelp, indent=prefix+" ", | |
1149 firstline_indent=prefix) | |
1150 if flag.default_as_str: | |
1151 flaghelp += "\n" | |
1152 flaghelp += TextWrap("(default: %s)" % flag.default_as_str, | |
1153 indent=prefix+" ") | |
1154 if flag.parser.syntactic_help: | |
1155 flaghelp += "\n" | |
1156 flaghelp += TextWrap("(%s)" % flag.parser.syntactic_help, | |
1157 indent=prefix+" ") | |
1158 output_lines.append(flaghelp) | |
1159 | |
1160 def get(self, name, default): | |
1161 """Returns the value of a flag (if not None) or a default value. | |
1162 | |
1163 Args: | |
1164 name: A string, the name of a flag. | |
1165 default: Default value to use if the flag value is None. | |
1166 """ | |
1167 | |
1168 value = self.__getattr__(name) | |
1169 if value is not None: # Can't do if not value, b/c value might be '0' or "" | |
1170 return value | |
1171 else: | |
1172 return default | |
1173 | |
1174 def ShortestUniquePrefixes(self, fl): | |
1175 """Returns: dictionary; maps flag names to their shortest unique prefix.""" | |
1176 # Sort the list of flag names | |
1177 sorted_flags = [] | |
1178 for name, flag in fl.items(): | |
1179 sorted_flags.append(name) | |
1180 if flag.boolean: | |
1181 sorted_flags.append('no%s' % name) | |
1182 sorted_flags.sort() | |
1183 | |
1184 # For each name in the sorted list, determine the shortest unique | |
1185 # prefix by comparing itself to the next name and to the previous | |
1186 # name (the latter check uses cached info from the previous loop). | |
1187 shortest_matches = {} | |
1188 prev_idx = 0 | |
1189 for flag_idx in range(len(sorted_flags)): | |
1190 curr = sorted_flags[flag_idx] | |
1191 if flag_idx == (len(sorted_flags) - 1): | |
1192 next = None | |
1193 else: | |
1194 next = sorted_flags[flag_idx+1] | |
1195 next_len = len(next) | |
1196 for curr_idx in range(len(curr)): | |
1197 if (next is None | |
1198 or curr_idx >= next_len | |
1199 or curr[curr_idx] != next[curr_idx]): | |
1200 # curr longer than next or no more chars in common | |
1201 shortest_matches[curr] = curr[:max(prev_idx, curr_idx) + 1] | |
1202 prev_idx = curr_idx | |
1203 break | |
1204 else: | |
1205 # curr shorter than (or equal to) next | |
1206 shortest_matches[curr] = curr | |
1207 prev_idx = curr_idx + 1 # next will need at least one more char | |
1208 return shortest_matches | |
1209 | |
1210 def __IsFlagFileDirective(self, flag_string): | |
1211 """Checks whether flag_string contain a --flagfile=<foo> directive.""" | |
1212 if isinstance(flag_string, type("")): | |
1213 if flag_string.startswith('--flagfile='): | |
1214 return 1 | |
1215 elif flag_string == '--flagfile': | |
1216 return 1 | |
1217 elif flag_string.startswith('-flagfile='): | |
1218 return 1 | |
1219 elif flag_string == '-flagfile': | |
1220 return 1 | |
1221 else: | |
1222 return 0 | |
1223 return 0 | |
1224 | |
1225 def ExtractFilename(self, flagfile_str): | |
1226 """Returns filename from a flagfile_str of form -[-]flagfile=filename. | |
1227 | |
1228 The cases of --flagfile foo and -flagfile foo shouldn't be hitting | |
1229 this function, as they are dealt with in the level above this | |
1230 function. | |
1231 """ | |
1232 if flagfile_str.startswith('--flagfile='): | |
1233 return os.path.expanduser((flagfile_str[(len('--flagfile=')):]).strip()) | |
1234 elif flagfile_str.startswith('-flagfile='): | |
1235 return os.path.expanduser((flagfile_str[(len('-flagfile=')):]).strip()) | |
1236 else: | |
1237 raise FlagsError('Hit illegal --flagfile type: %s' % flagfile_str) | |
1238 | |
1239 def __GetFlagFileLines(self, filename, parsed_file_list): | |
1240 """Returns the useful (!=comments, etc) lines from a file with flags. | |
1241 | |
1242 Args: | |
1243 filename: A string, the name of the flag file. | |
1244 parsed_file_list: A list of the names of the files we have | |
1245 already read. MUTATED BY THIS FUNCTION. | |
1246 | |
1247 Returns: | |
1248 List of strings. See the note below. | |
1249 | |
1250 NOTE(springer): This function checks for a nested --flagfile=<foo> | |
1251 tag and handles the lower file recursively. It returns a list of | |
1252 all the lines that _could_ contain command flags. This is | |
1253 EVERYTHING except whitespace lines and comments (lines starting | |
1254 with '#' or '//'). | |
1255 """ | |
1256 line_list = [] # All line from flagfile. | |
1257 flag_line_list = [] # Subset of lines w/o comments, blanks, flagfile= tags. | |
1258 try: | |
1259 file_obj = open(filename, 'r') | |
1260 except IOError, e_msg: | |
1261 print e_msg | |
1262 print 'ERROR:: Unable to open flagfile: %s' % (filename) | |
1263 return flag_line_list | |
1264 | |
1265 line_list = file_obj.readlines() | |
1266 file_obj.close() | |
1267 parsed_file_list.append(filename) | |
1268 | |
1269 # This is where we check each line in the file we just read. | |
1270 for line in line_list: | |
1271 if line.isspace(): | |
1272 pass | |
1273 # Checks for comment (a line that starts with '#'). | |
1274 elif line.startswith('#') or line.startswith('//'): | |
1275 pass | |
1276 # Checks for a nested "--flagfile=<bar>" flag in the current file. | |
1277 # If we find one, recursively parse down into that file. | |
1278 elif self.__IsFlagFileDirective(line): | |
1279 sub_filename = self.ExtractFilename(line) | |
1280 # We do a little safety check for reparsing a file we've already done. | |
1281 if not sub_filename in parsed_file_list: | |
1282 included_flags = self.__GetFlagFileLines(sub_filename, | |
1283 parsed_file_list) | |
1284 flag_line_list.extend(included_flags) | |
1285 else: # Case of hitting a circularly included file. | |
1286 print >>sys.stderr, ('Warning: Hit circular flagfile dependency: %s' | |
1287 % sub_filename) | |
1288 else: | |
1289 # Any line that's not a comment or a nested flagfile should get | |
1290 # copied into 2nd position. This leaves earlier arguements | |
1291 # further back in the list, thus giving them higher priority. | |
1292 flag_line_list.append(line.strip()) | |
1293 return flag_line_list | |
1294 | |
1295 def ReadFlagsFromFiles(self, argv): | |
1296 """Processes command line args, but also allow args to be read from file. | |
1297 Args: | |
1298 argv: A list of strings, usually sys.argv, which may contain one | |
1299 or more flagfile directives of the form --flagfile="./filename". | |
1300 | |
1301 Returns: | |
1302 | |
1303 A new list which has the original list combined with what we read | |
1304 from any flagfile(s). | |
1305 | |
1306 References: Global gflags.FLAG class instance. | |
1307 | |
1308 This function should be called before the normal FLAGS(argv) call. | |
1309 This function scans the input list for a flag that looks like: | |
1310 --flagfile=<somefile>. Then it opens <somefile>, reads all valid key | |
1311 and value pairs and inserts them into the input list between the | |
1312 first item of the list and any subsequent items in the list. | |
1313 | |
1314 Note that your application's flags are still defined the usual way | |
1315 using gflags DEFINE_flag() type functions. | |
1316 | |
1317 Notes (assuming we're getting a commandline of some sort as our input): | |
1318 --> Flags from the command line argv _should_ always take precedence! | |
1319 --> A further "--flagfile=<otherfile.cfg>" CAN be nested in a flagfile. | |
1320 It will be processed after the parent flag file is done. | |
1321 --> For duplicate flags, first one we hit should "win". | |
1322 --> In a flagfile, a line beginning with # or // is a comment. | |
1323 --> Entirely blank lines _should_ be ignored. | |
1324 """ | |
1325 parsed_file_list = [] | |
1326 rest_of_args = argv | |
1327 new_argv = [] | |
1328 while rest_of_args: | |
1329 current_arg = rest_of_args[0] | |
1330 rest_of_args = rest_of_args[1:] | |
1331 if self.__IsFlagFileDirective(current_arg): | |
1332 # This handles the case of -(-)flagfile foo. In this case the | |
1333 # next arg really is part of this one. | |
1334 if current_arg == '--flagfile' or current_arg == '-flagfile': | |
1335 if not rest_of_args: | |
1336 raise IllegalFlagValue('--flagfile with no argument') | |
1337 flag_filename = os.path.expanduser(rest_of_args[0]) | |
1338 rest_of_args = rest_of_args[1:] | |
1339 else: | |
1340 # This handles the case of (-)-flagfile=foo. | |
1341 flag_filename = self.ExtractFilename(current_arg) | |
1342 new_argv = (new_argv[:1] + | |
1343 self.__GetFlagFileLines(flag_filename, parsed_file_list) + | |
1344 new_argv[1:]) | |
1345 else: | |
1346 new_argv.append(current_arg) | |
1347 | |
1348 return new_argv | |
1349 | |
1350 def FlagsIntoString(self): | |
1351 """Returns a string with the flags assignments from this FlagValues object. | |
1352 | |
1353 This function ignores flags whose value is None. Each flag | |
1354 assignment is separated by a newline. | |
1355 | |
1356 NOTE: MUST mirror the behavior of the C++ function | |
1357 CommandlineFlagsIntoString from google3/base/commandlineflags.cc. | |
1358 """ | |
1359 s = '' | |
1360 for flag in self.FlagDict().values(): | |
1361 if flag.value is not None: | |
1362 s += flag.Serialize() + '\n' | |
1363 return s | |
1364 | |
1365 def AppendFlagsIntoFile(self, filename): | |
1366 """Appends all flags assignments from this FlagInfo object to a file. | |
1367 | |
1368 Output will be in the format of a flagfile. | |
1369 | |
1370 NOTE: MUST mirror the behavior of the C++ version of | |
1371 AppendFlagsIntoFile from google3/base/commandlineflags.cc. | |
1372 """ | |
1373 out_file = open(filename, 'a') | |
1374 out_file.write(self.FlagsIntoString()) | |
1375 out_file.close() | |
1376 | |
1377 def WriteHelpInXMLFormat(self, outfile=None): | |
1378 """Outputs flag documentation in XML format. | |
1379 | |
1380 NOTE: We use element names that are consistent with those used by | |
1381 the C++ command-line flag library, from | |
1382 google3/base/commandlineflags_reporting.cc. We also use a few new | |
1383 elements (e.g., <key>), but we do not interfere / overlap with | |
1384 existing XML elements used by the C++ library. Please maintain this | |
1385 consistency. | |
1386 | |
1387 Args: | |
1388 outfile: File object we write to. Default None means sys.stdout. | |
1389 """ | |
1390 outfile = outfile or sys.stdout | |
1391 | |
1392 outfile.write('<?xml version=\"1.0\"?>\n') | |
1393 outfile.write('<AllFlags>\n') | |
1394 indent = ' ' | |
1395 _WriteSimpleXMLElement(outfile, 'program', os.path.basename(sys.argv[0]), | |
1396 indent) | |
1397 | |
1398 usage_doc = sys.modules['__main__'].__doc__ | |
1399 if not usage_doc: | |
1400 usage_doc = '\nUSAGE: %s [flags]\n' % sys.argv[0] | |
1401 else: | |
1402 usage_doc = usage_doc.replace('%s', sys.argv[0]) | |
1403 _WriteSimpleXMLElement(outfile, 'usage', usage_doc, indent) | |
1404 | |
1405 # Get list of key flags for the main module. | |
1406 key_flags = self._GetKeyFlagsForModule(_GetMainModule()) | |
1407 | |
1408 # Sort flags by declaring module name and next by flag name. | |
1409 flags_by_module = self.FlagsByModuleDict() | |
1410 all_module_names = list(flags_by_module.keys()) | |
1411 all_module_names.sort() | |
1412 for module_name in all_module_names: | |
1413 flag_list = [(f.name, f) for f in flags_by_module[module_name]] | |
1414 flag_list.sort() | |
1415 for unused_flag_name, flag in flag_list: | |
1416 is_key = flag in key_flags | |
1417 flag.WriteInfoInXMLFormat(outfile, module_name, | |
1418 is_key=is_key, indent=indent) | |
1419 | |
1420 outfile.write('</AllFlags>\n') | |
1421 outfile.flush() | |
1422 # end of FlagValues definition | |
1423 | |
1424 | |
1425 # The global FlagValues instance | |
1426 FLAGS = FlagValues() | |
1427 | |
1428 | |
1429 def _MakeXMLSafe(s): | |
1430 """Escapes <, >, and & from s, and removes XML 1.0-illegal chars.""" | |
1431 s = cgi.escape(s) # Escape <, >, and & | |
1432 # Remove characters that cannot appear in an XML 1.0 document | |
1433 # (http://www.w3.org/TR/REC-xml/#charsets). | |
1434 # | |
1435 # NOTE: if there are problems with current solution, one may move to | |
1436 # XML 1.1, which allows such chars, if they're entity-escaped (&#xHH;). | |
1437 s = re.sub(r'[\x00-\x08\x0b\x0c\x0e-\x1f]', '', s) | |
1438 return s | |
1439 | |
1440 | |
1441 def _WriteSimpleXMLElement(outfile, name, value, indent): | |
1442 """Writes a simple XML element. | |
1443 | |
1444 Args: | |
1445 outfile: File object we write the XML element to. | |
1446 name: A string, the name of XML element. | |
1447 value: A Python object, whose string representation will be used | |
1448 as the value of the XML element. | |
1449 indent: A string, prepended to each line of generated output. | |
1450 """ | |
1451 value_str = str(value) | |
1452 if isinstance(value, bool): | |
1453 # Display boolean values as the C++ flag library does: no caps. | |
1454 value_str = value_str.lower() | |
1455 outfile.write('%s<%s>%s</%s>\n' % | |
1456 (indent, name, _MakeXMLSafe(value_str), name)) | |
1457 | |
1458 | |
1459 class Flag: | |
1460 """Information about a command-line flag. | |
1461 | |
1462 'Flag' objects define the following fields: | |
1463 .name - the name for this flag | |
1464 .default - the default value for this flag | |
1465 .default_as_str - default value as repr'd string, e.g., "'true'" (or None) | |
1466 .value - the most recent parsed value of this flag; set by Parse() | |
1467 .help - a help string or None if no help is available | |
1468 .short_name - the single letter alias for this flag (or None) | |
1469 .boolean - if 'true', this flag does not accept arguments | |
1470 .present - true if this flag was parsed from command line flags. | |
1471 .parser - an ArgumentParser object | |
1472 .serializer - an ArgumentSerializer object | |
1473 .allow_override - the flag may be redefined without raising an error | |
1474 | |
1475 The only public method of a 'Flag' object is Parse(), but it is | |
1476 typically only called by a 'FlagValues' object. The Parse() method is | |
1477 a thin wrapper around the 'ArgumentParser' Parse() method. The parsed | |
1478 value is saved in .value, and the .present attribute is updated. If | |
1479 this flag was already present, a FlagsError is raised. | |
1480 | |
1481 Parse() is also called during __init__ to parse the default value and | |
1482 initialize the .value attribute. This enables other python modules to | |
1483 safely use flags even if the __main__ module neglects to parse the | |
1484 command line arguments. The .present attribute is cleared after | |
1485 __init__ parsing. If the default value is set to None, then the | |
1486 __init__ parsing step is skipped and the .value attribute is | |
1487 initialized to None. | |
1488 | |
1489 Note: The default value is also presented to the user in the help | |
1490 string, so it is important that it be a legal value for this flag. | |
1491 """ | |
1492 | |
1493 def __init__(self, parser, serializer, name, default, help_string, | |
1494 short_name=None, boolean=0, allow_override=0): | |
1495 self.name = name | |
1496 | |
1497 if not help_string: | |
1498 help_string = '(no help available)' | |
1499 | |
1500 self.help = help_string | |
1501 self.short_name = short_name | |
1502 self.boolean = boolean | |
1503 self.present = 0 | |
1504 self.parser = parser | |
1505 self.serializer = serializer | |
1506 self.allow_override = allow_override | |
1507 self.value = None | |
1508 | |
1509 self.SetDefault(default) | |
1510 | |
1511 def __GetParsedValueAsString(self, value): | |
1512 if value is None: | |
1513 return None | |
1514 if self.serializer: | |
1515 return repr(self.serializer.Serialize(value)) | |
1516 if self.boolean: | |
1517 if value: | |
1518 return repr('true') | |
1519 else: | |
1520 return repr('false') | |
1521 return repr(str(value)) | |
1522 | |
1523 def Parse(self, argument): | |
1524 try: | |
1525 self.value = self.parser.Parse(argument) | |
1526 except ValueError, e: # recast ValueError as IllegalFlagValue | |
1527 raise IllegalFlagValue("flag --%s: %s" % (self.name, e)) | |
1528 self.present += 1 | |
1529 | |
1530 def Unparse(self): | |
1531 if self.default is None: | |
1532 self.value = None | |
1533 else: | |
1534 self.Parse(self.default) | |
1535 self.present = 0 | |
1536 | |
1537 def Serialize(self): | |
1538 if self.value is None: | |
1539 return '' | |
1540 if self.boolean: | |
1541 if self.value: | |
1542 return "--%s" % self.name | |
1543 else: | |
1544 return "--no%s" % self.name | |
1545 else: | |
1546 if not self.serializer: | |
1547 raise FlagsError("Serializer not present for flag %s" % self.name) | |
1548 return "--%s=%s" % (self.name, self.serializer.Serialize(self.value)) | |
1549 | |
1550 def SetDefault(self, value): | |
1551 """Changes the default value (and current value too) for this Flag.""" | |
1552 # We can't allow a None override because it may end up not being | |
1553 # passed to C++ code when we're overriding C++ flags. So we | |
1554 # cowardly bail out until someone fixes the semantics of trying to | |
1555 # pass None to a C++ flag. See swig_flags.Init() for details on | |
1556 # this behavior. | |
1557 if value is None and self.allow_override: | |
1558 raise DuplicateFlag(self.name) | |
1559 | |
1560 self.default = value | |
1561 self.Unparse() | |
1562 self.default_as_str = self.__GetParsedValueAsString(self.value) | |
1563 | |
1564 def Type(self): | |
1565 """Returns: a string that describes the type of this Flag.""" | |
1566 # NOTE: we use strings, and not the types.*Type constants because | |
1567 # our flags can have more exotic types, e.g., 'comma separated list | |
1568 # of strings', 'whitespace separated list of strings', etc. | |
1569 return self.parser.Type() | |
1570 | |
1571 def WriteInfoInXMLFormat(self, outfile, module_name, is_key=False, indent=''): | |
1572 """Writes common info about this flag, in XML format. | |
1573 | |
1574 This is information that is relevant to all flags (e.g., name, | |
1575 meaning, etc.). If you defined a flag that has some other pieces of | |
1576 info, then please override _WriteCustomInfoInXMLFormat. | |
1577 | |
1578 Please do NOT override this method. | |
1579 | |
1580 Args: | |
1581 outfile: File object we write to. | |
1582 module_name: A string, the name of the module that defines this flag. | |
1583 is_key: A boolean, True iff this flag is key for main module. | |
1584 indent: A string that is prepended to each generated line. | |
1585 """ | |
1586 outfile.write(indent + '<flag>\n') | |
1587 inner_indent = indent + ' ' | |
1588 if is_key: | |
1589 _WriteSimpleXMLElement(outfile, 'key', 'yes', inner_indent) | |
1590 _WriteSimpleXMLElement(outfile, 'file', module_name, inner_indent) | |
1591 # Print flag features that are relevant for all flags. | |
1592 _WriteSimpleXMLElement(outfile, 'name', self.name, inner_indent) | |
1593 if self.short_name: | |
1594 _WriteSimpleXMLElement(outfile, 'short_name', self.short_name, | |
1595 inner_indent) | |
1596 if self.help: | |
1597 _WriteSimpleXMLElement(outfile, 'meaning', self.help, inner_indent) | |
1598 _WriteSimpleXMLElement(outfile, 'default', self.default, inner_indent) | |
1599 _WriteSimpleXMLElement(outfile, 'current', self.value, inner_indent) | |
1600 _WriteSimpleXMLElement(outfile, 'type', self.Type(), inner_indent) | |
1601 # Print extra flag features this flag may have. | |
1602 self._WriteCustomInfoInXMLFormat(outfile, inner_indent) | |
1603 outfile.write(indent + '</flag>\n') | |
1604 | |
1605 def _WriteCustomInfoInXMLFormat(self, outfile, indent): | |
1606 """Writes extra info about this flag, in XML format. | |
1607 | |
1608 "Extra" means "not already printed by WriteInfoInXMLFormat above." | |
1609 | |
1610 Args: | |
1611 outfile: File object we write to. | |
1612 indent: A string that is prepended to each generated line. | |
1613 """ | |
1614 # Usually, the parser knows the extra details about the flag, so | |
1615 # we just forward the call to it. | |
1616 self.parser.WriteCustomInfoInXMLFormat(outfile, indent) | |
1617 # End of Flag definition | |
1618 | |
1619 | |
1620 class ArgumentParser: | |
1621 """Base class used to parse and convert arguments. | |
1622 | |
1623 The Parse() method checks to make sure that the string argument is a | |
1624 legal value and convert it to a native type. If the value cannot be | |
1625 converted, it should throw a 'ValueError' exception with a human | |
1626 readable explanation of why the value is illegal. | |
1627 | |
1628 Subclasses should also define a syntactic_help string which may be | |
1629 presented to the user to describe the form of the legal values. | |
1630 """ | |
1631 syntactic_help = "" | |
1632 | |
1633 def Parse(self, argument): | |
1634 """Default implementation: always returns its argument unmodified.""" | |
1635 return argument | |
1636 | |
1637 def Type(self): | |
1638 return 'string' | |
1639 | |
1640 def WriteCustomInfoInXMLFormat(self, outfile, indent): | |
1641 pass | |
1642 | |
1643 | |
1644 class ArgumentSerializer: | |
1645 """Base class for generating string representations of a flag value.""" | |
1646 | |
1647 def Serialize(self, value): | |
1648 return str(value) | |
1649 | |
1650 | |
1651 class ListSerializer(ArgumentSerializer): | |
1652 | |
1653 def __init__(self, list_sep): | |
1654 self.list_sep = list_sep | |
1655 | |
1656 def Serialize(self, value): | |
1657 return self.list_sep.join([str(x) for x in value]) | |
1658 | |
1659 | |
1660 # The DEFINE functions are explained in mode details in the module doc string. | |
1661 | |
1662 | |
1663 def DEFINE(parser, name, default, help, flag_values=FLAGS, serializer=None, | |
1664 **args): | |
1665 """Registers a generic Flag object. | |
1666 | |
1667 NOTE: in the docstrings of all DEFINE* functions, "registers" is short | |
1668 for "creates a new flag and registers it". | |
1669 | |
1670 Auxiliary function: clients should use the specialized DEFINE_<type> | |
1671 function instead. | |
1672 | |
1673 Args: | |
1674 parser: ArgumentParser that is used to parse the flag arguments. | |
1675 name: A string, the flag name. | |
1676 default: The default value of the flag. | |
1677 help: A help string. | |
1678 flag_values: FlagValues object the flag will be registered with. | |
1679 serializer: ArgumentSerializer that serializes the flag value. | |
1680 args: Dictionary with extra keyword args that are passes to the | |
1681 Flag __init__. | |
1682 """ | |
1683 DEFINE_flag(Flag(parser, serializer, name, default, help, **args), | |
1684 flag_values) | |
1685 | |
1686 | |
1687 def DEFINE_flag(flag, flag_values=FLAGS): | |
1688 """Registers a 'Flag' object with a 'FlagValues' object. | |
1689 | |
1690 By default, the global FLAGS 'FlagValue' object is used. | |
1691 | |
1692 Typical users will use one of the more specialized DEFINE_xxx | |
1693 functions, such as DEFINE_string or DEFINE_integer. But developers | |
1694 who need to create Flag objects themselves should use this function | |
1695 to register their flags. | |
1696 """ | |
1697 # copying the reference to flag_values prevents pychecker warnings | |
1698 fv = flag_values | |
1699 fv[flag.name] = flag | |
1700 # Tell flag_values who's defining the flag. | |
1701 if isinstance(flag_values, FlagValues): | |
1702 # Regarding the above isinstance test: some users pass funny | |
1703 # values of flag_values (e.g., {}) in order to avoid the flag | |
1704 # registration (in the past, there used to be a flag_values == | |
1705 # FLAGS test here) and redefine flags with the same name (e.g., | |
1706 # debug). To avoid breaking their code, we perform the | |
1707 # registration only if flag_values is a real FlagValues object. | |
1708 flag_values._RegisterFlagByModule(_GetCallingModule(), flag) | |
1709 | |
1710 | |
1711 def _InternalDeclareKeyFlags(flag_names, flag_values=FLAGS): | |
1712 """Declares a flag as key for the calling module. | |
1713 | |
1714 Internal function. User code should call DECLARE_key_flag or | |
1715 ADOPT_module_key_flags instead. | |
1716 | |
1717 Args: | |
1718 flag_names: A list of strings that are names of already-registered | |
1719 Flag objects. | |
1720 flag_values: A FlagValues object. This should almost never need | |
1721 to be overridden. | |
1722 | |
1723 Raises: | |
1724 UnrecognizedFlagError: when we refer to a flag that was not | |
1725 defined yet. | |
1726 """ | |
1727 module = _GetCallingModule() | |
1728 | |
1729 for flag_name in flag_names: | |
1730 if flag_name not in flag_values: | |
1731 raise UnrecognizedFlagError(flag_name) | |
1732 flag = flag_values.FlagDict()[flag_name] | |
1733 flag_values._RegisterKeyFlagForModule(module, flag) | |
1734 | |
1735 | |
1736 def DECLARE_key_flag(flag_name, flag_values=FLAGS): | |
1737 """Declares one flag as key to the current module. | |
1738 | |
1739 Key flags are flags that are deemed really important for a module. | |
1740 They are important when listing help messages; e.g., if the | |
1741 --helpshort command-line flag is used, then only the key flags of the | |
1742 main module are listed (instead of all flags, as in the case of | |
1743 --help). | |
1744 | |
1745 Sample usage: | |
1746 | |
1747 flags.DECLARED_key_flag('flag_1') | |
1748 | |
1749 Args: | |
1750 flag_name: A string, the name of an already declared flag. | |
1751 (Redeclaring flags as key, including flags implicitly key | |
1752 because they were declared in this module, is a no-op.) | |
1753 flag_values: A FlagValues object. This should almost never | |
1754 need to be overridden. | |
1755 """ | |
1756 _InternalDeclareKeyFlags([flag_name], flag_values=flag_values) | |
1757 | |
1758 | |
1759 def ADOPT_module_key_flags(module, flag_values=FLAGS): | |
1760 """Declares that all flags key to a module are key to the current module. | |
1761 | |
1762 Args: | |
1763 module: A module object. | |
1764 flag_values: A FlagValues object. This should almost never need | |
1765 to be overridden. | |
1766 | |
1767 Raises: | |
1768 FlagsError: When given an argument that is a module name (a | |
1769 string), instead of a module object. | |
1770 """ | |
1771 # NOTE(salcianu): an even better test would be if not | |
1772 # isinstance(module, types.ModuleType) but I didn't want to import | |
1773 # types for such a tiny use. | |
1774 if isinstance(module, str): | |
1775 raise FlagsError('Received module name %s; expected a module object.' | |
1776 % module) | |
1777 _InternalDeclareKeyFlags( | |
1778 [f.name for f in flag_values._GetKeyFlagsForModule(module.__name__)], | |
1779 flag_values=flag_values) | |
1780 | |
1781 | |
1782 # | |
1783 # STRING FLAGS | |
1784 # | |
1785 | |
1786 | |
1787 def DEFINE_string(name, default, help, flag_values=FLAGS, **args): | |
1788 """Registers a flag whose value can be any string.""" | |
1789 parser = ArgumentParser() | |
1790 serializer = ArgumentSerializer() | |
1791 DEFINE(parser, name, default, help, flag_values, serializer, **args) | |
1792 | |
1793 | |
1794 # | |
1795 # BOOLEAN FLAGS | |
1796 # | |
1797 # and the special HELP flags. | |
1798 | |
1799 class BooleanParser(ArgumentParser): | |
1800 """Parser of boolean values.""" | |
1801 | |
1802 def Convert(self, argument): | |
1803 """Converts the argument to a boolean; raise ValueError on errors.""" | |
1804 if type(argument) == str: | |
1805 if argument.lower() in ['true', 't', '1']: | |
1806 return True | |
1807 elif argument.lower() in ['false', 'f', '0']: | |
1808 return False | |
1809 | |
1810 bool_argument = bool(argument) | |
1811 if argument == bool_argument: | |
1812 # The argument is a valid boolean (True, False, 0, or 1), and not just | |
1813 # something that always converts to bool (list, string, int, etc.). | |
1814 return bool_argument | |
1815 | |
1816 raise ValueError('Non-boolean argument to boolean flag', argument) | |
1817 | |
1818 def Parse(self, argument): | |
1819 val = self.Convert(argument) | |
1820 return val | |
1821 | |
1822 def Type(self): | |
1823 return 'bool' | |
1824 | |
1825 | |
1826 class BooleanFlag(Flag): | |
1827 """Basic boolean flag. | |
1828 | |
1829 Boolean flags do not take any arguments, and their value is either | |
1830 True (1) or False (0). The false value is specified on the command | |
1831 line by prepending the word 'no' to either the long or the short flag | |
1832 name. | |
1833 | |
1834 For example, if a Boolean flag was created whose long name was | |
1835 'update' and whose short name was 'x', then this flag could be | |
1836 explicitly unset through either --noupdate or --nox. | |
1837 """ | |
1838 | |
1839 def __init__(self, name, default, help, short_name=None, **args): | |
1840 p = BooleanParser() | |
1841 Flag.__init__(self, p, None, name, default, help, short_name, 1, **args) | |
1842 if not self.help: self.help = "a boolean value" | |
1843 | |
1844 | |
1845 def DEFINE_boolean(name, default, help, flag_values=FLAGS, **args): | |
1846 """Registers a boolean flag. | |
1847 | |
1848 Such a boolean flag does not take an argument. If a user wants to | |
1849 specify a false value explicitly, the long option beginning with 'no' | |
1850 must be used: i.e. --noflag | |
1851 | |
1852 This flag will have a value of None, True or False. None is possible | |
1853 if default=None and the user does not specify the flag on the command | |
1854 line. | |
1855 """ | |
1856 DEFINE_flag(BooleanFlag(name, default, help, **args), flag_values) | |
1857 | |
1858 # Match C++ API to unconfuse C++ people. | |
1859 DEFINE_bool = DEFINE_boolean | |
1860 | |
1861 class HelpFlag(BooleanFlag): | |
1862 """ | |
1863 HelpFlag is a special boolean flag that prints usage information and | |
1864 raises a SystemExit exception if it is ever found in the command | |
1865 line arguments. Note this is called with allow_override=1, so other | |
1866 apps can define their own --help flag, replacing this one, if they want. | |
1867 """ | |
1868 def __init__(self): | |
1869 BooleanFlag.__init__(self, "help", 0, "show this help", | |
1870 short_name="?", allow_override=1) | |
1871 def Parse(self, arg): | |
1872 if arg: | |
1873 doc = sys.modules["__main__"].__doc__ | |
1874 flags = str(FLAGS) | |
1875 print doc or ("\nUSAGE: %s [flags]\n" % sys.argv[0]) | |
1876 if flags: | |
1877 print "flags:" | |
1878 print flags | |
1879 sys.exit(1) | |
1880 | |
1881 | |
1882 class HelpXMLFlag(BooleanFlag): | |
1883 """Similar to HelpFlag, but generates output in XML format.""" | |
1884 | |
1885 def __init__(self): | |
1886 BooleanFlag.__init__(self, 'helpxml', False, | |
1887 'like --help, but generates XML output', | |
1888 allow_override=1) | |
1889 | |
1890 def Parse(self, arg): | |
1891 if arg: | |
1892 FLAGS.WriteHelpInXMLFormat(sys.stdout) | |
1893 sys.exit(1) | |
1894 | |
1895 | |
1896 class HelpshortFlag(BooleanFlag): | |
1897 """ | |
1898 HelpshortFlag is a special boolean flag that prints usage | |
1899 information for the "main" module, and rasies a SystemExit exception | |
1900 if it is ever found in the command line arguments. Note this is | |
1901 called with allow_override=1, so other apps can define their own | |
1902 --helpshort flag, replacing this one, if they want. | |
1903 """ | |
1904 def __init__(self): | |
1905 BooleanFlag.__init__(self, "helpshort", 0, | |
1906 "show usage only for this module", allow_override=1) | |
1907 def Parse(self, arg): | |
1908 if arg: | |
1909 doc = sys.modules["__main__"].__doc__ | |
1910 flags = FLAGS.MainModuleHelp() | |
1911 print doc or ("\nUSAGE: %s [flags]\n" % sys.argv[0]) | |
1912 if flags: | |
1913 print "flags:" | |
1914 print flags | |
1915 sys.exit(1) | |
1916 | |
1917 | |
1918 # | |
1919 # FLOAT FLAGS | |
1920 # | |
1921 | |
1922 class FloatParser(ArgumentParser): | |
1923 """Parser of floating point values. | |
1924 | |
1925 Parsed value may be bounded to a given upper and lower bound. | |
1926 """ | |
1927 number_article = "a" | |
1928 number_name = "number" | |
1929 syntactic_help = " ".join((number_article, number_name)) | |
1930 | |
1931 def __init__(self, lower_bound=None, upper_bound=None): | |
1932 self.lower_bound = lower_bound | |
1933 self.upper_bound = upper_bound | |
1934 sh = self.syntactic_help | |
1935 if lower_bound != None and upper_bound != None: | |
1936 sh = ("%s in the range [%s, %s]" % (sh, lower_bound, upper_bound)) | |
1937 elif lower_bound == 1: | |
1938 sh = "a positive %s" % self.number_name | |
1939 elif upper_bound == -1: | |
1940 sh = "a negative %s" % self.number_name | |
1941 elif lower_bound == 0: | |
1942 sh = "a non-negative %s" % self.number_name | |
1943 elif upper_bound != None: | |
1944 sh = "%s <= %s" % (self.number_name, upper_bound) | |
1945 elif lower_bound != None: | |
1946 sh = "%s >= %s" % (self.number_name, lower_bound) | |
1947 self.syntactic_help = sh | |
1948 | |
1949 def Convert(self, argument): | |
1950 """Converts argument to a float; raises ValueError on errors.""" | |
1951 return float(argument) | |
1952 | |
1953 def Parse(self, argument): | |
1954 val = self.Convert(argument) | |
1955 if ((self.lower_bound != None and val < self.lower_bound) or | |
1956 (self.upper_bound != None and val > self.upper_bound)): | |
1957 raise ValueError("%s is not %s" % (val, self.syntactic_help)) | |
1958 return val | |
1959 | |
1960 def Type(self): | |
1961 return 'float' | |
1962 | |
1963 def WriteCustomInfoInXMLFormat(self, outfile, indent): | |
1964 if self.lower_bound is not None: | |
1965 _WriteSimpleXMLElement(outfile, 'lower_bound', self.lower_bound, indent) | |
1966 if self.upper_bound is not None: | |
1967 _WriteSimpleXMLElement(outfile, 'upper_bound', self.upper_bound, indent) | |
1968 # End of FloatParser | |
1969 | |
1970 | |
1971 def DEFINE_float(name, default, help, lower_bound=None, upper_bound=None, | |
1972 flag_values=FLAGS, **args): | |
1973 """Registers a flag whose value must be a float. | |
1974 | |
1975 If lower_bound or upper_bound are set, then this flag must be | |
1976 within the given range. | |
1977 """ | |
1978 parser = FloatParser(lower_bound, upper_bound) | |
1979 serializer = ArgumentSerializer() | |
1980 DEFINE(parser, name, default, help, flag_values, serializer, **args) | |
1981 | |
1982 | |
1983 # | |
1984 # INTEGER FLAGS | |
1985 # | |
1986 | |
1987 | |
1988 class IntegerParser(FloatParser): | |
1989 """Parser of an integer value. | |
1990 | |
1991 Parsed value may be bounded to a given upper and lower bound. | |
1992 """ | |
1993 number_article = "an" | |
1994 number_name = "integer" | |
1995 syntactic_help = " ".join((number_article, number_name)) | |
1996 | |
1997 def Convert(self, argument): | |
1998 __pychecker__ = 'no-returnvalues' | |
1999 if type(argument) == str: | |
2000 base = 10 | |
2001 if len(argument) > 2 and argument[0] == "0" and argument[1] == "x": | |
2002 base = 16 | |
2003 try: | |
2004 return int(argument, base) | |
2005 # ValueError is thrown when argument is a string, and overflows an int. | |
2006 except ValueError: | |
2007 return long(argument, base) | |
2008 else: | |
2009 try: | |
2010 return int(argument) | |
2011 # OverflowError is thrown when argument is numeric, and overflows an int. | |
2012 except OverflowError: | |
2013 return long(argument) | |
2014 | |
2015 def Type(self): | |
2016 return 'int' | |
2017 | |
2018 | |
2019 def DEFINE_integer(name, default, help, lower_bound=None, upper_bound=None, | |
2020 flag_values=FLAGS, **args): | |
2021 """Registers a flag whose value must be an integer. | |
2022 | |
2023 If lower_bound, or upper_bound are set, then this flag must be | |
2024 within the given range. | |
2025 """ | |
2026 parser = IntegerParser(lower_bound, upper_bound) | |
2027 serializer = ArgumentSerializer() | |
2028 DEFINE(parser, name, default, help, flag_values, serializer, **args) | |
2029 | |
2030 | |
2031 # | |
2032 # ENUM FLAGS | |
2033 # | |
2034 | |
2035 | |
2036 class EnumParser(ArgumentParser): | |
2037 """Parser of a string enum value (a string value from a given set). | |
2038 | |
2039 If enum_values (see below) is not specified, any string is allowed. | |
2040 """ | |
2041 | |
2042 def __init__(self, enum_values=None): | |
2043 self.enum_values = enum_values | |
2044 | |
2045 def Parse(self, argument): | |
2046 if self.enum_values and argument not in self.enum_values: | |
2047 raise ValueError("value should be one of <%s>" % | |
2048 "|".join(self.enum_values)) | |
2049 return argument | |
2050 | |
2051 def Type(self): | |
2052 return 'string enum' | |
2053 | |
2054 | |
2055 class EnumFlag(Flag): | |
2056 """Basic enum flag; its value can be any string from list of enum_values.""" | |
2057 | |
2058 def __init__(self, name, default, help, enum_values=None, | |
2059 short_name=None, **args): | |
2060 enum_values = enum_values or [] | |
2061 p = EnumParser(enum_values) | |
2062 g = ArgumentSerializer() | |
2063 Flag.__init__(self, p, g, name, default, help, short_name, **args) | |
2064 if not self.help: self.help = "an enum string" | |
2065 self.help = "<%s>: %s" % ("|".join(enum_values), self.help) | |
2066 | |
2067 def _WriteCustomInfoInXMLFormat(self, outfile, indent): | |
2068 for enum_value in self.parser.enum_values: | |
2069 _WriteSimpleXMLElement(outfile, 'enum_value', enum_value, indent) | |
2070 | |
2071 | |
2072 def DEFINE_enum(name, default, enum_values, help, flag_values=FLAGS, | |
2073 **args): | |
2074 """Registers a flag whose value can be any string from enum_values.""" | |
2075 DEFINE_flag(EnumFlag(name, default, help, enum_values, ** args), | |
2076 flag_values) | |
2077 | |
2078 | |
2079 # | |
2080 # LIST FLAGS | |
2081 # | |
2082 | |
2083 | |
2084 class BaseListParser(ArgumentParser): | |
2085 """Base class for a parser of lists of strings. | |
2086 | |
2087 To extend, inherit from this class; from the subclass __init__, call | |
2088 | |
2089 BaseListParser.__init__(self, token, name) | |
2090 | |
2091 where token is a character used to tokenize, and name is a description | |
2092 of the separator. | |
2093 """ | |
2094 | |
2095 def __init__(self, token=None, name=None): | |
2096 assert name | |
2097 self._token = token | |
2098 self._name = name | |
2099 self.syntactic_help = "a %s separated list" % self._name | |
2100 | |
2101 def Parse(self, argument): | |
2102 if argument == '': | |
2103 return [] | |
2104 else: | |
2105 return [s.strip() for s in argument.split(self._token)] | |
2106 | |
2107 def Type(self): | |
2108 return '%s separated list of strings' % self._name | |
2109 | |
2110 | |
2111 class ListParser(BaseListParser): | |
2112 """Parser for a comma-separated list of strings.""" | |
2113 | |
2114 def __init__(self): | |
2115 BaseListParser.__init__(self, ',', 'comma') | |
2116 | |
2117 def WriteCustomInfoInXMLFormat(self, outfile, indent): | |
2118 BaseListParser.WriteCustomInfoInXMLFormat(self, outfile, indent) | |
2119 _WriteSimpleXMLElement(outfile, 'list_separator', repr(','), indent) | |
2120 | |
2121 | |
2122 class WhitespaceSeparatedListParser(BaseListParser): | |
2123 """Parser for a whitespace-separated list of strings.""" | |
2124 | |
2125 def __init__(self): | |
2126 BaseListParser.__init__(self, None, 'whitespace') | |
2127 | |
2128 def WriteCustomInfoInXMLFormat(self, outfile, indent): | |
2129 BaseListParser.WriteCustomInfoInXMLFormat(self, outfile, indent) | |
2130 separators = list(string.whitespace) | |
2131 separators.sort() | |
2132 for ws_char in string.whitespace: | |
2133 _WriteSimpleXMLElement(outfile, 'list_separator', repr(ws_char), indent) | |
2134 | |
2135 | |
2136 def DEFINE_list(name, default, help, flag_values=FLAGS, **args): | |
2137 """Registers a flag whose value is a comma-separated list of strings.""" | |
2138 parser = ListParser() | |
2139 serializer = ListSerializer(',') | |
2140 DEFINE(parser, name, default, help, flag_values, serializer, **args) | |
2141 | |
2142 | |
2143 def DEFINE_spaceseplist(name, default, help, flag_values=FLAGS, **args): | |
2144 """Registers a flag whose value is a whitespace-separated list of strings. | |
2145 | |
2146 Any whitespace can be used as a separator. | |
2147 """ | |
2148 parser = WhitespaceSeparatedListParser() | |
2149 serializer = ListSerializer(' ') | |
2150 DEFINE(parser, name, default, help, flag_values, serializer, **args) | |
2151 | |
2152 | |
2153 # | |
2154 # MULTI FLAGS | |
2155 # | |
2156 | |
2157 | |
2158 class MultiFlag(Flag): | |
2159 """A flag that can appear multiple time on the command-line. | |
2160 | |
2161 The value of such a flag is a list that contains the individual values | |
2162 from all the appearances of that flag on the command-line. | |
2163 | |
2164 See the __doc__ for Flag for most behavior of this class. Only | |
2165 differences in behavior are described here: | |
2166 | |
2167 * The default value may be either a single value or a list of values. | |
2168 A single value is interpreted as the [value] singleton list. | |
2169 | |
2170 * The value of the flag is always a list, even if the option was | |
2171 only supplied once, and even if the default value is a single | |
2172 value | |
2173 """ | |
2174 | |
2175 def __init__(self, *args, **kwargs): | |
2176 Flag.__init__(self, *args, **kwargs) | |
2177 self.help += ';\n repeat this option to specify a list of values' | |
2178 | |
2179 def Parse(self, arguments): | |
2180 """Parses one or more arguments with the installed parser. | |
2181 | |
2182 Args: | |
2183 arguments: a single argument or a list of arguments (typically a | |
2184 list of default values); a single argument is converted | |
2185 internally into a list containing one item. | |
2186 """ | |
2187 if not isinstance(arguments, list): | |
2188 # Default value may be a list of values. Most other arguments | |
2189 # will not be, so convert them into a single-item list to make | |
2190 # processing simpler below. | |
2191 arguments = [arguments] | |
2192 | |
2193 if self.present: | |
2194 # keep a backup reference to list of previously supplied option values | |
2195 values = self.value | |
2196 else: | |
2197 # "erase" the defaults with an empty list | |
2198 values = [] | |
2199 | |
2200 for item in arguments: | |
2201 # have Flag superclass parse argument, overwriting self.value reference | |
2202 Flag.Parse(self, item) # also increments self.present | |
2203 values.append(self.value) | |
2204 | |
2205 # put list of option values back in the 'value' attribute | |
2206 self.value = values | |
2207 | |
2208 def Serialize(self): | |
2209 if not self.serializer: | |
2210 raise FlagsError("Serializer not present for flag %s" % self.name) | |
2211 if self.value is None: | |
2212 return '' | |
2213 | |
2214 s = '' | |
2215 | |
2216 multi_value = self.value | |
2217 | |
2218 for self.value in multi_value: | |
2219 if s: s += ' ' | |
2220 s += Flag.Serialize(self) | |
2221 | |
2222 self.value = multi_value | |
2223 | |
2224 return s | |
2225 | |
2226 def Type(self): | |
2227 return 'multi ' + self.parser.Type() | |
2228 | |
2229 | |
2230 def DEFINE_multi(parser, serializer, name, default, help, flag_values=FLAGS, | |
2231 **args): | |
2232 """Registers a generic MultiFlag that parses its args with a given parser. | |
2233 | |
2234 Auxiliary function. Normal users should NOT use it directly. | |
2235 | |
2236 Developers who need to create their own 'Parser' classes for options | |
2237 which can appear multiple times can call this module function to | |
2238 register their flags. | |
2239 """ | |
2240 DEFINE_flag(MultiFlag(parser, serializer, name, default, help, **args), | |
2241 flag_values) | |
2242 | |
2243 | |
2244 def DEFINE_multistring(name, default, help, flag_values=FLAGS, **args): | |
2245 """Registers a flag whose value can be a list of any strings. | |
2246 | |
2247 Use the flag on the command line multiple times to place multiple | |
2248 string values into the list. The 'default' may be a single string | |
2249 (which will be converted into a single-element list) or a list of | |
2250 strings. | |
2251 """ | |
2252 parser = ArgumentParser() | |
2253 serializer = ArgumentSerializer() | |
2254 DEFINE_multi(parser, serializer, name, default, help, flag_values, **args) | |
2255 | |
2256 | |
2257 def DEFINE_multi_int(name, default, help, lower_bound=None, upper_bound=None, | |
2258 flag_values=FLAGS, **args): | |
2259 """Registers a flag whose value can be a list of arbitrary integers. | |
2260 | |
2261 Use the flag on the command line multiple times to place multiple | |
2262 integer values into the list. The 'default' may be a single integer | |
2263 (which will be converted into a single-element list) or a list of | |
2264 integers. | |
2265 """ | |
2266 parser = IntegerParser(lower_bound, upper_bound) | |
2267 serializer = ArgumentSerializer() | |
2268 DEFINE_multi(parser, serializer, name, default, help, flag_values, **args) | |
2269 | |
2270 | |
2271 # Now register the flags that we want to exist in all applications. | |
2272 # These are all defined with allow_override=1, so user-apps can use | |
2273 # these flagnames for their own purposes, if they want. | |
2274 DEFINE_flag(HelpFlag()) | |
2275 DEFINE_flag(HelpshortFlag()) | |
2276 DEFINE_flag(HelpXMLFlag()) | |
2277 | |
2278 # Define special flags here so that help may be generated for them. | |
2279 _SPECIAL_FLAGS = FlagValues() | |
2280 | |
2281 | |
2282 DEFINE_string( | |
2283 'flagfile', "", | |
2284 "Insert flag definitions from the given file into the command line.", | |
2285 _SPECIAL_FLAGS) | |
2286 | |
2287 DEFINE_string( | |
2288 'undefok', "", | |
2289 "comma-separated list of flag names that it is okay to specify " | |
2290 "on the command line even if the program does not define a flag " | |
2291 "with that name. IMPORTANT: flags in this list that have " | |
2292 "arguments MUST use the --flag=value format.", _SPECIAL_FLAGS) | |
OLD | NEW |