OLD | NEW |
| (Empty) |
1 #!/usr/bin/env python | |
2 # | |
3 # Copyright (c) 2002, Google Inc. | |
4 # All rights reserved. | |
5 # | |
6 # Redistribution and use in source and binary forms, with or without | |
7 # modification, are permitted provided that the following conditions are | |
8 # met: | |
9 # | |
10 # * Redistributions of source code must retain the above copyright | |
11 # notice, this list of conditions and the following disclaimer. | |
12 # * Redistributions in binary form must reproduce the above | |
13 # copyright notice, this list of conditions and the following disclaimer | |
14 # in the documentation and/or other materials provided with the | |
15 # distribution. | |
16 # * Neither the name of Google Inc. nor the names of its | |
17 # contributors may be used to endorse or promote products derived from | |
18 # this software without specific prior written permission. | |
19 # | |
20 # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | |
21 # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | |
22 # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR | |
23 # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT | |
24 # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, | |
25 # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT | |
26 # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, | |
27 # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY | |
28 # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | |
29 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | |
30 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | |
31 # | |
32 # --- | |
33 # Author: Chad Lester | |
34 # Design and style contributions by: | |
35 # Amit Patel, Bogdan Cocosel, Daniel Dulitz, Eric Tiedemann, | |
36 # Eric Veach, Laurence Gonsalves, Matthew Springer | |
37 # Code reorganized a bit by Craig Silverstein | |
38 | |
39 """This module is used to define and parse command line flags. | |
40 | |
41 This module defines a *distributed* flag-definition policy: rather than | |
42 an application having to define all flags in or near main(), each python | |
43 module defines flags that are useful to it. When one python module | |
44 imports another, it gains access to the other's flags. (This is | |
45 implemented by having all modules share a common, global registry object | |
46 containing all the flag information.) | |
47 | |
48 Flags are defined through the use of one of the DEFINE_xxx functions. | |
49 The specific function used determines how the flag is parsed, checked, | |
50 and optionally type-converted, when it's seen on the command line. | |
51 | |
52 | |
53 IMPLEMENTATION: DEFINE_* creates a 'Flag' object and registers it with a | |
54 'FlagValues' object (typically the global FlagValues FLAGS, defined | |
55 here). The 'FlagValues' object can scan the command line arguments and | |
56 pass flag arguments to the corresponding 'Flag' objects for | |
57 value-checking and type conversion. The converted flag values are | |
58 available as attributes of the 'FlagValues' object. | |
59 | |
60 Code can access the flag through a FlagValues object, for instance | |
61 gflags.FLAGS.myflag. Typically, the __main__ module passes the command | |
62 line arguments to gflags.FLAGS for parsing. | |
63 | |
64 At bottom, this module calls getopt(), so getopt functionality is | |
65 supported, including short- and long-style flags, and the use of -- to | |
66 terminate flags. | |
67 | |
68 Methods defined by the flag module will throw 'FlagsError' exceptions. | |
69 The exception argument will be a human-readable string. | |
70 | |
71 | |
72 FLAG TYPES: This is a list of the DEFINE_*'s that you can do. All flags | |
73 take a name, default value, help-string, and optional 'short' name | |
74 (one-letter name). Some flags have other arguments, which are described | |
75 with the flag. | |
76 | |
77 DEFINE_string: takes any input, and interprets it as a string. | |
78 | |
79 DEFINE_bool or | |
80 DEFINE_boolean: typically does not take an argument: say --myflag to | |
81 set FLAGS.myflag to true, or --nomyflag to set | |
82 FLAGS.myflag to false. Alternately, you can say | |
83 --myflag=true or --myflag=t or --myflag=1 or | |
84 --myflag=false or --myflag=f or --myflag=0 | |
85 | |
86 DEFINE_float: takes an input and interprets it as a floating point | |
87 number. Takes optional args lower_bound and upper_bound; | |
88 if the number specified on the command line is out of | |
89 range, it will raise a FlagError. | |
90 | |
91 DEFINE_integer: takes an input and interprets it as an integer. Takes | |
92 optional args lower_bound and upper_bound as for floats. | |
93 | |
94 DEFINE_enum: takes a list of strings which represents legal values. If | |
95 the command-line value is not in this list, raise a flag | |
96 error. Otherwise, assign to FLAGS.flag as a string. | |
97 | |
98 DEFINE_list: Takes a comma-separated list of strings on the commandline. | |
99 Stores them in a python list object. | |
100 | |
101 DEFINE_spaceseplist: Takes a space-separated list of strings on the | |
102 commandline. Stores them in a python list object. | |
103 Example: --myspacesepflag "foo bar baz" | |
104 | |
105 DEFINE_multistring: The same as DEFINE_string, except the flag can be | |
106 specified more than once on the commandline. The | |
107 result is a python list object (list of strings), | |
108 even if the flag is only on the command line once. | |
109 | |
110 DEFINE_multi_int: The same as DEFINE_integer, except the flag can be | |
111 specified more than once on the commandline. The | |
112 result is a python list object (list of ints), even if | |
113 the flag is only on the command line once. | |
114 | |
115 | |
116 SPECIAL FLAGS: There are a few flags that have special meaning: | |
117 --help prints a list of all the flags in a human-readable fashion | |
118 --helpshort prints a list of all key flags (see below). | |
119 --helpxml prints a list of all flags, in XML format. DO NOT parse | |
120 the output of --help and --helpshort. Instead, parse | |
121 the output of --helpxml. For more info, see | |
122 "OUTPUT FOR --helpxml" below. | |
123 --flagfile=foo read flags from file foo. | |
124 --undefok=f1,f2 ignore unrecognized option errors for f1,f2. | |
125 For boolean flags, you should use --undefok=boolflag, and | |
126 --boolflag and --noboolflag will be accepted. Do not use | |
127 --undefok=noboolflag. | |
128 -- as in getopt(), terminates flag-processing | |
129 | |
130 | |
131 FLAGS VALIDATORS: If your program: | |
132 - requires flag X to be specified | |
133 - needs flag Y to match a regular expression | |
134 - or requires any more general constraint to be satisfied | |
135 then validators are for you! | |
136 | |
137 Each validator represents a constraint over one flag, which is enforced | |
138 starting from the initial parsing of the flags and until the program | |
139 terminates. | |
140 | |
141 Also, lower_bound and upper_bound for numerical flags are enforced using flag | |
142 validators. | |
143 | |
144 Howto: | |
145 If you want to enforce a constraint over one flag, use | |
146 | |
147 gflags.RegisterValidator(flag_name, | |
148 checker, | |
149 message='Flag validation failed', | |
150 flag_values=FLAGS) | |
151 | |
152 After flag values are initially parsed, and after any change to the specified | |
153 flag, method checker(flag_value) will be executed. If constraint is not | |
154 satisfied, an IllegalFlagValue exception will be raised. See | |
155 RegisterValidator's docstring for a detailed explanation on how to construct | |
156 your own checker. | |
157 | |
158 | |
159 EXAMPLE USAGE: | |
160 | |
161 FLAGS = gflags.FLAGS | |
162 | |
163 gflags.DEFINE_integer('my_version', 0, 'Version number.') | |
164 gflags.DEFINE_string('filename', None, 'Input file name', short_name='f') | |
165 | |
166 gflags.RegisterValidator('my_version', | |
167 lambda value: value % 2 == 0, | |
168 message='--my_version must be divisible by 2') | |
169 gflags.MarkFlagAsRequired('filename') | |
170 | |
171 | |
172 NOTE ON --flagfile: | |
173 | |
174 Flags may be loaded from text files in addition to being specified on | |
175 the commandline. | |
176 | |
177 Any flags you don't feel like typing, throw them in a file, one flag per | |
178 line, for instance: | |
179 --myflag=myvalue | |
180 --nomyboolean_flag | |
181 You then specify your file with the special flag '--flagfile=somefile'. | |
182 You CAN recursively nest flagfile= tokens OR use multiple files on the | |
183 command line. Lines beginning with a single hash '#' or a double slash | |
184 '//' are comments in your flagfile. | |
185 | |
186 Any flagfile=<file> will be interpreted as having a relative path from | |
187 the current working directory rather than from the place the file was | |
188 included from: | |
189 myPythonScript.py --flagfile=config/somefile.cfg | |
190 | |
191 If somefile.cfg includes further --flagfile= directives, these will be | |
192 referenced relative to the original CWD, not from the directory the | |
193 including flagfile was found in! | |
194 | |
195 The caveat applies to people who are including a series of nested files | |
196 in a different dir than they are executing out of. Relative path names | |
197 are always from CWD, not from the directory of the parent include | |
198 flagfile. We do now support '~' expanded directory names. | |
199 | |
200 Absolute path names ALWAYS work! | |
201 | |
202 | |
203 EXAMPLE USAGE: | |
204 | |
205 | |
206 FLAGS = gflags.FLAGS | |
207 | |
208 # Flag names are globally defined! So in general, we need to be | |
209 # careful to pick names that are unlikely to be used by other libraries. | |
210 # If there is a conflict, we'll get an error at import time. | |
211 gflags.DEFINE_string('name', 'Mr. President', 'your name') | |
212 gflags.DEFINE_integer('age', None, 'your age in years', lower_bound=0) | |
213 gflags.DEFINE_boolean('debug', False, 'produces debugging output') | |
214 gflags.DEFINE_enum('gender', 'male', ['male', 'female'], 'your gender') | |
215 | |
216 def main(argv): | |
217 try: | |
218 argv = FLAGS(argv) # parse flags | |
219 except gflags.FlagsError, e: | |
220 print '%s\\nUsage: %s ARGS\\n%s' % (e, sys.argv[0], FLAGS) | |
221 sys.exit(1) | |
222 if FLAGS.debug: print 'non-flag arguments:', argv | |
223 print 'Happy Birthday', FLAGS.name | |
224 if FLAGS.age is not None: | |
225 print 'You are a %d year old %s' % (FLAGS.age, FLAGS.gender) | |
226 | |
227 if __name__ == '__main__': | |
228 main(sys.argv) | |
229 | |
230 | |
231 KEY FLAGS: | |
232 | |
233 As we already explained, each module gains access to all flags defined | |
234 by all the other modules it transitively imports. In the case of | |
235 non-trivial scripts, this means a lot of flags ... For documentation | |
236 purposes, it is good to identify the flags that are key (i.e., really | |
237 important) to a module. Clearly, the concept of "key flag" is a | |
238 subjective one. When trying to determine whether a flag is key to a | |
239 module or not, assume that you are trying to explain your module to a | |
240 potential user: which flags would you really like to mention first? | |
241 | |
242 We'll describe shortly how to declare which flags are key to a module. | |
243 For the moment, assume we know the set of key flags for each module. | |
244 Then, if you use the app.py module, you can use the --helpshort flag to | |
245 print only the help for the flags that are key to the main module, in a | |
246 human-readable format. | |
247 | |
248 NOTE: If you need to parse the flag help, do NOT use the output of | |
249 --help / --helpshort. That output is meant for human consumption, and | |
250 may be changed in the future. Instead, use --helpxml; flags that are | |
251 key for the main module are marked there with a <key>yes</key> element. | |
252 | |
253 The set of key flags for a module M is composed of: | |
254 | |
255 1. Flags defined by module M by calling a DEFINE_* function. | |
256 | |
257 2. Flags that module M explictly declares as key by using the function | |
258 | |
259 DECLARE_key_flag(<flag_name>) | |
260 | |
261 3. Key flags of other modules that M specifies by using the function | |
262 | |
263 ADOPT_module_key_flags(<other_module>) | |
264 | |
265 This is a "bulk" declaration of key flags: each flag that is key for | |
266 <other_module> becomes key for the current module too. | |
267 | |
268 Notice that if you do not use the functions described at points 2 and 3 | |
269 above, then --helpshort prints information only about the flags defined | |
270 by the main module of our script. In many cases, this behavior is good | |
271 enough. But if you move part of the main module code (together with the | |
272 related flags) into a different module, then it is nice to use | |
273 DECLARE_key_flag / ADOPT_module_key_flags and make sure --helpshort | |
274 lists all relevant flags (otherwise, your code refactoring may confuse | |
275 your users). | |
276 | |
277 Note: each of DECLARE_key_flag / ADOPT_module_key_flags has its own | |
278 pluses and minuses: DECLARE_key_flag is more targeted and may lead a | |
279 more focused --helpshort documentation. ADOPT_module_key_flags is good | |
280 for cases when an entire module is considered key to the current script. | |
281 Also, it does not require updates to client scripts when a new flag is | |
282 added to the module. | |
283 | |
284 | |
285 EXAMPLE USAGE 2 (WITH KEY FLAGS): | |
286 | |
287 Consider an application that contains the following three files (two | |
288 auxiliary modules and a main module) | |
289 | |
290 File libfoo.py: | |
291 | |
292 import gflags | |
293 | |
294 gflags.DEFINE_integer('num_replicas', 3, 'Number of replicas to start') | |
295 gflags.DEFINE_boolean('rpc2', True, 'Turn on the usage of RPC2.') | |
296 | |
297 ... some code ... | |
298 | |
299 File libbar.py: | |
300 | |
301 import gflags | |
302 | |
303 gflags.DEFINE_string('bar_gfs_path', '/gfs/path', | |
304 'Path to the GFS files for libbar.') | |
305 gflags.DEFINE_string('email_for_bar_errors', 'bar-team@google.com', | |
306 'Email address for bug reports about module libbar.') | |
307 gflags.DEFINE_boolean('bar_risky_hack', False, | |
308 'Turn on an experimental and buggy optimization.') | |
309 | |
310 ... some code ... | |
311 | |
312 File myscript.py: | |
313 | |
314 import gflags | |
315 import libfoo | |
316 import libbar | |
317 | |
318 gflags.DEFINE_integer('num_iterations', 0, 'Number of iterations.') | |
319 | |
320 # Declare that all flags that are key for libfoo are | |
321 # key for this module too. | |
322 gflags.ADOPT_module_key_flags(libfoo) | |
323 | |
324 # Declare that the flag --bar_gfs_path (defined in libbar) is key | |
325 # for this module. | |
326 gflags.DECLARE_key_flag('bar_gfs_path') | |
327 | |
328 ... some code ... | |
329 | |
330 When myscript is invoked with the flag --helpshort, the resulted help | |
331 message lists information about all the key flags for myscript: | |
332 --num_iterations, --num_replicas, --rpc2, and --bar_gfs_path. | |
333 | |
334 Of course, myscript uses all the flags declared by it (in this case, | |
335 just --num_replicas) or by any of the modules it transitively imports | |
336 (e.g., the modules libfoo, libbar). E.g., it can access the value of | |
337 FLAGS.bar_risky_hack, even if --bar_risky_hack is not declared as a key | |
338 flag for myscript. | |
339 | |
340 | |
341 OUTPUT FOR --helpxml: | |
342 | |
343 The --helpxml flag generates output with the following structure: | |
344 | |
345 <?xml version="1.0"?> | |
346 <AllFlags> | |
347 <program>PROGRAM_BASENAME</program> | |
348 <usage>MAIN_MODULE_DOCSTRING</usage> | |
349 (<flag> | |
350 [<key>yes</key>] | |
351 <file>DECLARING_MODULE</file> | |
352 <name>FLAG_NAME</name> | |
353 <meaning>FLAG_HELP_MESSAGE</meaning> | |
354 <default>DEFAULT_FLAG_VALUE</default> | |
355 <current>CURRENT_FLAG_VALUE</current> | |
356 <type>FLAG_TYPE</type> | |
357 [OPTIONAL_ELEMENTS] | |
358 </flag>)* | |
359 </AllFlags> | |
360 | |
361 Notes: | |
362 | |
363 1. The output is intentionally similar to the output generated by the | |
364 C++ command-line flag library. The few differences are due to the | |
365 Python flags that do not have a C++ equivalent (at least not yet), | |
366 e.g., DEFINE_list. | |
367 | |
368 2. New XML elements may be added in the future. | |
369 | |
370 3. DEFAULT_FLAG_VALUE is in serialized form, i.e., the string you can | |
371 pass for this flag on the command-line. E.g., for a flag defined | |
372 using DEFINE_list, this field may be foo,bar, not ['foo', 'bar']. | |
373 | |
374 4. CURRENT_FLAG_VALUE is produced using str(). This means that the | |
375 string 'false' will be represented in the same way as the boolean | |
376 False. Using repr() would have removed this ambiguity and simplified | |
377 parsing, but would have broken the compatibility with the C++ | |
378 command-line flags. | |
379 | |
380 5. OPTIONAL_ELEMENTS describe elements relevant for certain kinds of | |
381 flags: lower_bound, upper_bound (for flags that specify bounds), | |
382 enum_value (for enum flags), list_separator (for flags that consist of | |
383 a list of values, separated by a special token). | |
384 | |
385 6. We do not provide any example here: please use --helpxml instead. | |
386 | |
387 This module requires at least python 2.2.1 to run. | |
388 """ | |
389 | |
390 import cgi | |
391 import getopt | |
392 import os | |
393 import re | |
394 import string | |
395 import struct | |
396 import sys | |
397 # pylint: disable-msg=C6204 | |
398 try: | |
399 import fcntl | |
400 except ImportError: | |
401 fcntl = None | |
402 try: | |
403 # Importing termios will fail on non-unix platforms. | |
404 import termios | |
405 except ImportError: | |
406 termios = None | |
407 | |
408 import gflags_validators | |
409 # pylint: enable-msg=C6204 | |
410 | |
411 | |
412 # Are we running under pychecker? | |
413 _RUNNING_PYCHECKER = 'pychecker.python' in sys.modules | |
414 | |
415 | |
416 def _GetCallingModuleObjectAndName(): | |
417 """Returns the module that's calling into this module. | |
418 | |
419 We generally use this function to get the name of the module calling a | |
420 DEFINE_foo... function. | |
421 """ | |
422 # Walk down the stack to find the first globals dict that's not ours. | |
423 for depth in range(1, sys.getrecursionlimit()): | |
424 if not sys._getframe(depth).f_globals is globals(): | |
425 globals_for_frame = sys._getframe(depth).f_globals | |
426 module, module_name = _GetModuleObjectAndName(globals_for_frame) | |
427 if module_name is not None: | |
428 return module, module_name | |
429 raise AssertionError("No module was found") | |
430 | |
431 | |
432 def _GetCallingModule(): | |
433 """Returns the name of the module that's calling into this module.""" | |
434 return _GetCallingModuleObjectAndName()[1] | |
435 | |
436 | |
437 def _GetThisModuleObjectAndName(): | |
438 """Returns: (module object, module name) for this module.""" | |
439 return _GetModuleObjectAndName(globals()) | |
440 | |
441 | |
442 # module exceptions: | |
443 class FlagsError(Exception): | |
444 """The base class for all flags errors.""" | |
445 pass | |
446 | |
447 | |
448 class DuplicateFlag(FlagsError): | |
449 """Raised if there is a flag naming conflict.""" | |
450 pass | |
451 | |
452 class CantOpenFlagFileError(FlagsError): | |
453 """Raised if flagfile fails to open: doesn't exist, wrong permissions, etc.""" | |
454 pass | |
455 | |
456 | |
457 class DuplicateFlagCannotPropagateNoneToSwig(DuplicateFlag): | |
458 """Special case of DuplicateFlag -- SWIG flag value can't be set to None. | |
459 | |
460 This can be raised when a duplicate flag is created. Even if allow_override is | |
461 True, we still abort if the new value is None, because it's currently | |
462 impossible to pass None default value back to SWIG. See FlagValues.SetDefault | |
463 for details. | |
464 """ | |
465 pass | |
466 | |
467 | |
468 class DuplicateFlagError(DuplicateFlag): | |
469 """A DuplicateFlag whose message cites the conflicting definitions. | |
470 | |
471 A DuplicateFlagError conveys more information than a DuplicateFlag, | |
472 namely the modules where the conflicting definitions occur. This | |
473 class was created to avoid breaking external modules which depend on | |
474 the existing DuplicateFlags interface. | |
475 """ | |
476 | |
477 def __init__(self, flagname, flag_values, other_flag_values=None): | |
478 """Create a DuplicateFlagError. | |
479 | |
480 Args: | |
481 flagname: Name of the flag being redefined. | |
482 flag_values: FlagValues object containing the first definition of | |
483 flagname. | |
484 other_flag_values: If this argument is not None, it should be the | |
485 FlagValues object where the second definition of flagname occurs. | |
486 If it is None, we assume that we're being called when attempting | |
487 to create the flag a second time, and we use the module calling | |
488 this one as the source of the second definition. | |
489 """ | |
490 self.flagname = flagname | |
491 first_module = flag_values.FindModuleDefiningFlag( | |
492 flagname, default='<unknown>') | |
493 if other_flag_values is None: | |
494 second_module = _GetCallingModule() | |
495 else: | |
496 second_module = other_flag_values.FindModuleDefiningFlag( | |
497 flagname, default='<unknown>') | |
498 msg = "The flag '%s' is defined twice. First from %s, Second from %s" % ( | |
499 self.flagname, first_module, second_module) | |
500 DuplicateFlag.__init__(self, msg) | |
501 | |
502 | |
503 class IllegalFlagValue(FlagsError): | |
504 """The flag command line argument is illegal.""" | |
505 pass | |
506 | |
507 | |
508 class UnrecognizedFlag(FlagsError): | |
509 """Raised if a flag is unrecognized.""" | |
510 pass | |
511 | |
512 | |
513 # An UnrecognizedFlagError conveys more information than an UnrecognizedFlag. | |
514 # Since there are external modules that create DuplicateFlags, the interface to | |
515 # DuplicateFlag shouldn't change. The flagvalue will be assigned the full value | |
516 # of the flag and its argument, if any, allowing handling of unrecognized flags | |
517 # in an exception handler. | |
518 # If flagvalue is the empty string, then this exception is an due to a | |
519 # reference to a flag that was not already defined. | |
520 class UnrecognizedFlagError(UnrecognizedFlag): | |
521 def __init__(self, flagname, flagvalue=''): | |
522 self.flagname = flagname | |
523 self.flagvalue = flagvalue | |
524 UnrecognizedFlag.__init__( | |
525 self, "Unknown command line flag '%s'" % flagname) | |
526 | |
527 # Global variable used by expvar | |
528 _exported_flags = {} | |
529 _help_width = 80 # width of help output | |
530 | |
531 | |
532 def GetHelpWidth(): | |
533 """Returns: an integer, the width of help lines that is used in TextWrap.""" | |
534 if (not sys.stdout.isatty()) or (termios is None) or (fcntl is None): | |
535 return _help_width | |
536 try: | |
537 data = fcntl.ioctl(sys.stdout, termios.TIOCGWINSZ, '1234') | |
538 columns = struct.unpack('hh', data)[1] | |
539 # Emacs mode returns 0. | |
540 # Here we assume that any value below 40 is unreasonable | |
541 if columns >= 40: | |
542 return columns | |
543 # Returning an int as default is fine, int(int) just return the int. | |
544 return int(os.getenv('COLUMNS', _help_width)) | |
545 | |
546 except (TypeError, IOError, struct.error): | |
547 return _help_width | |
548 | |
549 | |
550 def CutCommonSpacePrefix(text): | |
551 """Removes a common space prefix from the lines of a multiline text. | |
552 | |
553 If the first line does not start with a space, it is left as it is and | |
554 only in the remaining lines a common space prefix is being searched | |
555 for. That means the first line will stay untouched. This is especially | |
556 useful to turn doc strings into help texts. This is because some | |
557 people prefer to have the doc comment start already after the | |
558 apostrophe and then align the following lines while others have the | |
559 apostrophes on a separate line. | |
560 | |
561 The function also drops trailing empty lines and ignores empty lines | |
562 following the initial content line while calculating the initial | |
563 common whitespace. | |
564 | |
565 Args: | |
566 text: text to work on | |
567 | |
568 Returns: | |
569 the resulting text | |
570 """ | |
571 text_lines = text.splitlines() | |
572 # Drop trailing empty lines | |
573 while text_lines and not text_lines[-1]: | |
574 text_lines = text_lines[:-1] | |
575 if text_lines: | |
576 # We got some content, is the first line starting with a space? | |
577 if text_lines[0] and text_lines[0][0].isspace(): | |
578 text_first_line = [] | |
579 else: | |
580 text_first_line = [text_lines.pop(0)] | |
581 # Calculate length of common leading whitespace (only over content lines) | |
582 common_prefix = os.path.commonprefix([line for line in text_lines if line]) | |
583 space_prefix_len = len(common_prefix) - len(common_prefix.lstrip()) | |
584 # If we have a common space prefix, drop it from all lines | |
585 if space_prefix_len: | |
586 for index in xrange(len(text_lines)): | |
587 if text_lines[index]: | |
588 text_lines[index] = text_lines[index][space_prefix_len:] | |
589 return '\n'.join(text_first_line + text_lines) | |
590 return '' | |
591 | |
592 | |
593 def TextWrap(text, length=None, indent='', firstline_indent=None, tabs=' '): | |
594 """Wraps a given text to a maximum line length and returns it. | |
595 | |
596 We turn lines that only contain whitespace into empty lines. We keep | |
597 new lines and tabs (e.g., we do not treat tabs as spaces). | |
598 | |
599 Args: | |
600 text: text to wrap | |
601 length: maximum length of a line, includes indentation | |
602 if this is None then use GetHelpWidth() | |
603 indent: indent for all but first line | |
604 firstline_indent: indent for first line; if None, fall back to indent | |
605 tabs: replacement for tabs | |
606 | |
607 Returns: | |
608 wrapped text | |
609 | |
610 Raises: | |
611 FlagsError: if indent not shorter than length | |
612 FlagsError: if firstline_indent not shorter than length | |
613 """ | |
614 # Get defaults where callee used None | |
615 if length is None: | |
616 length = GetHelpWidth() | |
617 if indent is None: | |
618 indent = '' | |
619 if len(indent) >= length: | |
620 raise FlagsError('Indent must be shorter than length') | |
621 # In line we will be holding the current line which is to be started | |
622 # with indent (or firstline_indent if available) and then appended | |
623 # with words. | |
624 if firstline_indent is None: | |
625 firstline_indent = '' | |
626 line = indent | |
627 else: | |
628 line = firstline_indent | |
629 if len(firstline_indent) >= length: | |
630 raise FlagsError('First line indent must be shorter than length') | |
631 | |
632 # If the callee does not care about tabs we simply convert them to | |
633 # spaces If callee wanted tabs to be single space then we do that | |
634 # already here. | |
635 if not tabs or tabs == ' ': | |
636 text = text.replace('\t', ' ') | |
637 else: | |
638 tabs_are_whitespace = not tabs.strip() | |
639 | |
640 line_regex = re.compile('([ ]*)(\t*)([^ \t]+)', re.MULTILINE) | |
641 | |
642 # Split the text into lines and the lines with the regex above. The | |
643 # resulting lines are collected in result[]. For each split we get the | |
644 # spaces, the tabs and the next non white space (e.g. next word). | |
645 result = [] | |
646 for text_line in text.splitlines(): | |
647 # Store result length so we can find out whether processing the next | |
648 # line gave any new content | |
649 old_result_len = len(result) | |
650 # Process next line with line_regex. For optimization we do an rstrip(). | |
651 # - process tabs (changes either line or word, see below) | |
652 # - process word (first try to squeeze on line, then wrap or force wrap) | |
653 # Spaces found on the line are ignored, they get added while wrapping as | |
654 # needed. | |
655 for spaces, current_tabs, word in line_regex.findall(text_line.rstrip()): | |
656 # If tabs weren't converted to spaces, handle them now | |
657 if current_tabs: | |
658 # If the last thing we added was a space anyway then drop | |
659 # it. But let's not get rid of the indentation. | |
660 if (((result and line != indent) or | |
661 (not result and line != firstline_indent)) and line[-1] == ' '): | |
662 line = line[:-1] | |
663 # Add the tabs, if that means adding whitespace, just add it at | |
664 # the line, the rstrip() code while shorten the line down if | |
665 # necessary | |
666 if tabs_are_whitespace: | |
667 line += tabs * len(current_tabs) | |
668 else: | |
669 # if not all tab replacement is whitespace we prepend it to the word | |
670 word = tabs * len(current_tabs) + word | |
671 # Handle the case where word cannot be squeezed onto current last line | |
672 if len(line) + len(word) > length and len(indent) + len(word) <= length: | |
673 result.append(line.rstrip()) | |
674 line = indent + word | |
675 word = '' | |
676 # No space left on line or can we append a space? | |
677 if len(line) + 1 >= length: | |
678 result.append(line.rstrip()) | |
679 line = indent | |
680 else: | |
681 line += ' ' | |
682 # Add word and shorten it up to allowed line length. Restart next | |
683 # line with indent and repeat, or add a space if we're done (word | |
684 # finished) This deals with words that cannot fit on one line | |
685 # (e.g. indent + word longer than allowed line length). | |
686 while len(line) + len(word) >= length: | |
687 line += word | |
688 result.append(line[:length]) | |
689 word = line[length:] | |
690 line = indent | |
691 # Default case, simply append the word and a space | |
692 if word: | |
693 line += word + ' ' | |
694 # End of input line. If we have content we finish the line. If the | |
695 # current line is just the indent but we had content in during this | |
696 # original line then we need to add an empty line. | |
697 if (result and line != indent) or (not result and line != firstline_indent): | |
698 result.append(line.rstrip()) | |
699 elif len(result) == old_result_len: | |
700 result.append('') | |
701 line = indent | |
702 | |
703 return '\n'.join(result) | |
704 | |
705 | |
706 def DocToHelp(doc): | |
707 """Takes a __doc__ string and reformats it as help.""" | |
708 | |
709 # Get rid of starting and ending white space. Using lstrip() or even | |
710 # strip() could drop more than maximum of first line and right space | |
711 # of last line. | |
712 doc = doc.strip() | |
713 | |
714 # Get rid of all empty lines | |
715 whitespace_only_line = re.compile('^[ \t]+$', re.M) | |
716 doc = whitespace_only_line.sub('', doc) | |
717 | |
718 # Cut out common space at line beginnings | |
719 doc = CutCommonSpacePrefix(doc) | |
720 | |
721 # Just like this module's comment, comments tend to be aligned somehow. | |
722 # In other words they all start with the same amount of white space | |
723 # 1) keep double new lines | |
724 # 2) keep ws after new lines if not empty line | |
725 # 3) all other new lines shall be changed to a space | |
726 # Solution: Match new lines between non white space and replace with space. | |
727 doc = re.sub('(?<=\S)\n(?=\S)', ' ', doc, re.M) | |
728 | |
729 return doc | |
730 | |
731 | |
732 def _GetModuleObjectAndName(globals_dict): | |
733 """Returns the module that defines a global environment, and its name. | |
734 | |
735 Args: | |
736 globals_dict: A dictionary that should correspond to an environment | |
737 providing the values of the globals. | |
738 | |
739 Returns: | |
740 A pair consisting of (1) module object and (2) module name (a | |
741 string). Returns (None, None) if the module could not be | |
742 identified. | |
743 """ | |
744 # The use of .items() (instead of .iteritems()) is NOT a mistake: if | |
745 # a parallel thread imports a module while we iterate over | |
746 # .iteritems() (not nice, but possible), we get a RuntimeError ... | |
747 # Hence, we use the slightly slower but safer .items(). | |
748 for name, module in sys.modules.items(): | |
749 if getattr(module, '__dict__', None) is globals_dict: | |
750 if name == '__main__': | |
751 # Pick a more informative name for the main module. | |
752 name = sys.argv[0] | |
753 return (module, name) | |
754 return (None, None) | |
755 | |
756 | |
757 def _GetMainModule(): | |
758 """Returns: string, name of the module from which execution started.""" | |
759 # First, try to use the same logic used by _GetCallingModuleObjectAndName(), | |
760 # i.e., call _GetModuleObjectAndName(). For that we first need to | |
761 # find the dictionary that the main module uses to store the | |
762 # globals. | |
763 # | |
764 # That's (normally) the same dictionary object that the deepest | |
765 # (oldest) stack frame is using for globals. | |
766 deepest_frame = sys._getframe(0) | |
767 while deepest_frame.f_back is not None: | |
768 deepest_frame = deepest_frame.f_back | |
769 globals_for_main_module = deepest_frame.f_globals | |
770 main_module_name = _GetModuleObjectAndName(globals_for_main_module)[1] | |
771 # The above strategy fails in some cases (e.g., tools that compute | |
772 # code coverage by redefining, among other things, the main module). | |
773 # If so, just use sys.argv[0]. We can probably always do this, but | |
774 # it's safest to try to use the same logic as _GetCallingModuleObjectAndName() | |
775 if main_module_name is None: | |
776 main_module_name = sys.argv[0] | |
777 return main_module_name | |
778 | |
779 | |
780 class FlagValues: | |
781 """Registry of 'Flag' objects. | |
782 | |
783 A 'FlagValues' can then scan command line arguments, passing flag | |
784 arguments through to the 'Flag' objects that it owns. It also | |
785 provides easy access to the flag values. Typically only one | |
786 'FlagValues' object is needed by an application: gflags.FLAGS | |
787 | |
788 This class is heavily overloaded: | |
789 | |
790 'Flag' objects are registered via __setitem__: | |
791 FLAGS['longname'] = x # register a new flag | |
792 | |
793 The .value attribute of the registered 'Flag' objects can be accessed | |
794 as attributes of this 'FlagValues' object, through __getattr__. Both | |
795 the long and short name of the original 'Flag' objects can be used to | |
796 access its value: | |
797 FLAGS.longname # parsed flag value | |
798 FLAGS.x # parsed flag value (short name) | |
799 | |
800 Command line arguments are scanned and passed to the registered 'Flag' | |
801 objects through the __call__ method. Unparsed arguments, including | |
802 argv[0] (e.g. the program name) are returned. | |
803 argv = FLAGS(sys.argv) # scan command line arguments | |
804 | |
805 The original registered Flag objects can be retrieved through the use | |
806 of the dictionary-like operator, __getitem__: | |
807 x = FLAGS['longname'] # access the registered Flag object | |
808 | |
809 The str() operator of a 'FlagValues' object provides help for all of | |
810 the registered 'Flag' objects. | |
811 """ | |
812 | |
813 def __init__(self): | |
814 # Since everything in this class is so heavily overloaded, the only | |
815 # way of defining and using fields is to access __dict__ directly. | |
816 | |
817 # Dictionary: flag name (string) -> Flag object. | |
818 self.__dict__['__flags'] = {} | |
819 # Dictionary: module name (string) -> list of Flag objects that are defined | |
820 # by that module. | |
821 self.__dict__['__flags_by_module'] = {} | |
822 # Dictionary: module id (int) -> list of Flag objects that are defined by | |
823 # that module. | |
824 self.__dict__['__flags_by_module_id'] = {} | |
825 # Dictionary: module name (string) -> list of Flag objects that are | |
826 # key for that module. | |
827 self.__dict__['__key_flags_by_module'] = {} | |
828 | |
829 # Set if we should use new style gnu_getopt rather than getopt when parsing | |
830 # the args. Only possible with Python 2.3+ | |
831 self.UseGnuGetOpt(False) | |
832 | |
833 def UseGnuGetOpt(self, use_gnu_getopt=True): | |
834 """Use GNU-style scanning. Allows mixing of flag and non-flag arguments. | |
835 | |
836 See http://docs.python.org/library/getopt.html#getopt.gnu_getopt | |
837 | |
838 Args: | |
839 use_gnu_getopt: wether or not to use GNU style scanning. | |
840 """ | |
841 self.__dict__['__use_gnu_getopt'] = use_gnu_getopt | |
842 | |
843 def IsGnuGetOpt(self): | |
844 return self.__dict__['__use_gnu_getopt'] | |
845 | |
846 def FlagDict(self): | |
847 return self.__dict__['__flags'] | |
848 | |
849 def FlagsByModuleDict(self): | |
850 """Returns the dictionary of module_name -> list of defined flags. | |
851 | |
852 Returns: | |
853 A dictionary. Its keys are module names (strings). Its values | |
854 are lists of Flag objects. | |
855 """ | |
856 return self.__dict__['__flags_by_module'] | |
857 | |
858 def FlagsByModuleIdDict(self): | |
859 """Returns the dictionary of module_id -> list of defined flags. | |
860 | |
861 Returns: | |
862 A dictionary. Its keys are module IDs (ints). Its values | |
863 are lists of Flag objects. | |
864 """ | |
865 return self.__dict__['__flags_by_module_id'] | |
866 | |
867 def KeyFlagsByModuleDict(self): | |
868 """Returns the dictionary of module_name -> list of key flags. | |
869 | |
870 Returns: | |
871 A dictionary. Its keys are module names (strings). Its values | |
872 are lists of Flag objects. | |
873 """ | |
874 return self.__dict__['__key_flags_by_module'] | |
875 | |
876 def _RegisterFlagByModule(self, module_name, flag): | |
877 """Records the module that defines a specific flag. | |
878 | |
879 We keep track of which flag is defined by which module so that we | |
880 can later sort the flags by module. | |
881 | |
882 Args: | |
883 module_name: A string, the name of a Python module. | |
884 flag: A Flag object, a flag that is key to the module. | |
885 """ | |
886 flags_by_module = self.FlagsByModuleDict() | |
887 flags_by_module.setdefault(module_name, []).append(flag) | |
888 | |
889 def _RegisterFlagByModuleId(self, module_id, flag): | |
890 """Records the module that defines a specific flag. | |
891 | |
892 Args: | |
893 module_id: An int, the ID of the Python module. | |
894 flag: A Flag object, a flag that is key to the module. | |
895 """ | |
896 flags_by_module_id = self.FlagsByModuleIdDict() | |
897 flags_by_module_id.setdefault(module_id, []).append(flag) | |
898 | |
899 def _RegisterKeyFlagForModule(self, module_name, flag): | |
900 """Specifies that a flag is a key flag for a module. | |
901 | |
902 Args: | |
903 module_name: A string, the name of a Python module. | |
904 flag: A Flag object, a flag that is key to the module. | |
905 """ | |
906 key_flags_by_module = self.KeyFlagsByModuleDict() | |
907 # The list of key flags for the module named module_name. | |
908 key_flags = key_flags_by_module.setdefault(module_name, []) | |
909 # Add flag, but avoid duplicates. | |
910 if flag not in key_flags: | |
911 key_flags.append(flag) | |
912 | |
913 def _GetFlagsDefinedByModule(self, module): | |
914 """Returns the list of flags defined by a module. | |
915 | |
916 Args: | |
917 module: A module object or a module name (a string). | |
918 | |
919 Returns: | |
920 A new list of Flag objects. Caller may update this list as he | |
921 wishes: none of those changes will affect the internals of this | |
922 FlagValue object. | |
923 """ | |
924 if not isinstance(module, str): | |
925 module = module.__name__ | |
926 | |
927 return list(self.FlagsByModuleDict().get(module, [])) | |
928 | |
929 def _GetKeyFlagsForModule(self, module): | |
930 """Returns the list of key flags for a module. | |
931 | |
932 Args: | |
933 module: A module object or a module name (a string) | |
934 | |
935 Returns: | |
936 A new list of Flag objects. Caller may update this list as he | |
937 wishes: none of those changes will affect the internals of this | |
938 FlagValue object. | |
939 """ | |
940 if not isinstance(module, str): | |
941 module = module.__name__ | |
942 | |
943 # Any flag is a key flag for the module that defined it. NOTE: | |
944 # key_flags is a fresh list: we can update it without affecting the | |
945 # internals of this FlagValues object. | |
946 key_flags = self._GetFlagsDefinedByModule(module) | |
947 | |
948 # Take into account flags explicitly declared as key for a module. | |
949 for flag in self.KeyFlagsByModuleDict().get(module, []): | |
950 if flag not in key_flags: | |
951 key_flags.append(flag) | |
952 return key_flags | |
953 | |
954 def FindModuleDefiningFlag(self, flagname, default=None): | |
955 """Return the name of the module defining this flag, or default. | |
956 | |
957 Args: | |
958 flagname: Name of the flag to lookup. | |
959 default: Value to return if flagname is not defined. Defaults | |
960 to None. | |
961 | |
962 Returns: | |
963 The name of the module which registered the flag with this name. | |
964 If no such module exists (i.e. no flag with this name exists), | |
965 we return default. | |
966 """ | |
967 for module, flags in self.FlagsByModuleDict().iteritems(): | |
968 for flag in flags: | |
969 if flag.name == flagname or flag.short_name == flagname: | |
970 return module | |
971 return default | |
972 | |
973 def FindModuleIdDefiningFlag(self, flagname, default=None): | |
974 """Return the ID of the module defining this flag, or default. | |
975 | |
976 Args: | |
977 flagname: Name of the flag to lookup. | |
978 default: Value to return if flagname is not defined. Defaults | |
979 to None. | |
980 | |
981 Returns: | |
982 The ID of the module which registered the flag with this name. | |
983 If no such module exists (i.e. no flag with this name exists), | |
984 we return default. | |
985 """ | |
986 for module_id, flags in self.FlagsByModuleIdDict().iteritems(): | |
987 for flag in flags: | |
988 if flag.name == flagname or flag.short_name == flagname: | |
989 return module_id | |
990 return default | |
991 | |
992 def AppendFlagValues(self, flag_values): | |
993 """Appends flags registered in another FlagValues instance. | |
994 | |
995 Args: | |
996 flag_values: registry to copy from | |
997 """ | |
998 for flag_name, flag in flag_values.FlagDict().iteritems(): | |
999 # Each flags with shortname appears here twice (once under its | |
1000 # normal name, and again with its short name). To prevent | |
1001 # problems (DuplicateFlagError) with double flag registration, we | |
1002 # perform a check to make sure that the entry we're looking at is | |
1003 # for its normal name. | |
1004 if flag_name == flag.name: | |
1005 try: | |
1006 self[flag_name] = flag | |
1007 except DuplicateFlagError: | |
1008 raise DuplicateFlagError(flag_name, self, | |
1009 other_flag_values=flag_values) | |
1010 | |
1011 def RemoveFlagValues(self, flag_values): | |
1012 """Remove flags that were previously appended from another FlagValues. | |
1013 | |
1014 Args: | |
1015 flag_values: registry containing flags to remove. | |
1016 """ | |
1017 for flag_name in flag_values.FlagDict(): | |
1018 self.__delattr__(flag_name) | |
1019 | |
1020 def __setitem__(self, name, flag): | |
1021 """Registers a new flag variable.""" | |
1022 fl = self.FlagDict() | |
1023 if not isinstance(flag, Flag): | |
1024 raise IllegalFlagValue(flag) | |
1025 if not isinstance(name, type("")): | |
1026 raise FlagsError("Flag name must be a string") | |
1027 if len(name) == 0: | |
1028 raise FlagsError("Flag name cannot be empty") | |
1029 # If running under pychecker, duplicate keys are likely to be | |
1030 # defined. Disable check for duplicate keys when pycheck'ing. | |
1031 if (name in fl and not flag.allow_override and | |
1032 not fl[name].allow_override and not _RUNNING_PYCHECKER): | |
1033 module, module_name = _GetCallingModuleObjectAndName() | |
1034 if (self.FindModuleDefiningFlag(name) == module_name and | |
1035 id(module) != self.FindModuleIdDefiningFlag(name)): | |
1036 # If the flag has already been defined by a module with the same name, | |
1037 # but a different ID, we can stop here because it indicates that the | |
1038 # module is simply being imported a subsequent time. | |
1039 return | |
1040 raise DuplicateFlagError(name, self) | |
1041 short_name = flag.short_name | |
1042 if short_name is not None: | |
1043 if (short_name in fl and not flag.allow_override and | |
1044 not fl[short_name].allow_override and not _RUNNING_PYCHECKER): | |
1045 raise DuplicateFlagError(short_name, self) | |
1046 fl[short_name] = flag | |
1047 fl[name] = flag | |
1048 global _exported_flags | |
1049 _exported_flags[name] = flag | |
1050 | |
1051 def __getitem__(self, name): | |
1052 """Retrieves the Flag object for the flag --name.""" | |
1053 return self.FlagDict()[name] | |
1054 | |
1055 def __getattr__(self, name): | |
1056 """Retrieves the 'value' attribute of the flag --name.""" | |
1057 fl = self.FlagDict() | |
1058 if name not in fl: | |
1059 raise AttributeError(name) | |
1060 return fl[name].value | |
1061 | |
1062 def __setattr__(self, name, value): | |
1063 """Sets the 'value' attribute of the flag --name.""" | |
1064 fl = self.FlagDict() | |
1065 fl[name].value = value | |
1066 self._AssertValidators(fl[name].validators) | |
1067 return value | |
1068 | |
1069 def _AssertAllValidators(self): | |
1070 all_validators = set() | |
1071 for flag in self.FlagDict().itervalues(): | |
1072 for validator in flag.validators: | |
1073 all_validators.add(validator) | |
1074 self._AssertValidators(all_validators) | |
1075 | |
1076 def _AssertValidators(self, validators): | |
1077 """Assert if all validators in the list are satisfied. | |
1078 | |
1079 Asserts validators in the order they were created. | |
1080 Args: | |
1081 validators: Iterable(gflags_validators.Validator), validators to be | |
1082 verified | |
1083 Raises: | |
1084 AttributeError: if validators work with a non-existing flag. | |
1085 IllegalFlagValue: if validation fails for at least one validator | |
1086 """ | |
1087 for validator in sorted( | |
1088 validators, key=lambda validator: validator.insertion_index): | |
1089 try: | |
1090 validator.Verify(self) | |
1091 except gflags_validators.Error, e: | |
1092 message = validator.PrintFlagsWithValues(self) | |
1093 raise IllegalFlagValue('%s: %s' % (message, str(e))) | |
1094 | |
1095 def _FlagIsRegistered(self, flag_obj): | |
1096 """Checks whether a Flag object is registered under some name. | |
1097 | |
1098 Note: this is non trivial: in addition to its normal name, a flag | |
1099 may have a short name too. In self.FlagDict(), both the normal and | |
1100 the short name are mapped to the same flag object. E.g., calling | |
1101 only "del FLAGS.short_name" is not unregistering the corresponding | |
1102 Flag object (it is still registered under the longer name). | |
1103 | |
1104 Args: | |
1105 flag_obj: A Flag object. | |
1106 | |
1107 Returns: | |
1108 A boolean: True iff flag_obj is registered under some name. | |
1109 """ | |
1110 flag_dict = self.FlagDict() | |
1111 # Check whether flag_obj is registered under its long name. | |
1112 name = flag_obj.name | |
1113 if flag_dict.get(name, None) == flag_obj: | |
1114 return True | |
1115 # Check whether flag_obj is registered under its short name. | |
1116 short_name = flag_obj.short_name | |
1117 if (short_name is not None and | |
1118 flag_dict.get(short_name, None) == flag_obj): | |
1119 return True | |
1120 # The flag cannot be registered under any other name, so we do not | |
1121 # need to do a full search through the values of self.FlagDict(). | |
1122 return False | |
1123 | |
1124 def __delattr__(self, flag_name): | |
1125 """Deletes a previously-defined flag from a flag object. | |
1126 | |
1127 This method makes sure we can delete a flag by using | |
1128 | |
1129 del flag_values_object.<flag_name> | |
1130 | |
1131 E.g., | |
1132 | |
1133 gflags.DEFINE_integer('foo', 1, 'Integer flag.') | |
1134 del gflags.FLAGS.foo | |
1135 | |
1136 Args: | |
1137 flag_name: A string, the name of the flag to be deleted. | |
1138 | |
1139 Raises: | |
1140 AttributeError: When there is no registered flag named flag_name. | |
1141 """ | |
1142 fl = self.FlagDict() | |
1143 if flag_name not in fl: | |
1144 raise AttributeError(flag_name) | |
1145 | |
1146 flag_obj = fl[flag_name] | |
1147 del fl[flag_name] | |
1148 | |
1149 if not self._FlagIsRegistered(flag_obj): | |
1150 # If the Flag object indicated by flag_name is no longer | |
1151 # registered (please see the docstring of _FlagIsRegistered), then | |
1152 # we delete the occurrences of the flag object in all our internal | |
1153 # dictionaries. | |
1154 self.__RemoveFlagFromDictByModule(self.FlagsByModuleDict(), flag_obj) | |
1155 self.__RemoveFlagFromDictByModule(self.FlagsByModuleIdDict(), flag_obj) | |
1156 self.__RemoveFlagFromDictByModule(self.KeyFlagsByModuleDict(), flag_obj) | |
1157 | |
1158 def __RemoveFlagFromDictByModule(self, flags_by_module_dict, flag_obj): | |
1159 """Removes a flag object from a module -> list of flags dictionary. | |
1160 | |
1161 Args: | |
1162 flags_by_module_dict: A dictionary that maps module names to lists of | |
1163 flags. | |
1164 flag_obj: A flag object. | |
1165 """ | |
1166 for unused_module, flags_in_module in flags_by_module_dict.iteritems(): | |
1167 # while (as opposed to if) takes care of multiple occurrences of a | |
1168 # flag in the list for the same module. | |
1169 while flag_obj in flags_in_module: | |
1170 flags_in_module.remove(flag_obj) | |
1171 | |
1172 def SetDefault(self, name, value): | |
1173 """Changes the default value of the named flag object.""" | |
1174 fl = self.FlagDict() | |
1175 if name not in fl: | |
1176 raise AttributeError(name) | |
1177 fl[name].SetDefault(value) | |
1178 self._AssertValidators(fl[name].validators) | |
1179 | |
1180 def __contains__(self, name): | |
1181 """Returns True if name is a value (flag) in the dict.""" | |
1182 return name in self.FlagDict() | |
1183 | |
1184 has_key = __contains__ # a synonym for __contains__() | |
1185 | |
1186 def __iter__(self): | |
1187 return iter(self.FlagDict()) | |
1188 | |
1189 def __call__(self, argv): | |
1190 """Parses flags from argv; stores parsed flags into this FlagValues object. | |
1191 | |
1192 All unparsed arguments are returned. Flags are parsed using the GNU | |
1193 Program Argument Syntax Conventions, using getopt: | |
1194 | |
1195 http://www.gnu.org/software/libc/manual/html_mono/libc.html#Getopt | |
1196 | |
1197 Args: | |
1198 argv: argument list. Can be of any type that may be converted to a list. | |
1199 | |
1200 Returns: | |
1201 The list of arguments not parsed as options, including argv[0] | |
1202 | |
1203 Raises: | |
1204 FlagsError: on any parsing error | |
1205 """ | |
1206 # Support any sequence type that can be converted to a list | |
1207 argv = list(argv) | |
1208 | |
1209 shortopts = "" | |
1210 longopts = [] | |
1211 | |
1212 fl = self.FlagDict() | |
1213 | |
1214 # This pre parses the argv list for --flagfile=<> options. | |
1215 argv = argv[:1] + self.ReadFlagsFromFiles(argv[1:], force_gnu=False) | |
1216 | |
1217 # Correct the argv to support the google style of passing boolean | |
1218 # parameters. Boolean parameters may be passed by using --mybool, | |
1219 # --nomybool, --mybool=(true|false|1|0). getopt does not support | |
1220 # having options that may or may not have a parameter. We replace | |
1221 # instances of the short form --mybool and --nomybool with their | |
1222 # full forms: --mybool=(true|false). | |
1223 original_argv = list(argv) # list() makes a copy | |
1224 shortest_matches = None | |
1225 for name, flag in fl.items(): | |
1226 if not flag.boolean: | |
1227 continue | |
1228 if shortest_matches is None: | |
1229 # Determine the smallest allowable prefix for all flag names | |
1230 shortest_matches = self.ShortestUniquePrefixes(fl) | |
1231 no_name = 'no' + name | |
1232 prefix = shortest_matches[name] | |
1233 no_prefix = shortest_matches[no_name] | |
1234 | |
1235 # Replace all occurrences of this boolean with extended forms | |
1236 for arg_idx in range(1, len(argv)): | |
1237 arg = argv[arg_idx] | |
1238 if arg.find('=') >= 0: continue | |
1239 if arg.startswith('--'+prefix) and ('--'+name).startswith(arg): | |
1240 argv[arg_idx] = ('--%s=true' % name) | |
1241 elif arg.startswith('--'+no_prefix) and ('--'+no_name).startswith(arg): | |
1242 argv[arg_idx] = ('--%s=false' % name) | |
1243 | |
1244 # Loop over all of the flags, building up the lists of short options | |
1245 # and long options that will be passed to getopt. Short options are | |
1246 # specified as a string of letters, each letter followed by a colon | |
1247 # if it takes an argument. Long options are stored in an array of | |
1248 # strings. Each string ends with an '=' if it takes an argument. | |
1249 for name, flag in fl.items(): | |
1250 longopts.append(name + "=") | |
1251 if len(name) == 1: # one-letter option: allow short flag type also | |
1252 shortopts += name | |
1253 if not flag.boolean: | |
1254 shortopts += ":" | |
1255 | |
1256 longopts.append('undefok=') | |
1257 undefok_flags = [] | |
1258 | |
1259 # In case --undefok is specified, loop to pick up unrecognized | |
1260 # options one by one. | |
1261 unrecognized_opts = [] | |
1262 args = argv[1:] | |
1263 while True: | |
1264 try: | |
1265 if self.__dict__['__use_gnu_getopt']: | |
1266 optlist, unparsed_args = getopt.gnu_getopt(args, shortopts, longopts) | |
1267 else: | |
1268 optlist, unparsed_args = getopt.getopt(args, shortopts, longopts) | |
1269 break | |
1270 except getopt.GetoptError, e: | |
1271 if not e.opt or e.opt in fl: | |
1272 # Not an unrecognized option, re-raise the exception as a FlagsError | |
1273 raise FlagsError(e) | |
1274 # Remove offender from args and try again | |
1275 for arg_index in range(len(args)): | |
1276 if ((args[arg_index] == '--' + e.opt) or | |
1277 (args[arg_index] == '-' + e.opt) or | |
1278 (args[arg_index].startswith('--' + e.opt + '='))): | |
1279 unrecognized_opts.append((e.opt, args[arg_index])) | |
1280 args = args[0:arg_index] + args[arg_index+1:] | |
1281 break | |
1282 else: | |
1283 # We should have found the option, so we don't expect to get | |
1284 # here. We could assert, but raising the original exception | |
1285 # might work better. | |
1286 raise FlagsError(e) | |
1287 | |
1288 for name, arg in optlist: | |
1289 if name == '--undefok': | |
1290 flag_names = arg.split(',') | |
1291 undefok_flags.extend(flag_names) | |
1292 # For boolean flags, if --undefok=boolflag is specified, then we should | |
1293 # also accept --noboolflag, in addition to --boolflag. | |
1294 # Since we don't know the type of the undefok'd flag, this will affect | |
1295 # non-boolean flags as well. | |
1296 # NOTE: You shouldn't use --undefok=noboolflag, because then we will | |
1297 # accept --nonoboolflag here. We are choosing not to do the conversion | |
1298 # from noboolflag -> boolflag because of the ambiguity that flag names | |
1299 # can start with 'no'. | |
1300 undefok_flags.extend('no' + name for name in flag_names) | |
1301 continue | |
1302 if name.startswith('--'): | |
1303 # long option | |
1304 name = name[2:] | |
1305 short_option = 0 | |
1306 else: | |
1307 # short option | |
1308 name = name[1:] | |
1309 short_option = 1 | |
1310 if name in fl: | |
1311 flag = fl[name] | |
1312 if flag.boolean and short_option: arg = 1 | |
1313 flag.Parse(arg) | |
1314 | |
1315 # If there were unrecognized options, raise an exception unless | |
1316 # the options were named via --undefok. | |
1317 for opt, value in unrecognized_opts: | |
1318 if opt not in undefok_flags: | |
1319 raise UnrecognizedFlagError(opt, value) | |
1320 | |
1321 if unparsed_args: | |
1322 if self.__dict__['__use_gnu_getopt']: | |
1323 # if using gnu_getopt just return the program name + remainder of argv. | |
1324 ret_val = argv[:1] + unparsed_args | |
1325 else: | |
1326 # unparsed_args becomes the first non-flag detected by getopt to | |
1327 # the end of argv. Because argv may have been modified above, | |
1328 # return original_argv for this region. | |
1329 ret_val = argv[:1] + original_argv[-len(unparsed_args):] | |
1330 else: | |
1331 ret_val = argv[:1] | |
1332 | |
1333 self._AssertAllValidators() | |
1334 return ret_val | |
1335 | |
1336 def Reset(self): | |
1337 """Resets the values to the point before FLAGS(argv) was called.""" | |
1338 for f in self.FlagDict().values(): | |
1339 f.Unparse() | |
1340 | |
1341 def RegisteredFlags(self): | |
1342 """Returns: a list of the names and short names of all registered flags.""" | |
1343 return list(self.FlagDict()) | |
1344 | |
1345 def FlagValuesDict(self): | |
1346 """Returns: a dictionary that maps flag names to flag values.""" | |
1347 flag_values = {} | |
1348 | |
1349 for flag_name in self.RegisteredFlags(): | |
1350 flag = self.FlagDict()[flag_name] | |
1351 flag_values[flag_name] = flag.value | |
1352 | |
1353 return flag_values | |
1354 | |
1355 def __str__(self): | |
1356 """Generates a help string for all known flags.""" | |
1357 return self.GetHelp() | |
1358 | |
1359 def GetHelp(self, prefix=''): | |
1360 """Generates a help string for all known flags.""" | |
1361 helplist = [] | |
1362 | |
1363 flags_by_module = self.FlagsByModuleDict() | |
1364 if flags_by_module: | |
1365 | |
1366 modules = sorted(flags_by_module) | |
1367 | |
1368 # Print the help for the main module first, if possible. | |
1369 main_module = _GetMainModule() | |
1370 if main_module in modules: | |
1371 modules.remove(main_module) | |
1372 modules = [main_module] + modules | |
1373 | |
1374 for module in modules: | |
1375 self.__RenderOurModuleFlags(module, helplist) | |
1376 | |
1377 self.__RenderModuleFlags('gflags', | |
1378 _SPECIAL_FLAGS.FlagDict().values(), | |
1379 helplist) | |
1380 | |
1381 else: | |
1382 # Just print one long list of flags. | |
1383 self.__RenderFlagList( | |
1384 self.FlagDict().values() + _SPECIAL_FLAGS.FlagDict().values(), | |
1385 helplist, prefix) | |
1386 | |
1387 return '\n'.join(helplist) | |
1388 | |
1389 def __RenderModuleFlags(self, module, flags, output_lines, prefix=""): | |
1390 """Generates a help string for a given module.""" | |
1391 if not isinstance(module, str): | |
1392 module = module.__name__ | |
1393 output_lines.append('\n%s%s:' % (prefix, module)) | |
1394 self.__RenderFlagList(flags, output_lines, prefix + " ") | |
1395 | |
1396 def __RenderOurModuleFlags(self, module, output_lines, prefix=""): | |
1397 """Generates a help string for a given module.""" | |
1398 flags = self._GetFlagsDefinedByModule(module) | |
1399 if flags: | |
1400 self.__RenderModuleFlags(module, flags, output_lines, prefix) | |
1401 | |
1402 def __RenderOurModuleKeyFlags(self, module, output_lines, prefix=""): | |
1403 """Generates a help string for the key flags of a given module. | |
1404 | |
1405 Args: | |
1406 module: A module object or a module name (a string). | |
1407 output_lines: A list of strings. The generated help message | |
1408 lines will be appended to this list. | |
1409 prefix: A string that is prepended to each generated help line. | |
1410 """ | |
1411 key_flags = self._GetKeyFlagsForModule(module) | |
1412 if key_flags: | |
1413 self.__RenderModuleFlags(module, key_flags, output_lines, prefix) | |
1414 | |
1415 def ModuleHelp(self, module): | |
1416 """Describe the key flags of a module. | |
1417 | |
1418 Args: | |
1419 module: A module object or a module name (a string). | |
1420 | |
1421 Returns: | |
1422 string describing the key flags of a module. | |
1423 """ | |
1424 helplist = [] | |
1425 self.__RenderOurModuleKeyFlags(module, helplist) | |
1426 return '\n'.join(helplist) | |
1427 | |
1428 def MainModuleHelp(self): | |
1429 """Describe the key flags of the main module. | |
1430 | |
1431 Returns: | |
1432 string describing the key flags of a module. | |
1433 """ | |
1434 return self.ModuleHelp(_GetMainModule()) | |
1435 | |
1436 def __RenderFlagList(self, flaglist, output_lines, prefix=" "): | |
1437 fl = self.FlagDict() | |
1438 special_fl = _SPECIAL_FLAGS.FlagDict() | |
1439 flaglist = [(flag.name, flag) for flag in flaglist] | |
1440 flaglist.sort() | |
1441 flagset = {} | |
1442 for (name, flag) in flaglist: | |
1443 # It's possible this flag got deleted or overridden since being | |
1444 # registered in the per-module flaglist. Check now against the | |
1445 # canonical source of current flag information, the FlagDict. | |
1446 if fl.get(name, None) != flag and special_fl.get(name, None) != flag: | |
1447 # a different flag is using this name now | |
1448 continue | |
1449 # only print help once | |
1450 if flag in flagset: continue | |
1451 flagset[flag] = 1 | |
1452 flaghelp = "" | |
1453 if flag.short_name: flaghelp += "-%s," % flag.short_name | |
1454 if flag.boolean: | |
1455 flaghelp += "--[no]%s" % flag.name + ":" | |
1456 else: | |
1457 flaghelp += "--%s" % flag.name + ":" | |
1458 flaghelp += " " | |
1459 if flag.help: | |
1460 flaghelp += flag.help | |
1461 flaghelp = TextWrap(flaghelp, indent=prefix+" ", | |
1462 firstline_indent=prefix) | |
1463 if flag.default_as_str: | |
1464 flaghelp += "\n" | |
1465 flaghelp += TextWrap("(default: %s)" % flag.default_as_str, | |
1466 indent=prefix+" ") | |
1467 if flag.parser.syntactic_help: | |
1468 flaghelp += "\n" | |
1469 flaghelp += TextWrap("(%s)" % flag.parser.syntactic_help, | |
1470 indent=prefix+" ") | |
1471 output_lines.append(flaghelp) | |
1472 | |
1473 def get(self, name, default): | |
1474 """Returns the value of a flag (if not None) or a default value. | |
1475 | |
1476 Args: | |
1477 name: A string, the name of a flag. | |
1478 default: Default value to use if the flag value is None. | |
1479 """ | |
1480 | |
1481 value = self.__getattr__(name) | |
1482 if value is not None: # Can't do if not value, b/c value might be '0' or "" | |
1483 return value | |
1484 else: | |
1485 return default | |
1486 | |
1487 def ShortestUniquePrefixes(self, fl): | |
1488 """Returns: dictionary; maps flag names to their shortest unique prefix.""" | |
1489 # Sort the list of flag names | |
1490 sorted_flags = [] | |
1491 for name, flag in fl.items(): | |
1492 sorted_flags.append(name) | |
1493 if flag.boolean: | |
1494 sorted_flags.append('no%s' % name) | |
1495 sorted_flags.sort() | |
1496 | |
1497 # For each name in the sorted list, determine the shortest unique | |
1498 # prefix by comparing itself to the next name and to the previous | |
1499 # name (the latter check uses cached info from the previous loop). | |
1500 shortest_matches = {} | |
1501 prev_idx = 0 | |
1502 for flag_idx in range(len(sorted_flags)): | |
1503 curr = sorted_flags[flag_idx] | |
1504 if flag_idx == (len(sorted_flags) - 1): | |
1505 next = None | |
1506 else: | |
1507 next = sorted_flags[flag_idx+1] | |
1508 next_len = len(next) | |
1509 for curr_idx in range(len(curr)): | |
1510 if (next is None | |
1511 or curr_idx >= next_len | |
1512 or curr[curr_idx] != next[curr_idx]): | |
1513 # curr longer than next or no more chars in common | |
1514 shortest_matches[curr] = curr[:max(prev_idx, curr_idx) + 1] | |
1515 prev_idx = curr_idx | |
1516 break | |
1517 else: | |
1518 # curr shorter than (or equal to) next | |
1519 shortest_matches[curr] = curr | |
1520 prev_idx = curr_idx + 1 # next will need at least one more char | |
1521 return shortest_matches | |
1522 | |
1523 def __IsFlagFileDirective(self, flag_string): | |
1524 """Checks whether flag_string contain a --flagfile=<foo> directive.""" | |
1525 if isinstance(flag_string, type("")): | |
1526 if flag_string.startswith('--flagfile='): | |
1527 return 1 | |
1528 elif flag_string == '--flagfile': | |
1529 return 1 | |
1530 elif flag_string.startswith('-flagfile='): | |
1531 return 1 | |
1532 elif flag_string == '-flagfile': | |
1533 return 1 | |
1534 else: | |
1535 return 0 | |
1536 return 0 | |
1537 | |
1538 def ExtractFilename(self, flagfile_str): | |
1539 """Returns filename from a flagfile_str of form -[-]flagfile=filename. | |
1540 | |
1541 The cases of --flagfile foo and -flagfile foo shouldn't be hitting | |
1542 this function, as they are dealt with in the level above this | |
1543 function. | |
1544 """ | |
1545 if flagfile_str.startswith('--flagfile='): | |
1546 return os.path.expanduser((flagfile_str[(len('--flagfile=')):]).strip()) | |
1547 elif flagfile_str.startswith('-flagfile='): | |
1548 return os.path.expanduser((flagfile_str[(len('-flagfile=')):]).strip()) | |
1549 else: | |
1550 raise FlagsError('Hit illegal --flagfile type: %s' % flagfile_str) | |
1551 | |
1552 def __GetFlagFileLines(self, filename, parsed_file_list): | |
1553 """Returns the useful (!=comments, etc) lines from a file with flags. | |
1554 | |
1555 Args: | |
1556 filename: A string, the name of the flag file. | |
1557 parsed_file_list: A list of the names of the files we have | |
1558 already read. MUTATED BY THIS FUNCTION. | |
1559 | |
1560 Returns: | |
1561 List of strings. See the note below. | |
1562 | |
1563 NOTE(springer): This function checks for a nested --flagfile=<foo> | |
1564 tag and handles the lower file recursively. It returns a list of | |
1565 all the lines that _could_ contain command flags. This is | |
1566 EVERYTHING except whitespace lines and comments (lines starting | |
1567 with '#' or '//'). | |
1568 """ | |
1569 line_list = [] # All line from flagfile. | |
1570 flag_line_list = [] # Subset of lines w/o comments, blanks, flagfile= tags. | |
1571 try: | |
1572 file_obj = open(filename, 'r') | |
1573 except IOError, e_msg: | |
1574 raise CantOpenFlagFileError('ERROR:: Unable to open flagfile: %s' % e_msg) | |
1575 | |
1576 line_list = file_obj.readlines() | |
1577 file_obj.close() | |
1578 parsed_file_list.append(filename) | |
1579 | |
1580 # This is where we check each line in the file we just read. | |
1581 for line in line_list: | |
1582 if line.isspace(): | |
1583 pass | |
1584 # Checks for comment (a line that starts with '#'). | |
1585 elif line.startswith('#') or line.startswith('//'): | |
1586 pass | |
1587 # Checks for a nested "--flagfile=<bar>" flag in the current file. | |
1588 # If we find one, recursively parse down into that file. | |
1589 elif self.__IsFlagFileDirective(line): | |
1590 sub_filename = self.ExtractFilename(line) | |
1591 # We do a little safety check for reparsing a file we've already done. | |
1592 if not sub_filename in parsed_file_list: | |
1593 included_flags = self.__GetFlagFileLines(sub_filename, | |
1594 parsed_file_list) | |
1595 flag_line_list.extend(included_flags) | |
1596 else: # Case of hitting a circularly included file. | |
1597 sys.stderr.write('Warning: Hit circular flagfile dependency: %s\n' % | |
1598 (sub_filename,)) | |
1599 else: | |
1600 # Any line that's not a comment or a nested flagfile should get | |
1601 # copied into 2nd position. This leaves earlier arguments | |
1602 # further back in the list, thus giving them higher priority. | |
1603 flag_line_list.append(line.strip()) | |
1604 return flag_line_list | |
1605 | |
1606 def ReadFlagsFromFiles(self, argv, force_gnu=True): | |
1607 """Processes command line args, but also allow args to be read from file. | |
1608 | |
1609 Args: | |
1610 argv: A list of strings, usually sys.argv[1:], which may contain one or | |
1611 more flagfile directives of the form --flagfile="./filename". | |
1612 Note that the name of the program (sys.argv[0]) should be omitted. | |
1613 force_gnu: If False, --flagfile parsing obeys normal flag semantics. | |
1614 If True, --flagfile parsing instead follows gnu_getopt semantics. | |
1615 *** WARNING *** force_gnu=False may become the future default! | |
1616 | |
1617 Returns: | |
1618 | |
1619 A new list which has the original list combined with what we read | |
1620 from any flagfile(s). | |
1621 | |
1622 References: Global gflags.FLAG class instance. | |
1623 | |
1624 This function should be called before the normal FLAGS(argv) call. | |
1625 This function scans the input list for a flag that looks like: | |
1626 --flagfile=<somefile>. Then it opens <somefile>, reads all valid key | |
1627 and value pairs and inserts them into the input list between the | |
1628 first item of the list and any subsequent items in the list. | |
1629 | |
1630 Note that your application's flags are still defined the usual way | |
1631 using gflags DEFINE_flag() type functions. | |
1632 | |
1633 Notes (assuming we're getting a commandline of some sort as our input): | |
1634 --> Flags from the command line argv _should_ always take precedence! | |
1635 --> A further "--flagfile=<otherfile.cfg>" CAN be nested in a flagfile. | |
1636 It will be processed after the parent flag file is done. | |
1637 --> For duplicate flags, first one we hit should "win". | |
1638 --> In a flagfile, a line beginning with # or // is a comment. | |
1639 --> Entirely blank lines _should_ be ignored. | |
1640 """ | |
1641 parsed_file_list = [] | |
1642 rest_of_args = argv | |
1643 new_argv = [] | |
1644 while rest_of_args: | |
1645 current_arg = rest_of_args[0] | |
1646 rest_of_args = rest_of_args[1:] | |
1647 if self.__IsFlagFileDirective(current_arg): | |
1648 # This handles the case of -(-)flagfile foo. In this case the | |
1649 # next arg really is part of this one. | |
1650 if current_arg == '--flagfile' or current_arg == '-flagfile': | |
1651 if not rest_of_args: | |
1652 raise IllegalFlagValue('--flagfile with no argument') | |
1653 flag_filename = os.path.expanduser(rest_of_args[0]) | |
1654 rest_of_args = rest_of_args[1:] | |
1655 else: | |
1656 # This handles the case of (-)-flagfile=foo. | |
1657 flag_filename = self.ExtractFilename(current_arg) | |
1658 new_argv.extend( | |
1659 self.__GetFlagFileLines(flag_filename, parsed_file_list)) | |
1660 else: | |
1661 new_argv.append(current_arg) | |
1662 # Stop parsing after '--', like getopt and gnu_getopt. | |
1663 if current_arg == '--': | |
1664 break | |
1665 # Stop parsing after a non-flag, like getopt. | |
1666 if not current_arg.startswith('-'): | |
1667 if not force_gnu and not self.__dict__['__use_gnu_getopt']: | |
1668 break | |
1669 | |
1670 if rest_of_args: | |
1671 new_argv.extend(rest_of_args) | |
1672 | |
1673 return new_argv | |
1674 | |
1675 def FlagsIntoString(self): | |
1676 """Returns a string with the flags assignments from this FlagValues object. | |
1677 | |
1678 This function ignores flags whose value is None. Each flag | |
1679 assignment is separated by a newline. | |
1680 | |
1681 NOTE: MUST mirror the behavior of the C++ CommandlineFlagsIntoString | |
1682 from http://code.google.com/p/google-gflags | |
1683 """ | |
1684 s = '' | |
1685 for flag in self.FlagDict().values(): | |
1686 if flag.value is not None: | |
1687 s += flag.Serialize() + '\n' | |
1688 return s | |
1689 | |
1690 def AppendFlagsIntoFile(self, filename): | |
1691 """Appends all flags assignments from this FlagInfo object to a file. | |
1692 | |
1693 Output will be in the format of a flagfile. | |
1694 | |
1695 NOTE: MUST mirror the behavior of the C++ AppendFlagsIntoFile | |
1696 from http://code.google.com/p/google-gflags | |
1697 """ | |
1698 out_file = open(filename, 'a') | |
1699 out_file.write(self.FlagsIntoString()) | |
1700 out_file.close() | |
1701 | |
1702 def WriteHelpInXMLFormat(self, outfile=None): | |
1703 """Outputs flag documentation in XML format. | |
1704 | |
1705 NOTE: We use element names that are consistent with those used by | |
1706 the C++ command-line flag library, from | |
1707 http://code.google.com/p/google-gflags | |
1708 We also use a few new elements (e.g., <key>), but we do not | |
1709 interfere / overlap with existing XML elements used by the C++ | |
1710 library. Please maintain this consistency. | |
1711 | |
1712 Args: | |
1713 outfile: File object we write to. Default None means sys.stdout. | |
1714 """ | |
1715 outfile = outfile or sys.stdout | |
1716 | |
1717 outfile.write('<?xml version=\"1.0\"?>\n') | |
1718 outfile.write('<AllFlags>\n') | |
1719 indent = ' ' | |
1720 _WriteSimpleXMLElement(outfile, 'program', os.path.basename(sys.argv[0]), | |
1721 indent) | |
1722 | |
1723 usage_doc = sys.modules['__main__'].__doc__ | |
1724 if not usage_doc: | |
1725 usage_doc = '\nUSAGE: %s [flags]\n' % sys.argv[0] | |
1726 else: | |
1727 usage_doc = usage_doc.replace('%s', sys.argv[0]) | |
1728 _WriteSimpleXMLElement(outfile, 'usage', usage_doc, indent) | |
1729 | |
1730 # Get list of key flags for the main module. | |
1731 key_flags = self._GetKeyFlagsForModule(_GetMainModule()) | |
1732 | |
1733 # Sort flags by declaring module name and next by flag name. | |
1734 flags_by_module = self.FlagsByModuleDict() | |
1735 all_module_names = list(flags_by_module.keys()) | |
1736 all_module_names.sort() | |
1737 for module_name in all_module_names: | |
1738 flag_list = [(f.name, f) for f in flags_by_module[module_name]] | |
1739 flag_list.sort() | |
1740 for unused_flag_name, flag in flag_list: | |
1741 is_key = flag in key_flags | |
1742 flag.WriteInfoInXMLFormat(outfile, module_name, | |
1743 is_key=is_key, indent=indent) | |
1744 | |
1745 outfile.write('</AllFlags>\n') | |
1746 outfile.flush() | |
1747 | |
1748 def AddValidator(self, validator): | |
1749 """Register new flags validator to be checked. | |
1750 | |
1751 Args: | |
1752 validator: gflags_validators.Validator | |
1753 Raises: | |
1754 AttributeError: if validators work with a non-existing flag. | |
1755 """ | |
1756 for flag_name in validator.GetFlagsNames(): | |
1757 flag = self.FlagDict()[flag_name] | |
1758 flag.validators.append(validator) | |
1759 | |
1760 # end of FlagValues definition | |
1761 | |
1762 | |
1763 # The global FlagValues instance | |
1764 FLAGS = FlagValues() | |
1765 | |
1766 | |
1767 def _StrOrUnicode(value): | |
1768 """Converts value to a python string or, if necessary, unicode-string.""" | |
1769 try: | |
1770 return str(value) | |
1771 except UnicodeEncodeError: | |
1772 return unicode(value) | |
1773 | |
1774 | |
1775 def _MakeXMLSafe(s): | |
1776 """Escapes <, >, and & from s, and removes XML 1.0-illegal chars.""" | |
1777 s = cgi.escape(s) # Escape <, >, and & | |
1778 # Remove characters that cannot appear in an XML 1.0 document | |
1779 # (http://www.w3.org/TR/REC-xml/#charsets). | |
1780 # | |
1781 # NOTE: if there are problems with current solution, one may move to | |
1782 # XML 1.1, which allows such chars, if they're entity-escaped (&#xHH;). | |
1783 s = re.sub(r'[\x00-\x08\x0b\x0c\x0e-\x1f]', '', s) | |
1784 # Convert non-ascii characters to entities. Note: requires python >=2.3 | |
1785 s = s.encode('ascii', 'xmlcharrefreplace') # u'\xce\x88' -> 'uΈ' | |
1786 return s | |
1787 | |
1788 | |
1789 def _WriteSimpleXMLElement(outfile, name, value, indent): | |
1790 """Writes a simple XML element. | |
1791 | |
1792 Args: | |
1793 outfile: File object we write the XML element to. | |
1794 name: A string, the name of XML element. | |
1795 value: A Python object, whose string representation will be used | |
1796 as the value of the XML element. | |
1797 indent: A string, prepended to each line of generated output. | |
1798 """ | |
1799 value_str = _StrOrUnicode(value) | |
1800 if isinstance(value, bool): | |
1801 # Display boolean values as the C++ flag library does: no caps. | |
1802 value_str = value_str.lower() | |
1803 safe_value_str = _MakeXMLSafe(value_str) | |
1804 outfile.write('%s<%s>%s</%s>\n' % (indent, name, safe_value_str, name)) | |
1805 | |
1806 | |
1807 class Flag: | |
1808 """Information about a command-line flag. | |
1809 | |
1810 'Flag' objects define the following fields: | |
1811 .name - the name for this flag | |
1812 .default - the default value for this flag | |
1813 .default_as_str - default value as repr'd string, e.g., "'true'" (or None) | |
1814 .value - the most recent parsed value of this flag; set by Parse() | |
1815 .help - a help string or None if no help is available | |
1816 .short_name - the single letter alias for this flag (or None) | |
1817 .boolean - if 'true', this flag does not accept arguments | |
1818 .present - true if this flag was parsed from command line flags. | |
1819 .parser - an ArgumentParser object | |
1820 .serializer - an ArgumentSerializer object | |
1821 .allow_override - the flag may be redefined without raising an error | |
1822 | |
1823 The only public method of a 'Flag' object is Parse(), but it is | |
1824 typically only called by a 'FlagValues' object. The Parse() method is | |
1825 a thin wrapper around the 'ArgumentParser' Parse() method. The parsed | |
1826 value is saved in .value, and the .present attribute is updated. If | |
1827 this flag was already present, a FlagsError is raised. | |
1828 | |
1829 Parse() is also called during __init__ to parse the default value and | |
1830 initialize the .value attribute. This enables other python modules to | |
1831 safely use flags even if the __main__ module neglects to parse the | |
1832 command line arguments. The .present attribute is cleared after | |
1833 __init__ parsing. If the default value is set to None, then the | |
1834 __init__ parsing step is skipped and the .value attribute is | |
1835 initialized to None. | |
1836 | |
1837 Note: The default value is also presented to the user in the help | |
1838 string, so it is important that it be a legal value for this flag. | |
1839 """ | |
1840 | |
1841 def __init__(self, parser, serializer, name, default, help_string, | |
1842 short_name=None, boolean=0, allow_override=0): | |
1843 self.name = name | |
1844 | |
1845 if not help_string: | |
1846 help_string = '(no help available)' | |
1847 | |
1848 self.help = help_string | |
1849 self.short_name = short_name | |
1850 self.boolean = boolean | |
1851 self.present = 0 | |
1852 self.parser = parser | |
1853 self.serializer = serializer | |
1854 self.allow_override = allow_override | |
1855 self.value = None | |
1856 self.validators = [] | |
1857 | |
1858 self.SetDefault(default) | |
1859 | |
1860 def __hash__(self): | |
1861 return hash(id(self)) | |
1862 | |
1863 def __eq__(self, other): | |
1864 return self is other | |
1865 | |
1866 def __lt__(self, other): | |
1867 if isinstance(other, Flag): | |
1868 return id(self) < id(other) | |
1869 return NotImplemented | |
1870 | |
1871 def __GetParsedValueAsString(self, value): | |
1872 if value is None: | |
1873 return None | |
1874 if self.serializer: | |
1875 return repr(self.serializer.Serialize(value)) | |
1876 if self.boolean: | |
1877 if value: | |
1878 return repr('true') | |
1879 else: | |
1880 return repr('false') | |
1881 return repr(_StrOrUnicode(value)) | |
1882 | |
1883 def Parse(self, argument): | |
1884 try: | |
1885 self.value = self.parser.Parse(argument) | |
1886 except ValueError, e: # recast ValueError as IllegalFlagValue | |
1887 raise IllegalFlagValue("flag --%s=%s: %s" % (self.name, argument, e)) | |
1888 self.present += 1 | |
1889 | |
1890 def Unparse(self): | |
1891 if self.default is None: | |
1892 self.value = None | |
1893 else: | |
1894 self.Parse(self.default) | |
1895 self.present = 0 | |
1896 | |
1897 def Serialize(self): | |
1898 if self.value is None: | |
1899 return '' | |
1900 if self.boolean: | |
1901 if self.value: | |
1902 return "--%s" % self.name | |
1903 else: | |
1904 return "--no%s" % self.name | |
1905 else: | |
1906 if not self.serializer: | |
1907 raise FlagsError("Serializer not present for flag %s" % self.name) | |
1908 return "--%s=%s" % (self.name, self.serializer.Serialize(self.value)) | |
1909 | |
1910 def SetDefault(self, value): | |
1911 """Changes the default value (and current value too) for this Flag.""" | |
1912 # We can't allow a None override because it may end up not being | |
1913 # passed to C++ code when we're overriding C++ flags. So we | |
1914 # cowardly bail out until someone fixes the semantics of trying to | |
1915 # pass None to a C++ flag. See swig_flags.Init() for details on | |
1916 # this behavior. | |
1917 # TODO(olexiy): Users can directly call this method, bypassing all flags | |
1918 # validators (we don't have FlagValues here, so we can not check | |
1919 # validators). | |
1920 # The simplest solution I see is to make this method private. | |
1921 # Another approach would be to store reference to the corresponding | |
1922 # FlagValues with each flag, but this seems to be an overkill. | |
1923 if value is None and self.allow_override: | |
1924 raise DuplicateFlagCannotPropagateNoneToSwig(self.name) | |
1925 | |
1926 self.default = value | |
1927 self.Unparse() | |
1928 self.default_as_str = self.__GetParsedValueAsString(self.value) | |
1929 | |
1930 def Type(self): | |
1931 """Returns: a string that describes the type of this Flag.""" | |
1932 # NOTE: we use strings, and not the types.*Type constants because | |
1933 # our flags can have more exotic types, e.g., 'comma separated list | |
1934 # of strings', 'whitespace separated list of strings', etc. | |
1935 return self.parser.Type() | |
1936 | |
1937 def WriteInfoInXMLFormat(self, outfile, module_name, is_key=False, indent=''): | |
1938 """Writes common info about this flag, in XML format. | |
1939 | |
1940 This is information that is relevant to all flags (e.g., name, | |
1941 meaning, etc.). If you defined a flag that has some other pieces of | |
1942 info, then please override _WriteCustomInfoInXMLFormat. | |
1943 | |
1944 Please do NOT override this method. | |
1945 | |
1946 Args: | |
1947 outfile: File object we write to. | |
1948 module_name: A string, the name of the module that defines this flag. | |
1949 is_key: A boolean, True iff this flag is key for main module. | |
1950 indent: A string that is prepended to each generated line. | |
1951 """ | |
1952 outfile.write(indent + '<flag>\n') | |
1953 inner_indent = indent + ' ' | |
1954 if is_key: | |
1955 _WriteSimpleXMLElement(outfile, 'key', 'yes', inner_indent) | |
1956 _WriteSimpleXMLElement(outfile, 'file', module_name, inner_indent) | |
1957 # Print flag features that are relevant for all flags. | |
1958 _WriteSimpleXMLElement(outfile, 'name', self.name, inner_indent) | |
1959 if self.short_name: | |
1960 _WriteSimpleXMLElement(outfile, 'short_name', self.short_name, | |
1961 inner_indent) | |
1962 if self.help: | |
1963 _WriteSimpleXMLElement(outfile, 'meaning', self.help, inner_indent) | |
1964 # The default flag value can either be represented as a string like on the | |
1965 # command line, or as a Python object. We serialize this value in the | |
1966 # latter case in order to remain consistent. | |
1967 if self.serializer and not isinstance(self.default, str): | |
1968 default_serialized = self.serializer.Serialize(self.default) | |
1969 else: | |
1970 default_serialized = self.default | |
1971 _WriteSimpleXMLElement(outfile, 'default', default_serialized, inner_indent) | |
1972 _WriteSimpleXMLElement(outfile, 'current', self.value, inner_indent) | |
1973 _WriteSimpleXMLElement(outfile, 'type', self.Type(), inner_indent) | |
1974 # Print extra flag features this flag may have. | |
1975 self._WriteCustomInfoInXMLFormat(outfile, inner_indent) | |
1976 outfile.write(indent + '</flag>\n') | |
1977 | |
1978 def _WriteCustomInfoInXMLFormat(self, outfile, indent): | |
1979 """Writes extra info about this flag, in XML format. | |
1980 | |
1981 "Extra" means "not already printed by WriteInfoInXMLFormat above." | |
1982 | |
1983 Args: | |
1984 outfile: File object we write to. | |
1985 indent: A string that is prepended to each generated line. | |
1986 """ | |
1987 # Usually, the parser knows the extra details about the flag, so | |
1988 # we just forward the call to it. | |
1989 self.parser.WriteCustomInfoInXMLFormat(outfile, indent) | |
1990 # End of Flag definition | |
1991 | |
1992 | |
1993 class _ArgumentParserCache(type): | |
1994 """Metaclass used to cache and share argument parsers among flags.""" | |
1995 | |
1996 _instances = {} | |
1997 | |
1998 def __call__(mcs, *args, **kwargs): | |
1999 """Returns an instance of the argument parser cls. | |
2000 | |
2001 This method overrides behavior of the __new__ methods in | |
2002 all subclasses of ArgumentParser (inclusive). If an instance | |
2003 for mcs with the same set of arguments exists, this instance is | |
2004 returned, otherwise a new instance is created. | |
2005 | |
2006 If any keyword arguments are defined, or the values in args | |
2007 are not hashable, this method always returns a new instance of | |
2008 cls. | |
2009 | |
2010 Args: | |
2011 args: Positional initializer arguments. | |
2012 kwargs: Initializer keyword arguments. | |
2013 | |
2014 Returns: | |
2015 An instance of cls, shared or new. | |
2016 """ | |
2017 if kwargs: | |
2018 return type.__call__(mcs, *args, **kwargs) | |
2019 else: | |
2020 instances = mcs._instances | |
2021 key = (mcs,) + tuple(args) | |
2022 try: | |
2023 return instances[key] | |
2024 except KeyError: | |
2025 # No cache entry for key exists, create a new one. | |
2026 return instances.setdefault(key, type.__call__(mcs, *args)) | |
2027 except TypeError: | |
2028 # An object in args cannot be hashed, always return | |
2029 # a new instance. | |
2030 return type.__call__(mcs, *args) | |
2031 | |
2032 | |
2033 class ArgumentParser(object): | |
2034 """Base class used to parse and convert arguments. | |
2035 | |
2036 The Parse() method checks to make sure that the string argument is a | |
2037 legal value and convert it to a native type. If the value cannot be | |
2038 converted, it should throw a 'ValueError' exception with a human | |
2039 readable explanation of why the value is illegal. | |
2040 | |
2041 Subclasses should also define a syntactic_help string which may be | |
2042 presented to the user to describe the form of the legal values. | |
2043 | |
2044 Argument parser classes must be stateless, since instances are cached | |
2045 and shared between flags. Initializer arguments are allowed, but all | |
2046 member variables must be derived from initializer arguments only. | |
2047 """ | |
2048 __metaclass__ = _ArgumentParserCache | |
2049 | |
2050 syntactic_help = "" | |
2051 | |
2052 def Parse(self, argument): | |
2053 """Default implementation: always returns its argument unmodified.""" | |
2054 return argument | |
2055 | |
2056 def Type(self): | |
2057 return 'string' | |
2058 | |
2059 def WriteCustomInfoInXMLFormat(self, outfile, indent): | |
2060 pass | |
2061 | |
2062 | |
2063 class ArgumentSerializer: | |
2064 """Base class for generating string representations of a flag value.""" | |
2065 | |
2066 def Serialize(self, value): | |
2067 return _StrOrUnicode(value) | |
2068 | |
2069 | |
2070 class ListSerializer(ArgumentSerializer): | |
2071 | |
2072 def __init__(self, list_sep): | |
2073 self.list_sep = list_sep | |
2074 | |
2075 def Serialize(self, value): | |
2076 return self.list_sep.join([_StrOrUnicode(x) for x in value]) | |
2077 | |
2078 | |
2079 # Flags validators | |
2080 | |
2081 | |
2082 def RegisterValidator(flag_name, | |
2083 checker, | |
2084 message='Flag validation failed', | |
2085 flag_values=FLAGS): | |
2086 """Adds a constraint, which will be enforced during program execution. | |
2087 | |
2088 The constraint is validated when flags are initially parsed, and after each | |
2089 change of the corresponding flag's value. | |
2090 Args: | |
2091 flag_name: string, name of the flag to be checked. | |
2092 checker: method to validate the flag. | |
2093 input - value of the corresponding flag (string, boolean, etc. | |
2094 This value will be passed to checker by the library). See file's | |
2095 docstring for examples. | |
2096 output - Boolean. | |
2097 Must return True if validator constraint is satisfied. | |
2098 If constraint is not satisfied, it should either return False or | |
2099 raise gflags_validators.Error(desired_error_message). | |
2100 message: error text to be shown to the user if checker returns False. | |
2101 If checker raises gflags_validators.Error, message from the raised | |
2102 Error will be shown. | |
2103 flag_values: FlagValues | |
2104 Raises: | |
2105 AttributeError: if flag_name is not registered as a valid flag name. | |
2106 """ | |
2107 flag_values.AddValidator(gflags_validators.SimpleValidator(flag_name, | |
2108 checker, | |
2109 message)) | |
2110 | |
2111 | |
2112 def MarkFlagAsRequired(flag_name, flag_values=FLAGS): | |
2113 """Ensure that flag is not None during program execution. | |
2114 | |
2115 Registers a flag validator, which will follow usual validator | |
2116 rules. | |
2117 Args: | |
2118 flag_name: string, name of the flag | |
2119 flag_values: FlagValues | |
2120 Raises: | |
2121 AttributeError: if flag_name is not registered as a valid flag name. | |
2122 """ | |
2123 RegisterValidator(flag_name, | |
2124 lambda value: value is not None, | |
2125 message='Flag --%s must be specified.' % flag_name, | |
2126 flag_values=flag_values) | |
2127 | |
2128 | |
2129 def _RegisterBoundsValidatorIfNeeded(parser, name, flag_values): | |
2130 """Enforce lower and upper bounds for numeric flags. | |
2131 | |
2132 Args: | |
2133 parser: NumericParser (either FloatParser or IntegerParser). Provides lower | |
2134 and upper bounds, and help text to display. | |
2135 name: string, name of the flag | |
2136 flag_values: FlagValues | |
2137 """ | |
2138 if parser.lower_bound is not None or parser.upper_bound is not None: | |
2139 | |
2140 def Checker(value): | |
2141 if value is not None and parser.IsOutsideBounds(value): | |
2142 message = '%s is not %s' % (value, parser.syntactic_help) | |
2143 raise gflags_validators.Error(message) | |
2144 return True | |
2145 | |
2146 RegisterValidator(name, | |
2147 Checker, | |
2148 flag_values=flag_values) | |
2149 | |
2150 | |
2151 # The DEFINE functions are explained in mode details in the module doc string. | |
2152 | |
2153 | |
2154 def DEFINE(parser, name, default, help, flag_values=FLAGS, serializer=None, | |
2155 **args): | |
2156 """Registers a generic Flag object. | |
2157 | |
2158 NOTE: in the docstrings of all DEFINE* functions, "registers" is short | |
2159 for "creates a new flag and registers it". | |
2160 | |
2161 Auxiliary function: clients should use the specialized DEFINE_<type> | |
2162 function instead. | |
2163 | |
2164 Args: | |
2165 parser: ArgumentParser that is used to parse the flag arguments. | |
2166 name: A string, the flag name. | |
2167 default: The default value of the flag. | |
2168 help: A help string. | |
2169 flag_values: FlagValues object the flag will be registered with. | |
2170 serializer: ArgumentSerializer that serializes the flag value. | |
2171 args: Dictionary with extra keyword args that are passes to the | |
2172 Flag __init__. | |
2173 """ | |
2174 DEFINE_flag(Flag(parser, serializer, name, default, help, **args), | |
2175 flag_values) | |
2176 | |
2177 | |
2178 def DEFINE_flag(flag, flag_values=FLAGS): | |
2179 """Registers a 'Flag' object with a 'FlagValues' object. | |
2180 | |
2181 By default, the global FLAGS 'FlagValue' object is used. | |
2182 | |
2183 Typical users will use one of the more specialized DEFINE_xxx | |
2184 functions, such as DEFINE_string or DEFINE_integer. But developers | |
2185 who need to create Flag objects themselves should use this function | |
2186 to register their flags. | |
2187 """ | |
2188 # copying the reference to flag_values prevents pychecker warnings | |
2189 fv = flag_values | |
2190 fv[flag.name] = flag | |
2191 # Tell flag_values who's defining the flag. | |
2192 if isinstance(flag_values, FlagValues): | |
2193 # Regarding the above isinstance test: some users pass funny | |
2194 # values of flag_values (e.g., {}) in order to avoid the flag | |
2195 # registration (in the past, there used to be a flag_values == | |
2196 # FLAGS test here) and redefine flags with the same name (e.g., | |
2197 # debug). To avoid breaking their code, we perform the | |
2198 # registration only if flag_values is a real FlagValues object. | |
2199 module, module_name = _GetCallingModuleObjectAndName() | |
2200 flag_values._RegisterFlagByModule(module_name, flag) | |
2201 flag_values._RegisterFlagByModuleId(id(module), flag) | |
2202 | |
2203 | |
2204 def _InternalDeclareKeyFlags(flag_names, | |
2205 flag_values=FLAGS, key_flag_values=None): | |
2206 """Declares a flag as key for the calling module. | |
2207 | |
2208 Internal function. User code should call DECLARE_key_flag or | |
2209 ADOPT_module_key_flags instead. | |
2210 | |
2211 Args: | |
2212 flag_names: A list of strings that are names of already-registered | |
2213 Flag objects. | |
2214 flag_values: A FlagValues object that the flags listed in | |
2215 flag_names have registered with (the value of the flag_values | |
2216 argument from the DEFINE_* calls that defined those flags). | |
2217 This should almost never need to be overridden. | |
2218 key_flag_values: A FlagValues object that (among possibly many | |
2219 other things) keeps track of the key flags for each module. | |
2220 Default None means "same as flag_values". This should almost | |
2221 never need to be overridden. | |
2222 | |
2223 Raises: | |
2224 UnrecognizedFlagError: when we refer to a flag that was not | |
2225 defined yet. | |
2226 """ | |
2227 key_flag_values = key_flag_values or flag_values | |
2228 | |
2229 module = _GetCallingModule() | |
2230 | |
2231 for flag_name in flag_names: | |
2232 if flag_name not in flag_values: | |
2233 raise UnrecognizedFlagError(flag_name) | |
2234 flag = flag_values.FlagDict()[flag_name] | |
2235 key_flag_values._RegisterKeyFlagForModule(module, flag) | |
2236 | |
2237 | |
2238 def DECLARE_key_flag(flag_name, flag_values=FLAGS): | |
2239 """Declares one flag as key to the current module. | |
2240 | |
2241 Key flags are flags that are deemed really important for a module. | |
2242 They are important when listing help messages; e.g., if the | |
2243 --helpshort command-line flag is used, then only the key flags of the | |
2244 main module are listed (instead of all flags, as in the case of | |
2245 --help). | |
2246 | |
2247 Sample usage: | |
2248 | |
2249 gflags.DECLARED_key_flag('flag_1') | |
2250 | |
2251 Args: | |
2252 flag_name: A string, the name of an already declared flag. | |
2253 (Redeclaring flags as key, including flags implicitly key | |
2254 because they were declared in this module, is a no-op.) | |
2255 flag_values: A FlagValues object. This should almost never | |
2256 need to be overridden. | |
2257 """ | |
2258 if flag_name in _SPECIAL_FLAGS: | |
2259 # Take care of the special flags, e.g., --flagfile, --undefok. | |
2260 # These flags are defined in _SPECIAL_FLAGS, and are treated | |
2261 # specially during flag parsing, taking precedence over the | |
2262 # user-defined flags. | |
2263 _InternalDeclareKeyFlags([flag_name], | |
2264 flag_values=_SPECIAL_FLAGS, | |
2265 key_flag_values=flag_values) | |
2266 return | |
2267 _InternalDeclareKeyFlags([flag_name], flag_values=flag_values) | |
2268 | |
2269 | |
2270 def ADOPT_module_key_flags(module, flag_values=FLAGS): | |
2271 """Declares that all flags key to a module are key to the current module. | |
2272 | |
2273 Args: | |
2274 module: A module object. | |
2275 flag_values: A FlagValues object. This should almost never need | |
2276 to be overridden. | |
2277 | |
2278 Raises: | |
2279 FlagsError: When given an argument that is a module name (a | |
2280 string), instead of a module object. | |
2281 """ | |
2282 # NOTE(salcianu): an even better test would be if not | |
2283 # isinstance(module, types.ModuleType) but I didn't want to import | |
2284 # types for such a tiny use. | |
2285 if isinstance(module, str): | |
2286 raise FlagsError('Received module name %s; expected a module object.' | |
2287 % module) | |
2288 _InternalDeclareKeyFlags( | |
2289 [f.name for f in flag_values._GetKeyFlagsForModule(module.__name__)], | |
2290 flag_values=flag_values) | |
2291 # If module is this flag module, take _SPECIAL_FLAGS into account. | |
2292 if module == _GetThisModuleObjectAndName()[0]: | |
2293 _InternalDeclareKeyFlags( | |
2294 # As we associate flags with _GetCallingModuleObjectAndName(), the | |
2295 # special flags defined in this module are incorrectly registered with | |
2296 # a different module. So, we can't use _GetKeyFlagsForModule. | |
2297 # Instead, we take all flags from _SPECIAL_FLAGS (a private | |
2298 # FlagValues, where no other module should register flags). | |
2299 [f.name for f in _SPECIAL_FLAGS.FlagDict().values()], | |
2300 flag_values=_SPECIAL_FLAGS, | |
2301 key_flag_values=flag_values) | |
2302 | |
2303 | |
2304 # | |
2305 # STRING FLAGS | |
2306 # | |
2307 | |
2308 | |
2309 def DEFINE_string(name, default, help, flag_values=FLAGS, **args): | |
2310 """Registers a flag whose value can be any string.""" | |
2311 parser = ArgumentParser() | |
2312 serializer = ArgumentSerializer() | |
2313 DEFINE(parser, name, default, help, flag_values, serializer, **args) | |
2314 | |
2315 | |
2316 # | |
2317 # BOOLEAN FLAGS | |
2318 # | |
2319 | |
2320 | |
2321 class BooleanParser(ArgumentParser): | |
2322 """Parser of boolean values.""" | |
2323 | |
2324 def Convert(self, argument): | |
2325 """Converts the argument to a boolean; raise ValueError on errors.""" | |
2326 if type(argument) == str: | |
2327 if argument.lower() in ['true', 't', '1']: | |
2328 return True | |
2329 elif argument.lower() in ['false', 'f', '0']: | |
2330 return False | |
2331 | |
2332 bool_argument = bool(argument) | |
2333 if argument == bool_argument: | |
2334 # The argument is a valid boolean (True, False, 0, or 1), and not just | |
2335 # something that always converts to bool (list, string, int, etc.). | |
2336 return bool_argument | |
2337 | |
2338 raise ValueError('Non-boolean argument to boolean flag', argument) | |
2339 | |
2340 def Parse(self, argument): | |
2341 val = self.Convert(argument) | |
2342 return val | |
2343 | |
2344 def Type(self): | |
2345 return 'bool' | |
2346 | |
2347 | |
2348 class BooleanFlag(Flag): | |
2349 """Basic boolean flag. | |
2350 | |
2351 Boolean flags do not take any arguments, and their value is either | |
2352 True (1) or False (0). The false value is specified on the command | |
2353 line by prepending the word 'no' to either the long or the short flag | |
2354 name. | |
2355 | |
2356 For example, if a Boolean flag was created whose long name was | |
2357 'update' and whose short name was 'x', then this flag could be | |
2358 explicitly unset through either --noupdate or --nox. | |
2359 """ | |
2360 | |
2361 def __init__(self, name, default, help, short_name=None, **args): | |
2362 p = BooleanParser() | |
2363 Flag.__init__(self, p, None, name, default, help, short_name, 1, **args) | |
2364 if not self.help: self.help = "a boolean value" | |
2365 | |
2366 | |
2367 def DEFINE_boolean(name, default, help, flag_values=FLAGS, **args): | |
2368 """Registers a boolean flag. | |
2369 | |
2370 Such a boolean flag does not take an argument. If a user wants to | |
2371 specify a false value explicitly, the long option beginning with 'no' | |
2372 must be used: i.e. --noflag | |
2373 | |
2374 This flag will have a value of None, True or False. None is possible | |
2375 if default=None and the user does not specify the flag on the command | |
2376 line. | |
2377 """ | |
2378 DEFINE_flag(BooleanFlag(name, default, help, **args), flag_values) | |
2379 | |
2380 | |
2381 # Match C++ API to unconfuse C++ people. | |
2382 DEFINE_bool = DEFINE_boolean | |
2383 | |
2384 | |
2385 class HelpFlag(BooleanFlag): | |
2386 """ | |
2387 HelpFlag is a special boolean flag that prints usage information and | |
2388 raises a SystemExit exception if it is ever found in the command | |
2389 line arguments. Note this is called with allow_override=1, so other | |
2390 apps can define their own --help flag, replacing this one, if they want. | |
2391 """ | |
2392 def __init__(self): | |
2393 BooleanFlag.__init__(self, "help", 0, "show this help", | |
2394 short_name="?", allow_override=1) | |
2395 def Parse(self, arg): | |
2396 if arg: | |
2397 doc = sys.modules["__main__"].__doc__ | |
2398 flags = str(FLAGS) | |
2399 print doc or ("\nUSAGE: %s [flags]\n" % sys.argv[0]) | |
2400 if flags: | |
2401 print "flags:" | |
2402 print flags | |
2403 sys.exit(1) | |
2404 class HelpXMLFlag(BooleanFlag): | |
2405 """Similar to HelpFlag, but generates output in XML format.""" | |
2406 def __init__(self): | |
2407 BooleanFlag.__init__(self, 'helpxml', False, | |
2408 'like --help, but generates XML output', | |
2409 allow_override=1) | |
2410 def Parse(self, arg): | |
2411 if arg: | |
2412 FLAGS.WriteHelpInXMLFormat(sys.stdout) | |
2413 sys.exit(1) | |
2414 class HelpshortFlag(BooleanFlag): | |
2415 """ | |
2416 HelpshortFlag is a special boolean flag that prints usage | |
2417 information for the "main" module, and rasies a SystemExit exception | |
2418 if it is ever found in the command line arguments. Note this is | |
2419 called with allow_override=1, so other apps can define their own | |
2420 --helpshort flag, replacing this one, if they want. | |
2421 """ | |
2422 def __init__(self): | |
2423 BooleanFlag.__init__(self, "helpshort", 0, | |
2424 "show usage only for this module", allow_override=1) | |
2425 def Parse(self, arg): | |
2426 if arg: | |
2427 doc = sys.modules["__main__"].__doc__ | |
2428 flags = FLAGS.MainModuleHelp() | |
2429 print doc or ("\nUSAGE: %s [flags]\n" % sys.argv[0]) | |
2430 if flags: | |
2431 print "flags:" | |
2432 print flags | |
2433 sys.exit(1) | |
2434 | |
2435 # | |
2436 # Numeric parser - base class for Integer and Float parsers | |
2437 # | |
2438 | |
2439 | |
2440 class NumericParser(ArgumentParser): | |
2441 """Parser of numeric values. | |
2442 | |
2443 Parsed value may be bounded to a given upper and lower bound. | |
2444 """ | |
2445 | |
2446 def IsOutsideBounds(self, val): | |
2447 return ((self.lower_bound is not None and val < self.lower_bound) or | |
2448 (self.upper_bound is not None and val > self.upper_bound)) | |
2449 | |
2450 def Parse(self, argument): | |
2451 val = self.Convert(argument) | |
2452 if self.IsOutsideBounds(val): | |
2453 raise ValueError("%s is not %s" % (val, self.syntactic_help)) | |
2454 return val | |
2455 | |
2456 def WriteCustomInfoInXMLFormat(self, outfile, indent): | |
2457 if self.lower_bound is not None: | |
2458 _WriteSimpleXMLElement(outfile, 'lower_bound', self.lower_bound, indent) | |
2459 if self.upper_bound is not None: | |
2460 _WriteSimpleXMLElement(outfile, 'upper_bound', self.upper_bound, indent) | |
2461 | |
2462 def Convert(self, argument): | |
2463 """Default implementation: always returns its argument unmodified.""" | |
2464 return argument | |
2465 | |
2466 # End of Numeric Parser | |
2467 | |
2468 # | |
2469 # FLOAT FLAGS | |
2470 # | |
2471 | |
2472 | |
2473 class FloatParser(NumericParser): | |
2474 """Parser of floating point values. | |
2475 | |
2476 Parsed value may be bounded to a given upper and lower bound. | |
2477 """ | |
2478 number_article = "a" | |
2479 number_name = "number" | |
2480 syntactic_help = " ".join((number_article, number_name)) | |
2481 | |
2482 def __init__(self, lower_bound=None, upper_bound=None): | |
2483 super(FloatParser, self).__init__() | |
2484 self.lower_bound = lower_bound | |
2485 self.upper_bound = upper_bound | |
2486 sh = self.syntactic_help | |
2487 if lower_bound is not None and upper_bound is not None: | |
2488 sh = ("%s in the range [%s, %s]" % (sh, lower_bound, upper_bound)) | |
2489 elif lower_bound == 0: | |
2490 sh = "a non-negative %s" % self.number_name | |
2491 elif upper_bound == 0: | |
2492 sh = "a non-positive %s" % self.number_name | |
2493 elif upper_bound is not None: | |
2494 sh = "%s <= %s" % (self.number_name, upper_bound) | |
2495 elif lower_bound is not None: | |
2496 sh = "%s >= %s" % (self.number_name, lower_bound) | |
2497 self.syntactic_help = sh | |
2498 | |
2499 def Convert(self, argument): | |
2500 """Converts argument to a float; raises ValueError on errors.""" | |
2501 return float(argument) | |
2502 | |
2503 def Type(self): | |
2504 return 'float' | |
2505 # End of FloatParser | |
2506 | |
2507 | |
2508 def DEFINE_float(name, default, help, lower_bound=None, upper_bound=None, | |
2509 flag_values=FLAGS, **args): | |
2510 """Registers a flag whose value must be a float. | |
2511 | |
2512 If lower_bound or upper_bound are set, then this flag must be | |
2513 within the given range. | |
2514 """ | |
2515 parser = FloatParser(lower_bound, upper_bound) | |
2516 serializer = ArgumentSerializer() | |
2517 DEFINE(parser, name, default, help, flag_values, serializer, **args) | |
2518 _RegisterBoundsValidatorIfNeeded(parser, name, flag_values=flag_values) | |
2519 | |
2520 # | |
2521 # INTEGER FLAGS | |
2522 # | |
2523 | |
2524 | |
2525 class IntegerParser(NumericParser): | |
2526 """Parser of an integer value. | |
2527 | |
2528 Parsed value may be bounded to a given upper and lower bound. | |
2529 """ | |
2530 number_article = "an" | |
2531 number_name = "integer" | |
2532 syntactic_help = " ".join((number_article, number_name)) | |
2533 | |
2534 def __init__(self, lower_bound=None, upper_bound=None): | |
2535 super(IntegerParser, self).__init__() | |
2536 self.lower_bound = lower_bound | |
2537 self.upper_bound = upper_bound | |
2538 sh = self.syntactic_help | |
2539 if lower_bound is not None and upper_bound is not None: | |
2540 sh = ("%s in the range [%s, %s]" % (sh, lower_bound, upper_bound)) | |
2541 elif lower_bound == 1: | |
2542 sh = "a positive %s" % self.number_name | |
2543 elif upper_bound == -1: | |
2544 sh = "a negative %s" % self.number_name | |
2545 elif lower_bound == 0: | |
2546 sh = "a non-negative %s" % self.number_name | |
2547 elif upper_bound == 0: | |
2548 sh = "a non-positive %s" % self.number_name | |
2549 elif upper_bound is not None: | |
2550 sh = "%s <= %s" % (self.number_name, upper_bound) | |
2551 elif lower_bound is not None: | |
2552 sh = "%s >= %s" % (self.number_name, lower_bound) | |
2553 self.syntactic_help = sh | |
2554 | |
2555 def Convert(self, argument): | |
2556 __pychecker__ = 'no-returnvalues' | |
2557 if type(argument) == str: | |
2558 base = 10 | |
2559 if len(argument) > 2 and argument[0] == "0" and argument[1] == "x": | |
2560 base = 16 | |
2561 return int(argument, base) | |
2562 else: | |
2563 return int(argument) | |
2564 | |
2565 def Type(self): | |
2566 return 'int' | |
2567 | |
2568 | |
2569 def DEFINE_integer(name, default, help, lower_bound=None, upper_bound=None, | |
2570 flag_values=FLAGS, **args): | |
2571 """Registers a flag whose value must be an integer. | |
2572 | |
2573 If lower_bound, or upper_bound are set, then this flag must be | |
2574 within the given range. | |
2575 """ | |
2576 parser = IntegerParser(lower_bound, upper_bound) | |
2577 serializer = ArgumentSerializer() | |
2578 DEFINE(parser, name, default, help, flag_values, serializer, **args) | |
2579 _RegisterBoundsValidatorIfNeeded(parser, name, flag_values=flag_values) | |
2580 | |
2581 | |
2582 # | |
2583 # ENUM FLAGS | |
2584 # | |
2585 | |
2586 | |
2587 class EnumParser(ArgumentParser): | |
2588 """Parser of a string enum value (a string value from a given set). | |
2589 | |
2590 If enum_values (see below) is not specified, any string is allowed. | |
2591 """ | |
2592 | |
2593 def __init__(self, enum_values=None): | |
2594 super(EnumParser, self).__init__() | |
2595 self.enum_values = enum_values | |
2596 | |
2597 def Parse(self, argument): | |
2598 if self.enum_values and argument not in self.enum_values: | |
2599 raise ValueError("value should be one of <%s>" % | |
2600 "|".join(self.enum_values)) | |
2601 return argument | |
2602 | |
2603 def Type(self): | |
2604 return 'string enum' | |
2605 | |
2606 | |
2607 class EnumFlag(Flag): | |
2608 """Basic enum flag; its value can be any string from list of enum_values.""" | |
2609 | |
2610 def __init__(self, name, default, help, enum_values=None, | |
2611 short_name=None, **args): | |
2612 enum_values = enum_values or [] | |
2613 p = EnumParser(enum_values) | |
2614 g = ArgumentSerializer() | |
2615 Flag.__init__(self, p, g, name, default, help, short_name, **args) | |
2616 if not self.help: self.help = "an enum string" | |
2617 self.help = "<%s>: %s" % ("|".join(enum_values), self.help) | |
2618 | |
2619 def _WriteCustomInfoInXMLFormat(self, outfile, indent): | |
2620 for enum_value in self.parser.enum_values: | |
2621 _WriteSimpleXMLElement(outfile, 'enum_value', enum_value, indent) | |
2622 | |
2623 | |
2624 def DEFINE_enum(name, default, enum_values, help, flag_values=FLAGS, | |
2625 **args): | |
2626 """Registers a flag whose value can be any string from enum_values.""" | |
2627 DEFINE_flag(EnumFlag(name, default, help, enum_values, ** args), | |
2628 flag_values) | |
2629 | |
2630 | |
2631 # | |
2632 # LIST FLAGS | |
2633 # | |
2634 | |
2635 | |
2636 class BaseListParser(ArgumentParser): | |
2637 """Base class for a parser of lists of strings. | |
2638 | |
2639 To extend, inherit from this class; from the subclass __init__, call | |
2640 | |
2641 BaseListParser.__init__(self, token, name) | |
2642 | |
2643 where token is a character used to tokenize, and name is a description | |
2644 of the separator. | |
2645 """ | |
2646 | |
2647 def __init__(self, token=None, name=None): | |
2648 assert name | |
2649 super(BaseListParser, self).__init__() | |
2650 self._token = token | |
2651 self._name = name | |
2652 self.syntactic_help = "a %s separated list" % self._name | |
2653 | |
2654 def Parse(self, argument): | |
2655 if isinstance(argument, list): | |
2656 return argument | |
2657 elif argument == '': | |
2658 return [] | |
2659 else: | |
2660 return [s.strip() for s in argument.split(self._token)] | |
2661 | |
2662 def Type(self): | |
2663 return '%s separated list of strings' % self._name | |
2664 | |
2665 | |
2666 class ListParser(BaseListParser): | |
2667 """Parser for a comma-separated list of strings.""" | |
2668 | |
2669 def __init__(self): | |
2670 BaseListParser.__init__(self, ',', 'comma') | |
2671 | |
2672 def WriteCustomInfoInXMLFormat(self, outfile, indent): | |
2673 BaseListParser.WriteCustomInfoInXMLFormat(self, outfile, indent) | |
2674 _WriteSimpleXMLElement(outfile, 'list_separator', repr(','), indent) | |
2675 | |
2676 | |
2677 class WhitespaceSeparatedListParser(BaseListParser): | |
2678 """Parser for a whitespace-separated list of strings.""" | |
2679 | |
2680 def __init__(self): | |
2681 BaseListParser.__init__(self, None, 'whitespace') | |
2682 | |
2683 def WriteCustomInfoInXMLFormat(self, outfile, indent): | |
2684 BaseListParser.WriteCustomInfoInXMLFormat(self, outfile, indent) | |
2685 separators = list(string.whitespace) | |
2686 separators.sort() | |
2687 for ws_char in string.whitespace: | |
2688 _WriteSimpleXMLElement(outfile, 'list_separator', repr(ws_char), indent) | |
2689 | |
2690 | |
2691 def DEFINE_list(name, default, help, flag_values=FLAGS, **args): | |
2692 """Registers a flag whose value is a comma-separated list of strings.""" | |
2693 parser = ListParser() | |
2694 serializer = ListSerializer(',') | |
2695 DEFINE(parser, name, default, help, flag_values, serializer, **args) | |
2696 | |
2697 | |
2698 def DEFINE_spaceseplist(name, default, help, flag_values=FLAGS, **args): | |
2699 """Registers a flag whose value is a whitespace-separated list of strings. | |
2700 | |
2701 Any whitespace can be used as a separator. | |
2702 """ | |
2703 parser = WhitespaceSeparatedListParser() | |
2704 serializer = ListSerializer(' ') | |
2705 DEFINE(parser, name, default, help, flag_values, serializer, **args) | |
2706 | |
2707 | |
2708 # | |
2709 # MULTI FLAGS | |
2710 # | |
2711 | |
2712 | |
2713 class MultiFlag(Flag): | |
2714 """A flag that can appear multiple time on the command-line. | |
2715 | |
2716 The value of such a flag is a list that contains the individual values | |
2717 from all the appearances of that flag on the command-line. | |
2718 | |
2719 See the __doc__ for Flag for most behavior of this class. Only | |
2720 differences in behavior are described here: | |
2721 | |
2722 * The default value may be either a single value or a list of values. | |
2723 A single value is interpreted as the [value] singleton list. | |
2724 | |
2725 * The value of the flag is always a list, even if the option was | |
2726 only supplied once, and even if the default value is a single | |
2727 value | |
2728 """ | |
2729 | |
2730 def __init__(self, *args, **kwargs): | |
2731 Flag.__init__(self, *args, **kwargs) | |
2732 self.help += ';\n repeat this option to specify a list of values' | |
2733 | |
2734 def Parse(self, arguments): | |
2735 """Parses one or more arguments with the installed parser. | |
2736 | |
2737 Args: | |
2738 arguments: a single argument or a list of arguments (typically a | |
2739 list of default values); a single argument is converted | |
2740 internally into a list containing one item. | |
2741 """ | |
2742 if not isinstance(arguments, list): | |
2743 # Default value may be a list of values. Most other arguments | |
2744 # will not be, so convert them into a single-item list to make | |
2745 # processing simpler below. | |
2746 arguments = [arguments] | |
2747 | |
2748 if self.present: | |
2749 # keep a backup reference to list of previously supplied option values | |
2750 values = self.value | |
2751 else: | |
2752 # "erase" the defaults with an empty list | |
2753 values = [] | |
2754 | |
2755 for item in arguments: | |
2756 # have Flag superclass parse argument, overwriting self.value reference | |
2757 Flag.Parse(self, item) # also increments self.present | |
2758 values.append(self.value) | |
2759 | |
2760 # put list of option values back in the 'value' attribute | |
2761 self.value = values | |
2762 | |
2763 def Serialize(self): | |
2764 if not self.serializer: | |
2765 raise FlagsError("Serializer not present for flag %s" % self.name) | |
2766 if self.value is None: | |
2767 return '' | |
2768 | |
2769 s = '' | |
2770 | |
2771 multi_value = self.value | |
2772 | |
2773 for self.value in multi_value: | |
2774 if s: s += ' ' | |
2775 s += Flag.Serialize(self) | |
2776 | |
2777 self.value = multi_value | |
2778 | |
2779 return s | |
2780 | |
2781 def Type(self): | |
2782 return 'multi ' + self.parser.Type() | |
2783 | |
2784 | |
2785 def DEFINE_multi(parser, serializer, name, default, help, flag_values=FLAGS, | |
2786 **args): | |
2787 """Registers a generic MultiFlag that parses its args with a given parser. | |
2788 | |
2789 Auxiliary function. Normal users should NOT use it directly. | |
2790 | |
2791 Developers who need to create their own 'Parser' classes for options | |
2792 which can appear multiple times can call this module function to | |
2793 register their flags. | |
2794 """ | |
2795 DEFINE_flag(MultiFlag(parser, serializer, name, default, help, **args), | |
2796 flag_values) | |
2797 | |
2798 | |
2799 def DEFINE_multistring(name, default, help, flag_values=FLAGS, **args): | |
2800 """Registers a flag whose value can be a list of any strings. | |
2801 | |
2802 Use the flag on the command line multiple times to place multiple | |
2803 string values into the list. The 'default' may be a single string | |
2804 (which will be converted into a single-element list) or a list of | |
2805 strings. | |
2806 """ | |
2807 parser = ArgumentParser() | |
2808 serializer = ArgumentSerializer() | |
2809 DEFINE_multi(parser, serializer, name, default, help, flag_values, **args) | |
2810 | |
2811 | |
2812 def DEFINE_multi_int(name, default, help, lower_bound=None, upper_bound=None, | |
2813 flag_values=FLAGS, **args): | |
2814 """Registers a flag whose value can be a list of arbitrary integers. | |
2815 | |
2816 Use the flag on the command line multiple times to place multiple | |
2817 integer values into the list. The 'default' may be a single integer | |
2818 (which will be converted into a single-element list) or a list of | |
2819 integers. | |
2820 """ | |
2821 parser = IntegerParser(lower_bound, upper_bound) | |
2822 serializer = ArgumentSerializer() | |
2823 DEFINE_multi(parser, serializer, name, default, help, flag_values, **args) | |
2824 | |
2825 | |
2826 def DEFINE_multi_float(name, default, help, lower_bound=None, upper_bound=None, | |
2827 flag_values=FLAGS, **args): | |
2828 """Registers a flag whose value can be a list of arbitrary floats. | |
2829 | |
2830 Use the flag on the command line multiple times to place multiple | |
2831 float values into the list. The 'default' may be a single float | |
2832 (which will be converted into a single-element list) or a list of | |
2833 floats. | |
2834 """ | |
2835 parser = FloatParser(lower_bound, upper_bound) | |
2836 serializer = ArgumentSerializer() | |
2837 DEFINE_multi(parser, serializer, name, default, help, flag_values, **args) | |
2838 | |
2839 | |
2840 # Now register the flags that we want to exist in all applications. | |
2841 # These are all defined with allow_override=1, so user-apps can use | |
2842 # these flagnames for their own purposes, if they want. | |
2843 DEFINE_flag(HelpFlag()) | |
2844 DEFINE_flag(HelpshortFlag()) | |
2845 DEFINE_flag(HelpXMLFlag()) | |
2846 | |
2847 # Define special flags here so that help may be generated for them. | |
2848 # NOTE: Please do NOT use _SPECIAL_FLAGS from outside this module. | |
2849 _SPECIAL_FLAGS = FlagValues() | |
2850 | |
2851 | |
2852 DEFINE_string( | |
2853 'flagfile', "", | |
2854 "Insert flag definitions from the given file into the command line.", | |
2855 _SPECIAL_FLAGS) | |
2856 | |
2857 DEFINE_string( | |
2858 'undefok', "", | |
2859 "comma-separated list of flag names that it is okay to specify " | |
2860 "on the command line even if the program does not define a flag " | |
2861 "with that name. IMPORTANT: flags in this list that have " | |
2862 "arguments MUST use the --flag=value format.", _SPECIAL_FLAGS) | |
OLD | NEW |