Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(215)

Side by Side Diff: lib/operation.py

Issue 6626039: Revert "Plumb in crprocess instead of RunCommand to allow quiet operation." (Closed) Base URL: http://git.chromium.org/git/chromite.git@master
Patch Set: Created 9 years, 9 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « lib/cros_subprocess.py ('k') | shell/chromite_env.py » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Copyright (c) 2011 The Chromium OS Authors. All rights reserved. 1 # Copyright (c) 2011 The Chromium OS Authors. All rights reserved.
2 # Use of this source code is governed by a BSD-style license that can be 2 # Use of this source code is governed by a BSD-style license that can be
3 # found in the LICENSE file. 3 # found in the LICENSE file.
4 4
5 """Operation, including output and progress display 5 """Operation, including output and progress display
6 6
7 This module implements the concept of an operation, which has regular progress 7 This module implements the concept of an operation, which has regular progress
8 updates, verbose text display and perhaps some errors. 8 updates, verbose text display and perhaps some errors.
9 """ 9 """
10 10
(...skipping 15 matching lines...) Expand all
26 it looking for errors and progress information. Optionally it can output the 26 it looking for errors and progress information. Optionally it can output the
27 stderr and stdout to the terminal, but it is normally supressed. 27 stderr and stdout to the terminal, but it is normally supressed.
28 28
29 Progress information is garnered from the subprocess output based on 29 Progress information is garnered from the subprocess output based on
30 knowledge of the legacy scripts, but at some point will move over to using 30 knowledge of the legacy scripts, but at some point will move over to using
31 real progress information reported through new python methods which will 31 real progress information reported through new python methods which will
32 replace the scripts. 32 replace the scripts.
33 33
34 Each operation has a name, and this class handles displaying this name 34 Each operation has a name, and this class handles displaying this name
35 as it reports progress. 35 as it reports progress.
36
37
38 Operation Objects
39 =================
40
41 verbose: True / False
42 In verbose mode all output from subprocesses is displayed, otherwise
43 this output is normally supressed, unless we think it indicates an error.
44
45 progress: True / False
46 The output from subprocesses can be analysed in a very basic manner to
47 try to present progress information to the user.
48 """ 36 """
49 # Force color on/off, or use color only if stdout is a terminal. 37 # Force color on/off, or use color only if stdout is a terminal.
50 COLOR_OFF, COLOR_ON, COLOR_IF_TERMINAL = range(3) 38 COLOR_OFF, COLOR_ON, COLOR_IF_TERMINAL = range(3)
51 39
52 def __init__(self, name, color=COLOR_IF_TERMINAL): 40 def __init__(self, name, color=COLOR_IF_TERMINAL):
53 """Create a new operation. 41 """Create a new operation.
54 42
55 Args: 43 Args:
56 name: Operation name in a form to be displayed for the user. 44 name: Operation name in a form to be displayed for the user.
57 color: Determines policy for sending color to stdout: 45 color: Determines policy for sending color to stdout:
58 COLOR_OFF: never send color. 46 COLOR_OFF: never send color.
59 COLOR_ON: always send color. 47 COLOR_ON: always send color.
60 COLOR_IF_TERMINAL: send color if output apperas to be a terminal. 48 COLOR_IF_TERMINAL: send color if output apperas to be a terminal.
61 """ 49 """
62 self._name = name # Operation name. 50 self._name = name # Operation name.
63 self.verbose = False # True to echo subprocess output. 51 self._verbose = False # True to echo subprocess output.
64 self.progress = True # True to report progress of the operation 52 self._progress = True # True to report progress of the operation
65 self._column = 0 # Current output column (always 0 unless verbose). 53 self._column = 0 # Current output column (always 0 unless verbose).
66 self._update_len = 0 # Length of last progress update message. 54 self._update_len = 0 # Length of last progress update message.
67 self._line = '' # text of current line, so far 55 self._line = '' # text of current line, so far
68 56
69 # By default, we display ANSI colors unless output is redirected. 57 # By default, we display ANSI colors unless output is redirected.
70 want_color = (color == self.COLOR_ON or 58 want_color = (color == self.COLOR_ON or
71 (color == self.COLOR_IF_TERMINAL and os.isatty(sys.stdout.fileno()))) 59 (color == self.COLOR_IF_TERMINAL and os.isatty(sys.stdout.fileno())))
72 self._color = Color(want_color) 60 self._color = Color(want_color)
73 61
74 # -1 = no newline pending 62 # -1 = no newline pending
(...skipping 10 matching lines...) Expand all
85 def __del__(self): 73 def __del__(self):
86 """Object is about to be destroyed, so finish out output cleanly.""" 74 """Object is about to be destroyed, so finish out output cleanly."""
87 self.FinishOutput() 75 self.FinishOutput()
88 76
89 def FinishOutput(self): 77 def FinishOutput(self):
90 """Finish off any pending output. 78 """Finish off any pending output.
91 79
92 This finishes any output line currently in progress and resets the color 80 This finishes any output line currently in progress and resets the color
93 back to normal. 81 back to normal.
94 """ 82 """
95 self._FinishLine(self.verbose, final=True) 83 self._FinishLine(self._verbose, final=True)
96 if self._column and self.verbose: 84 if self._column and self._verbose:
97 print self._color.Stop() 85 print self._color.Stop()
98 self._column = 0 86 self._column = 0
99 87
100 def WereErrorsDetected(self): 88 def WereErrorsDetected(self):
101 """Returns whether any errors have been detected. 89 """Returns whether any errors have been detected.
102 90
103 Returns: 91 Returns:
104 True if any errors have been detected in subprocess output so far. 92 True if any errors have been detected in subprocess output so far.
105 False otherwise 93 False otherwise
106 """ 94 """
107 return self._error_count > 0 95 return self._error_count > 0
108 96
97 def SetVerbose(self, verbose):
98 """Enable / disable verbose mode.
99
100 In verbose mode all output from subprocesses is displayed, otherwise
101 this output is normally supressed, unless we think it indicates an error.
102
103 Args:
104 verbose: True to display all subprocess output, False to supress.
105 """
106 self._verbose = verbose
107
108 def SetProgress(self, progress):
109 """Enable / disable progress display.
110
111 The output from subprocesses can be analysed in a very basic manner to
112 try to present progress information to the user.
113
114 Args:
115 progress: True to enable progress display, False to disable.
116 """
117 self._progress = progress
118
109 def SetName(self, name): 119 def SetName(self, name):
110 """Set the name of the operation as displayed to the user. 120 """Set the name of the operation as displayed to the user.
111 121
112 Args: 122 Args:
113 name: Operation name. 123 name: Operation name.
114 """ 124 """
115 self._name = name 125 self._name = name
116 126
117 def _FilterOutputForErrors(self, line, print_error): 127 def _FilterOutputForErrors(self, line, print_error):
118 """Filter a line of output to look for and display errors. 128 """Filter a line of output to look for and display errors.
119 129
120 This uses a few regular expression searches to spot common error reports 130 This uses a few regular expression searches to spot common error reports
121 from subprocesses. A count of these is kept so we know how many occurred. 131 from subprocesses. A count of these is kept so we know how many occurred.
122 Optionally they are displayed in red on the terminal. 132 Optionally they are displayed in red on the terminal.
123 133
124 Args: 134 Args:
125 line: the output line to filter, as a string. 135 line: the output line to filter, as a string.
126 print_error: True to print the error, False to just record it. 136 print_it: True to print the error, False to just record it.
127 """ 137 """
128 bad_things = ['Cannot GET', 'ERROR', '!!!', 'FAILED'] 138 bad_things = ['Cannot GET', 'ERROR', '!!!', 'FAILED']
129 for bad_thing in bad_things: 139 for bad_thing in bad_things:
130 if re.search(bad_thing, line, flags=re.IGNORECASE): 140 if re.search(bad_thing, line, flags=re.IGNORECASE):
131 self._error_count += 1 141 self._error_count += 1
132 if print_error: 142 if print_error:
133 print self._color.Color(self._color.RED, line) 143 print self._color.Color(self._color.RED, line)
134 break 144 break
135 145
136 def _FilterOutputForProgress(self, line): 146 def _FilterOutputForProgress(self, line):
(...skipping 14 matching lines...) Expand all
151 def _Progress(self, upto, total): 161 def _Progress(self, upto, total):
152 """Record and optionally display progress information. 162 """Record and optionally display progress information.
153 163
154 Args: 164 Args:
155 upto: which step we are up to in the operation (integer, from 0). 165 upto: which step we are up to in the operation (integer, from 0).
156 total: total number of steps in operation, 166 total: total number of steps in operation,
157 """ 167 """
158 if total > 0: 168 if total > 0:
159 update_str = '%s...%d%% (%d of %d)' % (self._name, 169 update_str = '%s...%d%% (%d of %d)' % (self._name,
160 upto * 100 // total, upto, total) 170 upto * 100 // total, upto, total)
161 if self.progress: 171 if self._progress:
162 # Finish the current line, print progress, and remember its length. 172 # Finish the current line, print progress, and remember its length.
163 self._FinishLine(self.verbose) 173 self._FinishLine(self._verbose)
164 174
165 # Sometimes the progress string shrinks and in this case we need to 175 # Sometimes the progress string shrinks and in this case we need to
166 # blank out the characters at the end of the line that will not be 176 # blank out the characters at the end of the line that will not be
167 # overwritten by the new line 177 # overwritten by the new line
168 pad = max(self._update_len - len(update_str), 0) 178 pad = max(self._update_len - len(update_str), 0)
169 sys.stdout.write(update_str + (' ' * pad) + '\r') 179 sys.stdout.write(update_str + (' ' * pad) + '\r')
170 self._update_len = len(update_str) 180 self._update_len = len(update_str)
171 181
172 def _FinishLine(self, display, final=False): 182 def _FinishLine(self, display, final=False):
173 """Finish off the current line and prepare to start a new one. 183 """Finish off the current line and prepare to start a new one.
(...skipping 103 matching lines...) Expand 10 before | Expand all | Expand 10 after
277 self._column += len(text) 287 self._column += len(text)
278 # If a newline is required, remember to output it later. 288 # If a newline is required, remember to output it later.
279 if newline: 289 if newline:
280 self._pending_nl = self._column 290 self._pending_nl = self._column
281 self._column = 0 291 self._column = 0
282 292
283 self._line += text 293 self._line += text
284 294
285 # If we now have a whole line, check it for errors and progress. 295 # If we now have a whole line, check it for errors and progress.
286 if newline: 296 if newline:
287 self._FilterOutputForErrors(self._line, print_error=not display) 297 self._FilterOutputForErrors(self._line, print_it=not display)
288 self._FilterOutputForProgress(self._line) 298 self._FilterOutputForProgress(self._line)
289 self._line = '' 299 self._line = ''
290 300
291 def Output(self, stream, data): 301 def Output(self, stream, data):
292 """Handle the output of a block of text from the subprocess. 302 """Handle the output of a block of text from the subprocess.
293 303
294 All subprocess output should be sent through this method. It is split into 304 All subprocess output should be sent through this method. It is split into
295 lines which are processed separately using the _Out() method. 305 lines which are processed separately using the _Out() method.
296 306
297 Args: 307 Args:
298 stream: Which file the output come in on: 308 stream: Which file the output come in on:
299 sys.stdout: stdout 309 sys.stdout: stdout
300 sys.stderr: stderr 310 sys.stderr: stderr
301 None: Our own internal output 311 None: Our own internal output
302 data: Output data as a big string, potentially containing many lines of 312 data: Output data as a big string, potentially containing many lines of
303 text. Each line should end with \r\n. There is no requirement to send 313 text. Each line should end with \r\n. There is no requirement to send
304 whole lines - this method happily handles fragments and tries to 314 whole lines - this method happily handles fragments and tries to
305 present then to the user as early as possible 315 present then to the user as early as possible
306 316
307 #TODO(sjg): Just use a list as the input parameter to avoid the split. 317 #TODO(sjg): Just use a list as the input parameter to avoid the split.
308 """ 318 """
309 # We cannot use splitlines() here as we need this exact behavior 319 lines = data.splitlines()
310 lines = data.split('\r\n')
311 320
312 # Output each full line, with a \n after it. 321 # Output each full line, with a \n after it.
313 for line in lines[:-1]: 322 for line in lines[:-1]:
314 self._Out(stream, line, display=self.verbose, newline=True) 323 self._Out(stream, line, display=self._verbose, newline=True)
315 324
316 # If we have a partial line at the end, output what we have. 325 # If we have a partial line at the end, output what we have.
317 # We will continue it later. 326 # We will continue it later.
318 if lines[-1]: 327 if lines[-1]:
319 self._Out(stream, lines[-1], display=self.verbose) 328 self._Out(stream, lines[-1], display=self._verbose)
320 329
321 # Flush so that the terminal will receive partial line output (now!) 330 # Flush so that the terminal will receive partial line output (now!)
322 sys.stdout.flush() 331 sys.stdout.flush()
323 332
324 def Outline(self, line): 333 def Outline(self, line):
325 """Output a line of text to the display. 334 """Output a line of text to the display.
326 335
327 This outputs text generated internally, such as a warning message or error 336 This outputs text generated internally, such as a warning message or error
328 summary. It ensures that our message plays nicely with child output if 337 summary. It ensures that our message plays nicely with child output if
329 any. 338 any.
330 339
331 Args: 340 Args:
332 line: text to output (without \n on the end) 341 line: text to output (without \n on the end)
333 """ 342 """
334 self._Out(None, line, display=True, newline=True) 343 self._Out(None, line, display=True, newline=True)
335 self._FinishLine(display=True) 344 self._FinishLine(display=True)
336
337 def Info(self, line):
338 """Output a line of information text to the display in verbose mode.
339
340 Args:
341 line: text to output (without \n on the end)
342 """
343 self._Out(None, self._color.Color(self._color.BLUE, line),
344 display=self.verbose, newline=True)
345 self._FinishLine(display=True)
OLDNEW
« no previous file with comments | « lib/cros_subprocess.py ('k') | shell/chromite_env.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698