Make pumpEventQueue() private and explain why it is used in 'results'.

pkg/analyzer/lib/src/dart/analysis/driver.dart

Index: pkg/analyzer/lib/src/dart/analysis/driver.dart
diff --git a/pkg/analyzer/lib/src/dart/analysis/driver.dart b/pkg/analyzer/lib/src/dart/analysis/driver.dart
index b485d8557961b5eb4cbbfcb99d42f964737d2408..6489a452e6734452772742aafef569aaea71f4e5 100644
--- a/pkg/analyzer/lib/src/dart/analysis/driver.dart
+++ b/pkg/analyzer/lib/src/dart/analysis/driver.dart
@@ -231,7 +231,17 @@ class AnalysisDriver {
     try {
       PerformanceLogSection analysisSection = null;
       while (true) {
-        await pumpEventQueue(100);
+        // Pump the event queue to allow IO and other asynchronous data
+        // processing while analysis is active. For example Analysis Server
+        // need to be able to process update content or set priority requests

[Brian Wilkerson, 2016/10/31 20:41:04]

"need" --> "needs"
"update content" --> "updateContent"
"set priority" --> "setPriorityFiles"

[scheglov, 2016/10/31 20:48:14]

Done.

+        // while background analysis is in progress.
+        //
+        // The number of pumpings is arbitrary, might be changed if we see that
+        // analysis or other data processing tasks are starving. Ideally we
+        // would need to be able to set priority of (continuous) asynchronous
+        // tasks.
+        await _pumpEventQueue(128);

[Florian Schneider, 2016/10/31 20:41:01]

dbc: This seems arbitrary. Why not wait on the completion of the analysis
results directly?

[scheglov, 2016/10/31 20:48:14]

Yes, the number is arbitrary.

The question does not make sense to me :-(

What I would like to wait is any other asynchronous processing that requires CPU
attention, i.e. does not sleep or wait for data to arrive.

[Florian Schneider, 2016/10/31 22:16:50]

What happens if this number is too low, or too high?

What I meant was that instead of pumpEventQueue, await on an the future with the
result: see my CL https://codereview.chromium.org/2059503002/ which fixed issue
#26556.

I'm asking because I replaced several uses of pumpEventQueue in tests in the
past to fix timing-dependent analyzer tests failing because some task takes a
bit longer (e.g. running in debug mode). Last time I checked there were still
tests that are timing-dependent. 

[Paul Berry, 2016/10/31 22:40:33]

Unfortunately this is a different situation from the aforementioned CL.  In this
CL, what's happening is that there isn't any particular result that we're
waiting for.  Instead, the situation is that we are trying to do a long-running
computation in such a way that it appears to the user to be a background
process, while allowing other code to use the same event queue to quickly handle
foreground requests from the user.  It would be really cool if there was a way
for this code to say to the event loop "yield to all other events in the queue,
and any other events they may spawn, until the event queue is completely empty. 
Then schedule me to continue executing".  But I don't know of any way to do
that.  So as a poor man's substitute, we are pumping the event queue a large
number of times in the hopes that any higher priority events get completely
taken care of (or go into a state where they are waiting for I/O or timers)
before we try to do more time consuming analysis.

As I mentioned in a previous email, in the long run I would like us to fix this
by moving the time-consuming analysis to its own isolate, so that the main
isolate can respond quickly to user requests.  But it is going to take time to
rework the code to make that possible, so in the short term pumping the event
queue is the best solution we've come up with.

+
         await _hasWork.signal;
 
         if (analysisSection == null) {
@@ -377,20 +387,6 @@ class AnalysisDriver {
   }
 
   /**
-   * Returns a [Future] that completes after pumping the event queue [times]
-   * times. By default, this should pump the event queue enough times to allow
-   * any code to run, as long as it's not waiting on some external event.
-   */
-  Future pumpEventQueue([int times = 5000]) {
-    if (times == 0) return new Future.value();
-    // We use a delayed future to allow microtask events to finish. The
-    // Future.value or Future() constructors use scheduleMicrotask themselves and
-    // would therefore not wait for microtask callbacks that are scheduled after
-    // invoking this method.
-    return new Future.delayed(Duration.ZERO, () => pumpEventQueue(times - 1));
-  }
-
-  /**
    * Remove the file with the given [path] from the list of files to analyze.
    *
    * The [path] must be absolute and normalized.
@@ -725,6 +721,17 @@ class AnalysisDriver {
   }
 
   /**
+   * Returns a [Future] that completes after performing [times] pumpings of
+   * the event queue.
+   */
+  static Future _pumpEventQueue(int times) {
+    if (times == 0) {
+      return new Future.value();
+    }
+    return new Future.delayed(Duration.ZERO, () => _pumpEventQueue(times - 1));
+  }
+
+  /**
    * Remove and return the first item in the given [set].
    */
   static Object/*=T*/ _removeFirst/*<T>*/(LinkedHashSet<Object/*=T*/ > set) {