Use file pool to handle running out of file descriptors.

pkg/barback/test/too_many_open_files_test.dart

+// Copyright (c) 2013, the Dart project authors.  Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+library barback.test.too_many_open_files_test;
+
+import 'dart:async';
+import 'dart:io';
+
+import 'package:barback/barback.dart';
+import 'package:path/path.dart' as pathos;
+import 'package:unittest/unittest.dart';
+
+import 'utils.dart';
+
+main() {
+  initConfig();
+
+  test("handles a large number of simultaneous asset reads", () {
+    // Make a 1MB binary file in a temp directory.
+    var tempDir = Directory.systemTemp.createTempSync("barback").path;
+    var filePath = pathos.join(tempDir, "out.bin");
+    var bytes = new Iterable.generate(1024 * 1024, (i) => i % 255).toList();
+    new File(filePath).writeAsBytesSync(bytes);
+
+    var id = new AssetId("myapp", "out.bin");
+
+    // Create a large number of assets, larger than the file descriptor limit
+    // of most machines and start reading from all of them.
+    var futures = [];
+    for (var i = 0; i < 5000; i++) {

[Anders Johnsen, 2013/10/15 10:48:26]

Currently we don't tract external memory, but once we start doing so, this could
lead to OOM:

  5000 * 64k = 327MB

Alive memory, at the bare minimum.

Can this be done using maybe 100b files?

[Bob Nystrom, 2013/10/15 17:57:15]

Yeah, 5k is a bit gratuitous. I needed that to hit the problem on my machine
because, somehow, my ulimit got bumped up for this. But it seems to be back to
the normal (but sad) 256. Lowered this some.

+      var completer = new Completer();
+      var asset = new Asset.fromPath(id, filePath);
+      var stream = asset.read();
+
+      stream.listen((data) {
+        // Do nothing.
+      }, onError: (error) {
+        completer.completeError(error);
+      }, onDone: () {
+        if (!completer.isCompleted) completer.complete();
+      });
+
+      futures.add(completer.future);
+    }
+
+    expect(Future.wait(futures).whenComplete(() {
+      new Directory(tempDir).delete(recursive: true);
+    }), completes);
+  });
+}