Added README, a bin/serve example server which can be used locally.

Added README, a bin/serve example server which can be used locally.

R=sgjesse@google.com

Committed: https://github.com/dart-lang/pub_server/commit/fe1ce36119cd241ec9288bc29bbfaec315bb5539

[kustermann, 2015-02-14 00:01:14]

kustermann@google.com changed reviewers:
+ sgjesse@google.com

[Søren Gjesse, 2015-02-16 10:23:55]

Should bin/server.dart move to bin/examples/server.dart?

or is that not supported by 'pub run'?

https://codereview.chromium.org/930513002/diff/1/README.md
File README.md (right):

https://codereview.chromium.org/930513002/diff/1/README.md#newcode14
README.md:14: `bin/serve.dart`. It uses a filesystem-based `PackageRepository`
for storing
Rename to server.dart?

https://codereview.chromium.org/930513002/diff/1/README.md#newcode22
README.md:22: ~ $ git clone https://github.com/dart-lang/pub_serverserver.git
pub_sererserver -> pub_server

https://codereview.chromium.org/930513002/diff/1/README.md#newcode28
README.md:28: 
End code block here.

https://codereview.chromium.org/930513002/diff/1/README.md#newcode30
README.md:30: 
Start new code block here.

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/cow_reposit...
File bin/src/examples/cow_repository.dart (right):

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/cow_reposit...
bin/src/examples/cow_repository.dart:20: ///
Please describe that this will also cache packages read from remote.

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/cow_reposit...
bin/src/examples/cow_repository.dart:64: Future<Stream> download(String package,
String version) {
You are already using async and await in other parts of the code, so just use
that here as well.

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/cow_reposit...
bin/src/examples/cow_repository.dart:87: _logger.info('Starting upload to local
package repository.');
Should it be allowed to upload packages which are already present on the remote?
Even if we check there is of course still a chance that this will be uploaded
remote at the same time.

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/cow_reposit...
bin/src/examples/cow_repository.dart:106: Map<String,
Completer<Set<PackageVersion>>> _versionCompleter = {};
_versionCompleters (add the plural s)

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/cow_reposit...
bin/src/examples/cow_repository.dart:110: // TODO: After a cache expiration we
should invalidate entries and re-fetch
What should the expiration time be? We have no way of knowing whether new
versions where added to the remote.

Are you thinking of just adding a local timer?

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/file_reposi...
File bin/src/examples/file_repository.dart (right):

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/file_reposi...
bin/src/examples/file_repository.dart:83: if (!packageVersionDir.existsSync()) {
What of it does exist? Shouldn't that be an error?

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/http_proxy_...
File bin/src/examples/http_proxy_repository.dart (right):

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/http_proxy_...
bin/src/examples/http_proxy_repository.dart:49: return
_streamViaPaging(nextPage);
Why are you using _streamViaPaging? There is always only one page.

https://codereview.chromium.org/930513002/diff/1/lib/shelf_pubserver.dart
File lib/shelf_pubserver.dart (right):

https://codereview.chromium.org/930513002/diff/1/lib/shelf_pubserver.dart#new...
lib/shelf_pubserver.dart:325: subscription.cancel();
:-)

[kustermann, 2015-02-16 11:42:10]

PTAL - Due to the fact that we might publish this package at some point to pub,
I don't want to put anything in bin/ yet. So people can't use "pub run" for now,
and have to manually "dart example/server.dart", but I think that is ok.

https://codereview.chromium.org/930513002/diff/1/README.md
File README.md (right):

https://codereview.chromium.org/930513002/diff/1/README.md#newcode14
README.md:14: `bin/serve.dart`. It uses a filesystem-based `PackageRepository`
for storing
On 2015/02/16 10:23:54, Søren Gjesse wrote:
> Rename to server.dart?

Done - Although I thought of it as "serve this directory".

https://codereview.chromium.org/930513002/diff/1/README.md#newcode22
README.md:22: ~ $ git clone https://github.com/dart-lang/pub_serverserver.git
On 2015/02/16 10:23:54, Søren Gjesse wrote:
> pub_sererserver -> pub_server

Done.

https://codereview.chromium.org/930513002/diff/1/README.md#newcode28
README.md:28: 
On 2015/02/16 10:23:54, Søren Gjesse wrote:
> End code block here.

Not really, the message is comming from the "pub run server" application

https://codereview.chromium.org/930513002/diff/1/README.md#newcode30
README.md:30: 
On 2015/02/16 10:23:54, Søren Gjesse wrote:
> Start new code block here.

See above, it's the output of the tool

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/cow_reposit...
File bin/src/examples/cow_repository.dart (right):

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/cow_reposit...
bin/src/examples/cow_repository.dart:20: ///
On 2015/02/16 10:23:54, Søren Gjesse wrote:
> Please describe that this will also cache packages read from remote.

Done.

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/cow_reposit...
bin/src/examples/cow_repository.dart:64: Future<Stream> download(String package,
String version) {
On 2015/02/16 10:23:54, Søren Gjesse wrote:
> You are already using async and await in other parts of the code, so just use
> that here as well.

It was very old code. - Done

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/cow_reposit...
bin/src/examples/cow_repository.dart:87: _logger.info('Starting upload to local
package repository.');
On 2015/02/16 10:23:54, Søren Gjesse wrote:
> Should it be allowed to upload packages which are already present on the
remote?
> Even if we check there is of course still a chance that this will be uploaded
> remote at the same time.

Exactly. We're also not ensuring that we cannot override a package. It's just a
simple development server for now.

We might want to move e.g. the uploading logic from the pub-dartlang-dart to
this package into lib/. But for now, this is just a proof-of-concept.

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/cow_reposit...
bin/src/examples/cow_repository.dart:106: Map<String,
Completer<Set<PackageVersion>>> _versionCompleter = {};
On 2015/02/16 10:23:54, Søren Gjesse wrote:
> _versionCompleters (add the plural s)

Done.

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/cow_reposit...
bin/src/examples/cow_repository.dart:110: // TODO: After a cache expiration we
should invalidate entries and re-fetch
On 2015/02/16 10:23:54, Søren Gjesse wrote:
> What should the expiration time be? We have no way of knowing whether new
> versions where added to the remote.
> 
> Are you thinking of just adding a local timer?

Well, the local cache could have the following policy:

if (entry was written within the last 5 minutes) return it
else {
  if (no async operation has been fired) {
    asynchronously do a fetch to re-populate the cache
  }
  return stale entry;
}

This way, we'll have low latency always, and after some (eventually consistent)
time frame we'll have a fresh cache.

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/file_reposi...
File bin/src/examples/file_repository.dart (right):

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/file_reposi...
bin/src/examples/file_repository.dart:83: if (!packageVersionDir.existsSync()) {
On 2015/02/16 10:23:54, Søren Gjesse wrote:
> What of it does exist? Shouldn't that be an error?

Well, currently we allow uploading any package, it will override one in case
there is an already existing one.  There is no race condition, since we're doing
sync File I/O.

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/http_proxy_...
File bin/src/examples/http_proxy_repository.dart (right):

https://codereview.chromium.org/930513002/diff/1/bin/src/examples/http_proxy_...
bin/src/examples/http_proxy_repository.dart:49: return
_streamViaPaging(nextPage);
On 2015/02/16 10:23:54, Søren Gjesse wrote:
> Why are you using _streamViaPaging? There is always only one page.

Good point.

This was old code from when I had the `Stream<Package> packages()` support --
which requires paging. Then I reused it in the `Stream<PackageVersino> 
versions()` function.

https://codereview.chromium.org/930513002/diff/1/lib/shelf_pubserver.dart
File lib/shelf_pubserver.dart (right):

https://codereview.chromium.org/930513002/diff/1/lib/shelf_pubserver.dart#new...
lib/shelf_pubserver.dart:325: subscription.cancel();
On 2015/02/16 10:23:54, Søren Gjesse wrote:
> :-)

Acknowledged.

[Søren Gjesse, 2015-02-16 11:54:51]

lgtm

https://codereview.chromium.org/930513002/diff/20001/README.md
File README.md (right):

https://codereview.chromium.org/930513002/diff/20001/README.md#newcode26
README.md:26: ~/pub_server $ pub run server -d /tmp/package-db
With the sample moved to examples I guess this should change.

https://codereview.chromium.org/930513002/diff/20001/example/src/examples/cow...
File example/src/examples/cow_repository.dart (right):

https://codereview.chromium.org/930513002/diff/20001/example/src/examples/cow...
example/src/examples/cow_repository.dart:19: /// repository. This caches
effectively all packages requested through this
caches effectively -> effectively caches

[kustermann, 2015-02-16 14:41:27]

New patchsets have been uploaded after l-g-t-m from sgjesse@google.com

[kustermann, 2015-02-16 14:44:03]

There seems to be still an issue with package:mime :'(

I'll land this now to make some progress and move back to the issue at another
point in time.

https://codereview.chromium.org/930513002/diff/20001/README.md
File README.md (right):

https://codereview.chromium.org/930513002/diff/20001/README.md#newcode26
README.md:26: ~/pub_server $ pub run server -d /tmp/package-db
On 2015/02/16 11:54:51, Søren Gjesse wrote:
> With the sample moved to examples I guess this should change.

Done.

https://codereview.chromium.org/930513002/diff/20001/example/src/examples/cow...
File example/src/examples/cow_repository.dart (right):

https://codereview.chromium.org/930513002/diff/20001/example/src/examples/cow...
example/src/examples/cow_repository.dart:19: /// repository. This caches
effectively all packages requested through this
On 2015/02/16 11:54:51, Søren Gjesse wrote:
> caches effectively -> effectively caches

Done.

[kustermann, 2015-02-16 14:45:00]

Committed patchset #3 (id:40001) manually as
fe1ce36119cd241ec9288bc29bbfaec315bb5539 (presubmit successful).