first cut fuzz test for analysis server

first cut fuzz test for analysis server

This very rough first cut drives the analysis server snapshot so that it...

- analyzes the specified source tree
- locates and 'opens' a file in that tree
- modifies the content of that file in memory without saving to disk
- requests code completions

I'm not happy with the readability of the ServerTest # test() method because of all the Future references.
I'm going to experiment with async / await to see if that can make it more readable.

There are several libraries that I pulled out of the analysis server code base and cleaned up to use
with an analysis server client. In the long run, these analysis server client libraries need to be cleaned up
so that clients can import them rather than having to do what I've done. These libraries include:

- example/server_test/bin/byte_stream_channel.dart
- example/server_test/bin/channel.dart
- example/server_test/bin/generated_protocol.dart
- example/server_test/bin/json.dart
- example/server_test/bin/protocol.dart

BUG=
R=lukechurch@google.com

Committed: https://code.google.com/p/dart/source/detail?r=40556

[danrubel, 2014-09-19 15:34:55]

danrubel@google.com changed reviewers:
+ lukechurch@google.com

[lukechurch, 2014-09-21 05:36:11]

lgtm on the basis that this is 'experimental' and 'work in progress'. :-)

Could you note that in the readme and the help test of the 'how to use'?

Once this has landed I'll send you a series of CLs to adjust the driver, the
mutators and the reporting mechanism.

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
File pkg/analysis_server/bin/fuzz.dart (right):

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz.dart:84: if (!appDir.existsSync()) {
What is an application directory? Do you mean that we target this at the place
where the pubspec is located? Or just a collection of dart files?

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz.dart:108: print('Test completed successfully');
I don't really see these as tests as such, but more as a tool that can be used
to locate errors. I am up or being persuaded that they should be run and
structured as tests.

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz.dart:111: stderr.writeln(error);
From experience of writing a couple of these things before, using stderr makes
doing post analysis of the results more complicated than it really needs to be.
I've been inclined to route all the data into a seperate log file and dumping
tool status information to stdout and very little to stderr.

This can be finessed going forwards.

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz.dart:113: exitCode = 12;
Why 12?

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz.dart:140: expect(list.length, equals(0));
I don't follow this. Why would the list has of length zero? If it is zero, why
do we then iterate over it?

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
File pkg/analysis_server/bin/fuzz/server_manager.dart (right):

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz/server_manager.dart:77: var id = new
CompletionGetSuggestionsResult.fromResponse(response).id;
It's not quite obvious from just reading the code what this is the id of. is it
the response?

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz/server_manager.dart:154: bool _stopRequested =
false;
whitespace is a bit whacky

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz/server_manager.dart:175: * Return a future that
completes when the analysis is finished.
Does this recurse into children? Given the ambiguity in Dart of the notion of an
'application' it's probably helpful to be specific here

[danrubel, 2014-09-22 18:27:26]

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
File pkg/analysis_server/bin/fuzz.dart (right):

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz.dart:84: if (!appDir.existsSync()) {
On 2014/09/21 05:36:10, lukechurch wrote:
> What is an application directory? Do you mean that we target this at the place
> where the pubspec is located? Or just a collection of dart files?

This is the root directory containing the files to be analyzed. This directory
may or may not contain a pubspec and represent a Dart package or could just be a
loose collection of Dart files to be analyzed.

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz.dart:108: print('Test completed successfully');
On 2014/09/21 05:36:11, lukechurch wrote:
> I don't really see these as tests as such, but more as a tool that can be used
> to locate errors. I am up or being persuaded that they should be run and
> structured as tests.

I'm happy to refactor in subsequent CLs as we/I better understand the
requirements.

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz.dart:111: stderr.writeln(error);
On 2014/09/21 05:36:11, lukechurch wrote:
> From experience of writing a couple of these things before, using stderr makes
> doing post analysis of the results more complicated than it really needs to
be.
> I've been inclined to route all the data into a seperate log file and dumping
> tool status information to stdout and very little to stderr.
> 
> This can be finessed going forwards. 

Fair enough. Will refactor in subsequent CLs.

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz.dart:113: exitCode = 12;
On 2014/09/21 05:36:11, lukechurch wrote:
> Why 12?

Just picking a random exit code that's not 1 and unique within this codebase.
Don't know if that's important or if I should just return exitCode = 1.

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz.dart:140: expect(list.length, equals(0));
On 2014/09/21 05:36:11, lukechurch wrote:
> I don't follow this. Why would the list has of length zero? If it is zero, why
> do we then iterate over it?

I'm getting an exception after text replacement but before this point in the
code so I set this to zero so that this would fail when I'm actually returning
real data. I want to land this CL and iterate.

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
File pkg/analysis_server/bin/fuzz/server_manager.dart (right):

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz/server_manager.dart:77: var id = new
CompletionGetSuggestionsResult.fromResponse(response).id;
On 2014/09/21 05:36:11, lukechurch wrote:
> It's not quite obvious from just reading the code what this is the id of. is
it
> the response?

Good point. This is the completionId not the responseId. Renamed so that its
more obvious from reading the code.

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz/server_manager.dart:154: bool _stopRequested =
false;
On 2014/09/21 05:36:11, lukechurch wrote:
> whitespace is a bit whacky

Added comments. Hopefully not so whacky now.

https://codereview.chromium.org/584963002/diff/1/pkg/analysis_server/bin/fuzz...
pkg/analysis_server/bin/fuzz/server_manager.dart:175: * Return a future that
completes when the analysis is finished.
On 2014/09/21 05:36:11, lukechurch wrote:
> Does this recurse into children? Given the ambiguity in Dart of the notion of
an
> 'application' it's probably helpful to be specific here

It analyzes all of the files in the given directory. I updated the method
comment to reflect this.

[danrubel, 2014-09-22 18:27:56]

Committed patchset #2 (id:20001) manually as 40556 (presubmit successful).