luci-go/common/dirwalk: Code for walking a directory tree efficiently

common/dirwalk/test.sh

Index: common/dirwalk/test.sh
diff --git a/common/dirwalk/test.sh b/common/dirwalk/test.sh
new file mode 100755
index 0000000000000000000000000000000000000000..4f261e01850d843c8b5782982afbbff5bad1cb8d
--- /dev/null
+++ b/common/dirwalk/test.sh
@@ -0,0 +1,39 @@
+#!/bin/bash

[M-A Ruel, 2016/09/15 14:31:02]

I'd like to have the test to be split up in a separate CL, that could be
committed first. It needs to be made to work on Windows, so I recommend writing
this in Go too. It's not super efficient LoC-wise but it'll make the whole thing
more maintainable.

[mithro, 2016/09/20 12:41:44]

This file is just to make it easy to run all the performance tests at once (so I
can step away for a coffee) -- I wouldn't expect to use if you wanted to run
these tests continuously.

Should I just remove the file?

[M-A Ruel, 2016/09/20 16:37:27]

Yes, we do not use bash unless it is expressively posix specific. As I said,
this has to be reimplemented in Go. I'm fine with the idea, just not the
language.

+# Copyright 2016 The LUCI Authors. All rights reserved.
+# Use of this source code is governed under the Apache License, Version 2.0
+# that can be found in the LICENSE file.
+
+TESTS=common/dirwalk/tests/*.json
+
+go install github.com/luci/luci-go/common/dirwalk/tests/tools/gendir
+go install github.com/luci/luci-go/common/dirwalk/tests/tools/walkdir
+
+echo "Generating the test directories"
+TMPDIR=/usr/local/google/tmp/luci-tests
+mkdir -p $TMPDIR
+for TESTFILE in $TESTS; do
+	TESTNAME="$(basename $TESTFILE .json)"
+	TESTDIR="$TMPDIR/$TESTNAME"
+	if ! [ -d $TESTDIR ]; then
+		echo "Generating test directory for $TESTNAME"
+		gendir -config $TESTFILE -outdir $TESTDIR
+		du -h $TESTDIR
+	fi
+done
+
+for METHOD in simple nostat parallel; do
+	echo "Running $METHOD"
+	for TESTFILE in $TESTS; do
+		TESTNAME="$(basename $TESTFILE .json)"
+		TESTDIR="$TMPDIR/$TESTNAME"
+		OUTPUT=output.$METHOD.$TESTNAME
+		echo "Running $METHOD.$TESTNAME"
+		rm $OUTPUT
+		$(which time) --verbose --output=$OUTPUT --append walkdir --dir $TESTDIR --method $METHOD $@ 2> $OUTPUT
+		tail -n 20 $OUTPUT
+		echo
+	done
+	echo
+	echo
+	echo
+done