logdog: Add bundler library.

client/internal/logdog/butler/bundler/bundler.go

Index: client/internal/logdog/butler/bundler/bundler.go
diff --git a/client/internal/logdog/butler/bundler/bundler.go b/client/internal/logdog/butler/bundler/bundler.go
new file mode 100644
index 0000000000000000000000000000000000000000..05bfd12dfd019c30f5780a42edbfec118acb4b33
--- /dev/null
+++ b/client/internal/logdog/butler/bundler/bundler.go
@@ -0,0 +1,74 @@
+// Copyright 2015 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package bundler

[tandrii(chromium), 2015/08/11 17:32:22]

I guess this CL introduces this package. If so, it'd be great to have doc.go :)

[dnj, 2015/08/11 18:14:28]

Done.

+
+import (
+	"github.com/luci/luci-go/common/logdog/protocol"
+)
+
+// Config contains configuration information for a Bundler instance.
+type Config struct {
+	// TemplateBundle is the base template ButlerLogBundle. Generated bundles
+	// will be derived from a copy of this bundle.
+	//
+	// The TemplateBundle may not have any log entries. If it does, they will
+	// not be retained.
+	TemplateBundle protocol.ButlerLogBundle
+
+	// NewSizer is the Sizer instance to use to help with Bundler accounting.
+	// If nil, a FastSizer will be used.
+	NewSizer func(*protocol.ButlerLogBundle) Sizer
+}
+
+// Sizer is a stateful instance that tracks the size of LogDog ButlerLogBundle
+// protobufs as they are constructed.
+//
+// A Sizer may overestimate the size, but it should strive to never
+// underestimate the size.
+type Sizer interface {
+	// Size returns the current size.
+	Size() int64
+
+	// AppendBundleEntry adds a ButlerLogBundle_Entry to the Sizer.
+	AppendBundleEntry(*protocol.ButlerLogBundle_Entry)
+
+	// AppendLogEntry adds a LogEntry to the Sizer.
+	AppendLogEntry(*protocol.ButlerLogBundle_Entry, *protocol.LogEntry)
+}
+
+// Bundler aggregates multiple Butler log bundle entries together into a single
+// log bundle. A Bundler is not goroutine-safe.
+//
+// A Bundler can vet incoming data, rejecting it if its serialized protobuf
+// exceeds a size threshold. For efficiency, the Bundler performs a conservative
+// estimate of the sizes.
+type Bundler interface {
+	// Append adds a single stream's bundle entry to the Bundler.
+	//
+	// The Bundler formally takes ownership of this entry and its fields
+	// (including its logs). The caller must not modify it after Append().
+	Append(*protocol.ButlerLogBundle_Entry)
+
+	// GetBundle converts the current set of buffered log data into a series of
+	// ButlerLogBundle instances.
+	//
+	// If the supplied threshold value is greater than zero, each returned bundle
+	// will only contain enough log messages such that its serialized protobuf
+	// byte count is less or equal to the threshold.
+	//
+	// Note: It may seem useful to have a single-bundle return function. However,
+	// in practice, the Bundler is used by the Butler to extract all bundles.
+	// Implementing the single-bundle use case would result in a lot of
+	// unnecessary intermediate size computations.

[tandrii(chromium), 2015/08/11 17:32:22]

agree

+	//
+	// Bundle may return nil if the Bundler has no accumulated data.
+	GetBundles(int64) []*protocol.ButlerLogBundle
+
+	// Size returns the Bundler's current size.
+	Size() int64
+
+	// Empty returns true if the Bundler has no buffered log data.
+	Empty() bool
+}