Use luci-config for infrequently changing settings, part 2.

appengine/auth_service/proto/config.proto

+// Copyright 2015 The Swarming Authors. All rights reserved.
+// Use of this source code is governed by the Apache v2.0 license that can be
+// found in the LICENSE file.
+
+// Configuration schema for configs fetched via luci-config.

[nodir, 2015/06/02 00:19:20]

very nit: say "config service" in code hosted in luci repo

[Vadim Sh., 2015/06/02 00:52:17]

Done.

+
+package auth_service;
+
+
+// Configuration of cron job that imports groups from external sources.
+message GroupImporterConfig {
+  // Import groups stored as files in a tarball.
+  message TarballEntry {
+    // Where to import data from.
+    optional string url = 1;
+    // List of OAuth scopes to use for authentication (or empty to skip auth).
+    repeated string oauth_scopes = 2;
+    // Email domain to append to imported identities.
+    optional string domain = 3;
+    // List of group systems expected to be found in the archive. They act as
+    // prefixes to group names, e.g 'ldap'. Each system corresponds to
+    // a subdirectory in the tarball.

[nodir, 2015/06/02 00:19:20]

In your particular import.cfg, groups are full filenames to the group files. All
of them contain the same prefix,  equal to the value of "systems". I am not sure
I understand why we can't have only groups will full names without systems.

If we still need, at this level of abstraction, "systems" seems to be too
specific to a particular implemention of group export implementation. Should it
be just "prefix"?

[Vadim Sh., 2015/06/02 00:52:17]

tl;dr It is not just prefix.

See module docstring in importer.py: if tarball claims to import groups of some
system it must contain ALL groups belonging to that system. In that case not
only groups synchronized by their absence too. For example if ldap.tar.gz no
longer contains file ldap/some-group, this group will be deleted from the
service too.

Actually "systems" appeared first. "groups" were added later to avoiding
importing crap that we do not really need.

Copied chunk of this docstring here.

Re "seems to be too specific to a particular implemention of group export
implementation": it isn't meant to be super generic and flexible. YAGNI.

[Vadim Sh., 2015/06/02 00:54:17]

Err.. "In that case not only groups are synchronized, but their absence too"

[nodir, 2015/06/02 16:52:29]

Acknowledged.

+    repeated string systems = 4;
+    // List of groups to import from the tarball. If empty, imports all groups.

[nodir, 2015/06/02 00:19:20]

Are these full filenames relative to the root of the tar archive? If yes, please
mention that

[Vadim Sh., 2015/06/02 00:52:17]

Done.

+    repeated string groups = 5;
+  }
+
+  // Import a single group stored as a plain list of identities.
+  message PlainlistEntry {
+    // Where to import data from.
+    optional string url = 1;
+    // List of OAuth scopes to use for authentication (or empty to skip auth).
+    repeated string oauth_scopes = 2;
+    // Email domain to append to imported identities.
+    optional string domain = 3;
+    // For PLAINLIST imports, a name of imported group. The full group name will
+    // be 'external/<group>'.
+    optional string group = 4;

[nodir, 2015/06/02 00:19:21]

Why you are putting "external" in plainlist and not in tarbarlls? Maybe we
should use common "prefix" or get rid of it altogether?

[Vadim Sh., 2015/06/02 00:52:17]

external/* is special - it is reserved for groups imported via "plainlist"
import config. Again, it is not just a prefix.

The general rule: group names <something>/<something> are imported. If
<something> is "external", it is imported as a plain list. I prefer not to use
long paths (<something>/<something>/<something>).

[nodir, 2015/06/02 16:52:29]

Acknowledged.

+  }
+
+  repeated TarballEntry tarball = 1;
+  repeated PlainlistEntry plainlist = 2;
+}