config services: services.cfg and validation

appengine/components/components/config/endpoint.py

Index: appengine/components/components/config/endpoint.py
diff --git a/appengine/components/components/config/endpoint.py b/appengine/components/components/config/endpoint.py
index e7c9e4ebd31b0cb7f941c0b8c8716993b10c5df8..4da3c09c21abcb30cb562faa9a8ae22c4fd4c74d 100644
--- a/appengine/components/components/config/endpoint.py
+++ b/appengine/components/components/config/endpoint.py
@@ -19,6 +19,10 @@ from . import common
 from . import validation
 
 
+def get_default_rule_set():
+  return validation.DEFAULT_RULE_SET
+
+
 class ConfigSettingsMessage(messages.Message):
   """Configuration service location."""
   # Example: 'luci-config.appspot.com'
@@ -46,6 +50,42 @@ class ValidateResponseMessage(messages.Message):
   messages = messages.MessageField(ValidationMessage, 1, repeated=True)
 
 
+def is_trusted_requester():

[Sergiy Byelozyorov, 2015/07/07 09:10:46]

please add a docstring explaining where this is used

[nodir, 2015/07/07 15:54:17]

Done.

+  if auth.is_admin():
+    return True
+
+  identity = auth.get_current_identity()
+  if identity.is_user:
+    settings = common.ConfigSettings.cached()
+    if settings and settings.service_hostname:
+      parts = settings.service_hostname.split('.')
+      trusted_domains = ('appspot', 'googleplex')
+      # Does not work for tainted versions.
+      if len(parts) > 2 and parts[-1] == 'com' and parts[-2] in trusted_domains:
+        trusted_config_account = (
+            '%s@%s.gserviceaccount.com' % (parts[-3], parts[-2]))
+        if identity.name == trusted_config_account:
+          return True
+
+  return False
+
+
+class ConfigPattern(messages.Message):

[Sergiy Byelozyorov, 2015/07/07 09:10:47]

Why are these classes not generated via protoc from the .proto file?

[nodir, 2015/07/07 15:54:17]

because Cloud Endpoints doesn't work with .proto. This is protorpc, a bit
different

+  """A pattern for one config file. See MetadataResponseMessage."""
+  config_set = messages.StringField(1, required=True)
+  path = messages.StringField(2, required=True)
+
+
+class MetadataResponseMessage(messages.Message):
+  """Compatible with config service's ServiceDynamicMetadata messsage."""
+
+  class Validator(messages.Message):
+    patterns = messages.MessageField(ConfigPattern, 1, repeated=True)
+    url = messages.StringField(2, required=True)
+
+  validation = messages.MessageField(Validator, 1)
+
+
 @auth.endpoints_api(name='config', version='v1', title='Configuration service')
 class ConfigApi(remote.Service):
   """Configuration service."""
@@ -85,3 +125,24 @@ class ConfigApi(remote.Service):
           text=m.text,
       ))
     return res
+
+  @auth.endpoints_method(
+      message_types.VoidMessage, MetadataResponseMessage, path='metadata')
+  @auth.require(is_trusted_requester)
+  def get_metadata(self, _request):
+    """Describes a service. Used by config service to discover other services.

[Sergiy Byelozyorov, 2015/07/07 09:10:46]

Do I understand correctly that this is used by the config service to request
service metadata from some remote endpoint (e.g.
chromium-commit-queue.appspot.com/_ah/api/config/v1/metadata)? Why is this
endpoint defined on the config service itself? Or is this file intended to be
used as a template for endpoint implementation on those other sites?

[nodir, 2015/07/07 15:54:17]

Yes
This is not a config service. This is a config component, used in other apps,
such as buildbucket, CIA.
It is used today:
https://apis-explorer.appspot.com/apis-explorer/?base=https://cr-buildbucket....
https://apis-explorer.appspot.com/apis-explorer/?base=https://chrome-infra-au...

[Sergiy Byelozyorov, 2015/07/07 23:04:38]

Acknowledged.

+    """
+    meta = MetadataResponseMessage()
+    http_headers = dict(self.request_state.headers)
+    assert 'host' in http_headers, http_headers
+    meta.validation = meta.Validator(
+        url='https://{hostname}/_ah/api/{name}/{version}/{path}validate'.format(

[Sergiy Byelozyorov, 2015/07/07 09:10:47]

why does url end with 'validate'?

[nodir, 2015/07/07 15:54:17]

Because that's how the "validate" method in this class is called, and thus this
is the suffix for the HTTP endpoint for validation. This is a URL to an endpoint
that implements validation protocol

[Sergiy Byelozyorov, 2015/07/07 23:04:38]

Acknowledged.

+            hostname=http_headers['host'],
+            name=self.api_info.name,
+            version=self.api_info.version,

[Sergiy Byelozyorov, 2015/07/07 09:10:46]

If this is a template for implementing an endpoint - what happens if the config
API version is upgraded and some other service is still using old version?

[nodir, 2015/07/07 15:54:17]

I've included "version" attribute to the metadata protocol

[Sergiy Byelozyorov, 2015/07/07 23:04:38]

Acknowledged.

+            path=self.api_info.path or '',
+        )
+    )
+    for p in sorted(get_default_rule_set().patterns()):
+      meta.validation.patterns.append(ConfigPattern(**p._asdict()))
+    return meta