Update new botlist to use dimensions endpoint

Update new botlist to use dimensions endpoint

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would only be able to filter based on the things already in the list, which is a problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT] commits.
- Dimensions are passed from the client-side filter to the server-side filter.  The items we have will be filtered out while waiting for the server to give us more. The spinner will be active while any of the ajax requests are active.
- The "devices" dimension was split (server side) into device_type, device_os, and android_devices.  The last one is the number of connected, ready devices.
- Added new method _attribute, which will look for an attribute in the "dimensions" and fallback to "state".  This allows a more consistent "API" than hodge-podge _devices/_state all over the place.
- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a dimension.  It is displayed in a human friendly manner.
- Remove "auto download" of demo json from Makefile
- Sinon was updated to bring in a bug fix.


BUG=631047
Committed: https://github.com/luci/luci-py/commit/bf775c6fcaaa324474208dfcaaac75b2a226789f

[kjlubick, 2016-08-05 19:01:21]

Description was changed from

==========
pull in changes from other branch.  Updated sinon and added a few dimensions

BUG=
==========

to

==========
pull in changes from other branch.  Updated sinon and added a few dimensions

BUG=631047
==========

[kjlubick, 2016-08-08 14:09:57]

Description was changed from

==========
pull in changes from other branch.  Updated sinon and added a few dimensions

BUG=631047
==========

to

==========
pull in changes from other branch.  Updated sinon and added a few dimensions

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.

- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.
- IDs are aggregated so they can still be filtered by.  Other columns may need
this as well
 

BUG=631047
==========

[kjlubick, 2016-08-08 14:11:58]

Description was changed from

==========
pull in changes from other branch.  Updated sinon and added a few dimensions

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.

- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.
- IDs are aggregated so they can still be filtered by.  Other columns may need
this as well
 

BUG=631047
==========

to

==========
pull in changes from other branch.  Updated sinon and added a few dimensions

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.

- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- IDs are aggregated so they can still be filtered by.  Other columns may need
this as well.
 

BUG=631047
==========

[kjlubick, 2016-08-08 14:12:33]

Description was changed from

==========
pull in changes from other branch.  Updated sinon and added a few dimensions

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.

- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- IDs are aggregated so they can still be filtered by.  Other columns may need
this as well.
 

BUG=631047
==========

to

==========
pull in changes from other branch.  Updated sinon and added a few dimensions

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.

- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- IDs are aggregated so they can still be filtered by.  Other columns may need
this as well.
- Sinon was updated to bring in a bug fix.

BUG=631047
==========

[kjlubick, 2016-08-08 14:14:32]

Description was changed from

==========
pull in changes from other branch.  Updated sinon and added a few dimensions

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.

- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- IDs are aggregated so they can still be filtered by.  Other columns may need
this as well.
- Sinon was updated to bring in a bug fix.

BUG=631047
==========

to

==========
pull in changes from other branch.  Updated sinon and added a few dimensions

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.
- The "devices" dimension was split (server side) into device_type, device_os,
and android_devices.  The last one is the number of connected, ready devices.
- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- IDs are aggregated so they can still be filtered by.  Other columns may need
this as well.
- Sinon was updated to bring in a bug fix.

BUG=631047
==========

[kjlubick, 2016-08-08 14:17:43]

Description was changed from

==========
pull in changes from other branch.  Updated sinon and added a few dimensions

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.
- The "devices" dimension was split (server side) into device_type, device_os,
and android_devices.  The last one is the number of connected, ready devices.
- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- IDs are aggregated so they can still be filtered by.  Other columns may need
this as well.
- Sinon was updated to bring in a bug fix.

BUG=631047
==========

to

==========
pull in changes from other branch.  Updated sinon and added a few dimensions

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.
- The "devices" dimension was split (server side) into device_type, device_os,
and android_devices.  The last one is the number of connected, ready devices.
- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- Sinon was updated to bring in a bug fix.

BUG=631047
==========

[kjlubick, 2016-08-08 14:18:28]

Description was changed from

==========
pull in changes from other branch.  Updated sinon and added a few dimensions

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.
- The "devices" dimension was split (server side) into device_type, device_os,
and android_devices.  The last one is the number of connected, ready devices.
- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- Sinon was updated to bring in a bug fix.

BUG=631047
==========

to

==========
Update new botlist to use dimensions endpoint

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.
- The "devices" dimension was split (server side) into device_type, device_os,
and android_devices.  The last one is the number of connected, ready devices.
- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- Sinon was updated to bring in a bug fix.

BUG=631047
==========

[kjlubick, 2016-08-08 14:23:11]

Description was changed from

==========
Update new botlist to use dimensions endpoint

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.
- The "devices" dimension was split (server side) into device_type, device_os,
and android_devices.  The last one is the number of connected, ready devices.
- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- Sinon was updated to bring in a bug fix.

BUG=631047
==========

to

==========
Update new botlist to use dimensions endpoint

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.
- The "devices" dimension was split (server side) into device_type, device_os,
and android_devices.  The last one is the number of connected, ready devices.
- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- Remove "auto download" of demo json from Makefile
- Sinon was updated to bring in a bug fix.


BUG=631047
==========

[kjlubick, 2016-08-08 14:26:47]

Description was changed from

==========
Update new botlist to use dimensions endpoint

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.
- The "devices" dimension was split (server side) into device_type, device_os,
and android_devices.  The last one is the number of connected, ready devices.
- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- Remove "auto download" of demo json from Makefile
- Sinon was updated to bring in a bug fix.


BUG=631047
==========

to

==========
Update new botlist to use dimensions endpoint

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.
- The "devices" dimension was split (server side) into device_type, device_os,
and android_devices.  The last one is the number of connected, ready devices.
- Added new method _attribute, which will look for an attribute in the
"dimensions" and fallback to "state".  This allows a more consistent "API" than
hodge-podge _devices/_state all over the place.
- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- Remove "auto download" of demo json from Makefile
- Sinon was updated to bring in a bug fix.


BUG=631047
==========

[kjlubick, 2016-08-08 15:12:34]

kjlubick@google.com changed reviewers:
+ jcgregorio@google.com, stephana@google.com

[kjlubick, 2016-08-08 15:12:35]

New Dead/quarantined messages:
https://screenshot.googleplex.com/Wegkkor5c63

New disk space columns:
https://screenshot.googleplex.com/STgE4TRF5iK
https://screenshot.googleplex.com/mKTBiRfmm2q

New device-based columns:
https://screenshot.googleplex.com/teVBQZvgu33

Live at 
https://2243-e448e8e-tainted-kjlubick-dot-chromium-swarm-dev.appspot.com/newu...

Better demo data at 
https://goto.google.com/sivef

[jcgregorio, 2016-08-08 16:14:42]

Still reviewing, but sending what I've got now.

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
File appengine/swarming/elements/Makefile (right):

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/Makefile:13: # See gs://luci-demo-data/swarming/ for
*demo.json
I don't have access to that bucket. Is the data considered protected?

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
File appengine/swarming/elements/res/imp/botlist/bot-list-data.html (right):

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-list-data.html:202: _primaryMap:
function(dimensions){
Blank line above.

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-list-data.html:211: // value is
an array of the values the
the values the...?

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
File appengine/swarming/elements/res/imp/botlist/bot-list-demo.html (right):

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-list-demo.html:27:
//server.autoRespondAfter = 2000;
leftover?

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
File appengine/swarming/elements/res/imp/botlist/bot-list-shared.html (right):

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-list-shared.html:89: var a =
ANDROID_ALIASES[dt];
return ANDROID_ALIASES[dt] || UNKNOWN;

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-list-shared.html:169: // it is
put as theonly element in an array.
the only

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
File appengine/swarming/elements/res/js/common.js (right):

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/res/js/common.js:5: window.swarming =
window.swarming || function() {
None of this code depends on the DOM, so it should be possible to create unit
tests for this code that is run through node. Can be done in a later CL.

[kjlubick, 2016-08-08 19:07:10]

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
File appengine/swarming/elements/Makefile (right):

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/Makefile:13: # See gs://luci-demo-data/swarming/ for
*demo.json
On 2016/08/08 at 16:14:42, jcgregorio wrote:
> I don't have access to that bucket. Is the data considered protected?

It's not super protected - those json objects are just the results of calls to
https://apis-explorer.appspot.com/apis-explorer/?base=https://chromium-swarm....
and
https://apis-explorer.appspot.com/apis-explorer/?base=https://chromium-swarm-...
 It's in a bucket that belongs to the chromium-swarm project, because that
seemed like the logical place to put it.  I see you are not a member of that
bucket.  M-A can fix that, if you like.

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
File appengine/swarming/elements/res/imp/botlist/bot-list-data.html (right):

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-list-data.html:202: _primaryMap:
function(dimensions){
On 2016/08/08 at 16:14:42, jcgregorio wrote:
> Blank line above.

Done.

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-list-data.html:211: // value is
an array of the values the
On 2016/08/08 at 16:14:42, jcgregorio wrote:
> the values the...?

Fixed

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
File appengine/swarming/elements/res/imp/botlist/bot-list-demo.html (right):

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-list-demo.html:27:
//server.autoRespondAfter = 2000;
On 2016/08/08 at 16:14:42, jcgregorio wrote:
> leftover?

This is convenient for testing how the demo looks when the api calls are laggy. 
I left it commented out so I don't have to keep going to the sinon API page when
I want to put it back.

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
File appengine/swarming/elements/res/imp/botlist/bot-list-shared.html (right):

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-list-shared.html:89: var a =
ANDROID_ALIASES[dt];
On 2016/08/08 at 16:14:42, jcgregorio wrote:
> return ANDROID_ALIASES[dt] || UNKNOWN;

Done.

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-list-shared.html:147: var a =
GPU_ALIASES[gpu];
Also fixed, like ANDROID_ALIASES

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-list-shared.html:169: // it is
put as theonly element in an array.
On 2016/08/08 at 16:14:42, jcgregorio wrote:
> the only

Done.

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
File appengine/swarming/elements/res/js/common.js (right):

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/res/js/common.js:5: window.swarming =
window.swarming || function() {
On 2016/08/08 at 16:14:42, jcgregorio wrote:
> None of this code depends on the DOM, so it should be possible to create unit
tests for this code that is run through node. Can be done in a later CL.

Changed window -> this, like we do for Skia in
https://github.com/google/skia-buildbot/blob/a5ddd8d63e3aa3b71d7fa4520c451fa1...

[M-A Ruel, 2016-08-08 19:31:22]

maruel@chromium.org changed reviewers:
+ maruel@chromium.org

[M-A Ruel, 2016-08-08 19:31:23]

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
File appengine/swarming/elements/Makefile (right):

https://codereview.chromium.org/2211163003/diff/100001/appengine/swarming/ele...
appengine/swarming/elements/Makefile:13: # See gs://luci-demo-data/swarming/ for
*demo.json
On 2016/08/08 19:07:10, kjlubick wrote:
> On 2016/08/08 at 16:14:42, jcgregorio wrote:
> > I don't have access to that bucket. Is the data considered protected?
> 
> It's not super protected - those json objects are just the results of calls to
>
https://apis-explorer.appspot.com/apis-explorer/?base=https://chromium-swarm....
> and
>
https://apis-explorer.appspot.com/apis-explorer/?base=https://chromium-swarm-...
>  It's in a bucket that belongs to the chromium-swarm project, because that
> seemed like the logical place to put it.  I see you are not a member of that
> bucket.  M-A can fix that, if you like.

Is it large? I'd rather have it committed if not.

[chromium-reviews, 2016-08-08 19:34:41]

bot-list-demo.json is 1.5MB

On Mon, Aug 8, 2016 at 3:31 PM, <maruel@chromium.org> wrote:

>
> https://codereview.chromium.org/2211163003/diff/100001/
> appengine/swarming/elements/Makefile
> File appengine/swarming/elements/Makefile (right):
>
> https://codereview.chromium.org/2211163003/diff/100001/
> appengine/swarming/elements/Makefile#newcode13
> appengine/swarming/elements/Makefile:13: # See
> gs://luci-demo-data/swarming/ for *demo.json
> On 2016/08/08 19:07:10, kjlubick wrote:
> > On 2016/08/08 at 16:14:42, jcgregorio wrote:
> > > I don't have access to that bucket. Is the data considered
> protected?
> >
> > It's not super protected - those json objects are just the results of
> calls to
> >
> https://apis-explorer.appspot.com/apis-explorer/?base=https:
> //chromium-swarm.appspot.com/_ah/api#p/swarming/v1/swarming.bots.list
> > and
> >
> https://apis-explorer.appspot.com/apis-explorer/?base=https:
> //chromium-swarm-dev.appspot.com/_ah/api#p/swarming/v1/
> swarming.bots.dimensions
> > It's in a bucket that belongs to the chromium-swarm project, because
> that
> > seemed like the logical place to put it. I see you are not a member
> of that
> > bucket. M-A can fix that, if you like.
>
> Is it large? I'd rather have it committed if not.
>
> https://codereview.chromium.org/2211163003/
>

-- 
You received this message because you are subscribed to the Google Groups
"Chromium-reviews" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[M-A Ruel, 2016-08-08 19:36:47]

On 2016/08/08 19:34:41, chromium-reviews wrote:
> bot-list-demo.json is 1.5MB

You could store it in luci-py.wiki then:

git clone git@github.com:luci/luci-py.wiki
cd luci-py.wiki
cp .. .
git add .
git ci -am "Add example data."
git push

Don't use chromium specific URLs (including cloud storage) as much as possible.

[kjlubick, 2016-08-08 20:24:03]

On 2016/08/08 at 19:36:47, maruel wrote:
> On 2016/08/08 19:34:41, chromium-reviews wrote:
> > bot-list-demo.json is 1.5MB
> 
> You could store it in luci-py.wiki then:
> 
> git clone git@github.com:luci/luci-py.wiki
> cd luci-py.wiki
> cp .. .
> git add .
> git ci -am "Add example data."
> git push
> 
> Don't use chromium specific URLs (including cloud storage) as much as
possible.

Done.  Makefile will download from there automatically on `make run`

[M-A Ruel, 2016-08-08 20:28:53]

On 2016/08/08 20:24:03, kjlubick wrote:
> On 2016/08/08 at 19:36:47, maruel wrote:
> > On 2016/08/08 19:34:41, chromium-reviews wrote:
> > > bot-list-demo.json is 1.5MB
> > 
> > You could store it in luci-py.wiki then:
> > 
> > git clone git@github.com:luci/luci-py.wiki
> > cd luci-py.wiki
> > cp .. .
> > git add .
> > git ci -am "Add example data."
> > git push
> > 
> > Don't use chromium specific URLs (including cloud storage) as much as
> possible.
> 
> Done.  Makefile will download from there automatically on `make run`

Good, thanks!

[jcgregorio, 2016-08-08 20:54:32]

https://codereview.chromium.org/2211163003/diff/140001/appengine/swarming/ele...
File appengine/swarming/elements/res/imp/botlist/bot-filters.html (right):

https://codereview.chromium.org/2211163003/diff/140001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-filters.html:567: var rest =
f.substring(col.length + FILTER_SEP.length);
Isn't rest = split[1]?

https://codereview.chromium.org/2211163003/diff/140001/appengine/swarming/ele...
File appengine/swarming/elements/res/js/common.js (right):

https://codereview.chromium.org/2211163003/diff/140001/appengine/swarming/ele...
appengine/swarming/elements/res/js/common.js:72: swarming.diffDate = function(s,
now) {
This is all copied from common.js. We should discuss making the sk.* library its
own bower component. We have the same issue with Ravi's Chrome extensions.

[kjlubick, 2016-08-09 12:07:21]

https://codereview.chromium.org/2211163003/diff/140001/appengine/swarming/ele...
File appengine/swarming/elements/res/imp/botlist/bot-filters.html (right):

https://codereview.chromium.org/2211163003/diff/140001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-filters.html:567: var rest =
f.substring(col.length + FILTER_SEP.length);
On 2016/08/08 at 20:54:32, jcgregorio wrote:
> Isn't rest = split[1]?

In Java, yes.  In Go, yes.  In Javascript.... no.  

"gpu:1234:5678".split(":",1)
 ["gpu"]

https://codereview.chromium.org/2211163003/diff/140001/appengine/swarming/ele...
File appengine/swarming/elements/res/js/common.js (right):

https://codereview.chromium.org/2211163003/diff/140001/appengine/swarming/ele...
appengine/swarming/elements/res/js/common.js:72: swarming.diffDate = function(s,
now) {
On 2016/08/08 at 20:54:32, jcgregorio wrote:
> This is all copied from common.js. We should discuss making the sk.* library
its own bower component. We have the same issue with Ravi's Chrome extensions.

Agreed.

[jcgregorio, 2016-08-09 13:26:13]

https://codereview.chromium.org/2211163003/diff/140001/appengine/swarming/ele...
File appengine/swarming/elements/res/imp/botlist/bot-filters.html (right):

https://codereview.chromium.org/2211163003/diff/140001/appengine/swarming/ele...
appengine/swarming/elements/res/imp/botlist/bot-filters.html:567: var rest =
f.substring(col.length + FILTER_SEP.length);
On 2016/08/09 at 12:07:21, kjlubick wrote:
> On 2016/08/08 at 20:54:32, jcgregorio wrote:
> > Isn't rest = split[1]?
> 
> In Java, yes.  In Go, yes.  In Javascript.... no.  
> 
> "gpu:1234:5678".split(":",1)
>  ["gpu"]

"gpu:1234:5678".split(":",2)
["gpu", "1234"]

So there are values that have ":" within them? What are the limitations for
chars allowed in dimension keys and values?

[kjlubick, 2016-08-09 13:35:53]

On 2016/08/09 at 13:26:13, jcgregorio wrote:
>
https://codereview.chromium.org/2211163003/diff/140001/appengine/swarming/ele...
> File appengine/swarming/elements/res/imp/botlist/bot-filters.html (right):
> 
>
https://codereview.chromium.org/2211163003/diff/140001/appengine/swarming/ele...
> appengine/swarming/elements/res/imp/botlist/bot-filters.html:567: var rest =
f.substring(col.length + FILTER_SEP.length);
> On 2016/08/09 at 12:07:21, kjlubick wrote:
> > On 2016/08/08 at 20:54:32, jcgregorio wrote:
> > > Isn't rest = split[1]?
> > 
> > In Java, yes.  In Go, yes.  In Javascript.... no.  
> > 
> > "gpu:1234:5678".split(":",1)
> >  ["gpu"]
> 
> "gpu:1234:5678".split(":",2)
> ["gpu", "1234"]
> 
> So there are values that have ":" within them? What are the limitations for
chars allowed in dimension keys and values?

There does not seem to be prescribed limitations for either: 
https://github.com/luci/luci-py/blob/a181841fc97325ed1ada5621aeda99dd105ec7f1...

By convention, keys seem to be [a-z_]+ and values are [a-zA-Z:\-_]+

[M-A Ruel, 2016-08-09 13:58:36]

On 2016/08/09 13:35:53, kjlubick wrote:
> On 2016/08/09 at 13:26:13, jcgregorio wrote:
> > So there are values that have ":" within them? What are the limitations for
> chars allowed in dimension keys and values?
> 
> There does not seem to be prescribed limitations for either: 
>
https://github.com/luci/luci-py/blob/a181841fc97325ed1ada5621aeda99dd105ec7f1...
> 
> By convention, keys seem to be [a-z_]+ and values are [a-zA-Z:\-_]+

https://github.com/luci/luci-py/blob/master/appengine/swarming/server/task_re...

[kjlubick, 2016-08-09 14:35:09]

On 2016/08/09 at 13:58:36, maruel wrote:
> On 2016/08/09 13:35:53, kjlubick wrote:
> > On 2016/08/09 at 13:26:13, jcgregorio wrote:
> > > So there are values that have ":" within them? What are the limitations
for
> > chars allowed in dimension keys and values?
> > 
> > There does not seem to be prescribed limitations for either: 
> >
https://github.com/luci/luci-py/blob/a181841fc97325ed1ada5621aeda99dd105ec7f1...
> > 
> > By convention, keys seem to be [a-z_]+ and values are [a-zA-Z:\-_]+
> 
>
https://github.com/luci/luci-py/blob/master/appengine/swarming/server/task_re...

But no such regex for the values of dimensions?

[M-A Ruel, 2016-08-09 14:55:33]

On 2016/08/09 14:35:09, kjlubick wrote:
> On 2016/08/09 at 13:58:36, maruel wrote:
> > On 2016/08/09 13:35:53, kjlubick wrote:
> > > On 2016/08/09 at 13:26:13, jcgregorio wrote:
> > > > So there are values that have ":" within them? What are the limitations
> for
> > > chars allowed in dimension keys and values?
> > > 
> > > There does not seem to be prescribed limitations for either: 
> > >
>
https://github.com/luci/luci-py/blob/a181841fc97325ed1ada5621aeda99dd105ec7f1...
> > > 
> > > By convention, keys seem to be [a-z_]+ and values are [a-zA-Z:\-_]+
> > 
> >
>
https://github.com/luci/luci-py/blob/master/appengine/swarming/server/task_re...
> 
> But no such regex for the values of dimensions?

Not at the moment. That's why you always limit the number of split(':') to 1.

[jcgregorio, 2016-08-09 17:05:09]

lgtm

[KevinL, 2016-08-09 18:44:56]

kjlubick@chromium.org changed reviewers:
+ kjlubick@chromium.org

[KevinL, 2016-08-09 18:44:56]

lgtm

[kjlubick, 2016-08-09 18:45:01]

The CQ bit was checked by kjlubick@google.com

[commit-bot: I haz the power, 2016-08-09 18:45:05]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-08-09 18:48:22]

Description was changed from

==========
Update new botlist to use dimensions endpoint

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.
- The "devices" dimension was split (server side) into device_type, device_os,
and android_devices.  The last one is the number of connected, ready devices.
- Added new method _attribute, which will look for an attribute in the
"dimensions" and fallback to "state".  This allows a more consistent "API" than
hodge-podge _devices/_state all over the place.
- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- Remove "auto download" of demo json from Makefile
- Sinon was updated to bring in a bug fix.


BUG=631047
==========

to

==========
Update new botlist to use dimensions endpoint

- Use new dimensions endpoint to get full fleet dimensions.  Before, you would
only be able to filter based on the things already in the list, which is a
problem if the thing you want doesn't show up in the first [BOT_LIST_LIMIT]
commits.
- Dimensions are passed from the client-side filter to the server-side filter. 
The items we have will be filtered out while waiting for the server to give us
more. The spinner will be active while any of the ajax requests are active.
- The "devices" dimension was split (server side) into device_type, device_os,
and android_devices.  The last one is the number of connected, ready devices.
- Added new method _attribute, which will look for an attribute in the
"dimensions" and fallback to "state".  This allows a more consistent "API" than
hodge-podge _devices/_state all over the place.
- Quarantined bot message actually show up (pulled from state).  
- Dead bot message augmented with human friendly "last seen" message.
- Disk space column shown up, which is pulled exclusively from state, not a
dimension.  It is displayed in a human friendly manner.
- Remove "auto download" of demo json from Makefile
- Sinon was updated to bring in a bug fix.


BUG=631047

Committed:
https://github.com/luci/luci-py/commit/bf775c6fcaaa324474208dfcaaac75b2a226789f
==========

[commit-bot: I haz the power, 2016-08-09 18:48:23]

Committed patchset #8 (id:140001) as
https://github.com/luci/luci-py/commit/bf775c6fcaaa324474208dfcaaac75b2a226789f