[Predator] Add MetaWeight and MetaFeatureValue to group multiple weights and features together.

[Predator] Add MetaWeight and MetaFeatureValue to group multiple weights and features together.

MetaWeight, MetaFeature and MetaFeatureValue inherit MetaDict, MetaDict is a dict-like class and its value can be an element or another dict. (e.g {'a': e(1), 'b': {'c': e(2), 'd': e(3)}}).

MetaFeature allows features that share code (e.g. ``TouchCrashedFile``, ``MinDistance``, ``TopFrameIndex``) to be grouped together.

In training MetaDictSerializer is used to serialize MetaWeight and MetaFeature to np arrays.



BUG=679097
TBR=stgao@chromium.org

Review-Url: https://codereview.chromium.org/2625073003
Committed: https://chromium.googlesource.com/infra/infra/+/6ca03f16713d70f5b1cc3696e556d839209ac0ee

[Sharu Jiang, 2017-01-11 21:58:01]

Patchset #1 (id:1) has been deleted

[Sharu Jiang, 2017-01-11 21:59:50]

katesonia@chromium.org changed reviewers:
+ mbarbella@chromium.org, wrengr@chromium.org

[Sharu Jiang, 2017-01-11 21:59:50]

PTAL.

[Sharu Jiang, 2017-01-11 22:00:06]

Description was changed from

==========
[Predator] Add MetaWeight and MetaFeatureValue to allow Feature return a list of
values.

BUG=679097
==========

to

==========
[Predator] Add MetaWeight and MetaFeatureValue to allow Feature return a list of
values.

BUG=679097
TBR=stgao@chromium.org
==========

[Sharu Jiang, 2017-01-18 23:22:07]

Description was changed from

==========
[Predator] Add MetaWeight and MetaFeatureValue to allow Feature return a list of
values.

BUG=679097
TBR=stgao@chromium.org
==========

to

==========
[Predator] Add MetaWeight and MetaFeatureValue to allow Feature return a
collection of values.

BUG=679097
TBR=stgao@chromium.org
==========

[Sharu Jiang, 2017-01-18 23:22:19]

Patchset #1 (id:20001) has been deleted

[Sharu Jiang, 2017-01-18 23:22:24]

Patchset #1 (id:40001) has been deleted

[Sharu Jiang, 2017-01-18 23:22:31]

Patchset #1 (id:60001) has been deleted

[Sharu Jiang, 2017-01-18 23:43:49]

Description was changed from

==========
[Predator] Add MetaWeight and MetaFeatureValue to allow Feature return a
collection of values.

BUG=679097
TBR=stgao@chromium.org
==========

to

==========
[Predator] Add MetaWeight and MetaFeatureValue to allow Feature return a
collection of values.

MetaWeight, MetaFeature and MetaFeatureValue inherit MetaDict, MetaDict is a
dict-like class and it's value can be an element or another dict. (e.g {'a':
e(1), 'b': {'c': e(2), 'd': e(3)}}).

MetaFeature can allow features that share code (e.g. ``TouchCrashedFile``,
``MinDistance``, ``TopFrameIndex``) can be grouped together.

In training MetaDictSerializer is used to serialize MetaWeight and MetaFeature
to np arrays.



BUG=679097
TBR=stgao@chromium.org
==========

[Sharu Jiang, 2017-01-18 23:44:11]

katesonia@chromium.org changed reviewers:
+ chanli@chromium.org, lijeffrey@chromium.org
- wrengr@chromium.org

[Sharu Jiang, 2017-01-18 23:44:23]

PTAL.

[chanli, 2017-01-19 04:26:06]

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
File appengine/findit/crash/loglinear/feature.py (right):

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
appengine/findit/crash/loglinear/feature.py:128: def __len__(self):
Is this necessary? Since Element has the same len?

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
File appengine/findit/crash/loglinear/model.py (right):

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
appengine/findit/crash/loglinear/model.py:50: wrapped_feature: A function ``X ->
Y -> list(FeatureValue)``. N.B.,
Not in argument list above?

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
appengine/findit/crash/loglinear/model.py:69: self._epsilon = epsilon
This if else can be simplified to 

self._epsilon = epsilon if epsilon else EPSILON  # based on docstring epsilon >
0

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
File appengine/findit/crash/loglinear/weight.py (right):

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
appengine/findit/crash/loglinear/weight.py:75: def DropZeroWeights(self,
epsilon=0.):
Not sure if 0. will work here... Maybe it does?

[lijeffrey, 2017-01-19 15:23:35]

lgtm but I don't have enough context for what this CL is for

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
File appengine/findit/crash/loglinear/model.py (right):

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
appengine/findit/crash/loglinear/model.py:69: self._epsilon = epsilon
On 2017/01/19 04:26:06, chanli wrote:
> This if else can be simplified to 
> 
> self._epsilon = epsilon if epsilon else EPSILON  # based on docstring epsilon
>
> 0

even simpler is self._epsilon = epsilon or EPSILON

[Sharu Jiang, 2017-01-19 23:49:52]

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
File appengine/findit/crash/loglinear/feature.py (right):

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
appengine/findit/crash/loglinear/feature.py:128: def __len__(self):
On 2017/01/19 04:26:06, chanli wrote:
> Is this necessary? Since Element has the same len?

Yes, the element can be a list of anything, but as long as it inherits the
Element, the length should be one.

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
File appengine/findit/crash/loglinear/model.py (right):

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
appengine/findit/crash/loglinear/model.py:50: wrapped_feature: A function ``X ->
Y -> list(FeatureValue)``. N.B.,
On 2017/01/19 04:26:06, chanli wrote:
> Not in argument list above?

Oops, should have deleted it.

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
appengine/findit/crash/loglinear/model.py:69: self._epsilon = epsilon
On 2017/01/19 15:23:35, lijeffrey wrote:
> On 2017/01/19 04:26:06, chanli wrote:
> > This if else can be simplified to 
> > 
> > self._epsilon = epsilon if epsilon else EPSILON  # based on docstring
epsilon
> >
> > 0
> 
> even simpler is self._epsilon = epsilon or EPSILON

if epsilon would be False if epsilon equals to zero, so we want to explicitly
check whether it's None or not. But yes, we can make this one line.

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
File appengine/findit/crash/loglinear/weight.py (right):

https://codereview.chromium.org/2625073003/diff/100001/appengine/findit/crash...
appengine/findit/crash/loglinear/weight.py:75: def DropZeroWeights(self,
epsilon=0.):
On 2017/01/19 04:26:06, chanli wrote:
> Not sure if 0. will work here... Maybe it does?

Good catch... I should use "math.fabs(self._value) <= epsilon" instead of
"math.fabs(self._value) < epsilon" in IsZero.

[chanli, 2017-01-22 00:01:28]

lgtm

[Martin Barbella, 2017-01-23 21:04:12]

lgtm

https://codereview.chromium.org/2625073003/diff/120001/appengine/findit/crash...
File appengine/findit/crash/loglinear/changelist_classifier.py (right):

https://codereview.chromium.org/2625073003/diff/120001/appengine/findit/crash...
appengine/findit/crash/loglinear/changelist_classifier.py:140: # features is
``MetaFeatureValue`` object containing all feature values.
Nit: "Features is a", maybe also s/containing/with/ to keep it a one-liner.

https://codereview.chromium.org/2625073003/diff/120001/appengine/findit/libs/...
File appengine/findit/libs/math/vectors.py (right):

https://codereview.chromium.org/2625073003/diff/120001/appengine/findit/libs/...
appengine/findit/libs/math/vectors.py:41: except TypeError:
What case is this meant to handle?

[Sharu Jiang, 2017-01-23 22:58:49]

Description was changed from

==========
[Predator] Add MetaWeight and MetaFeatureValue to allow Feature return a
collection of values.

MetaWeight, MetaFeature and MetaFeatureValue inherit MetaDict, MetaDict is a
dict-like class and it's value can be an element or another dict. (e.g {'a':
e(1), 'b': {'c': e(2), 'd': e(3)}}).

MetaFeature can allow features that share code (e.g. ``TouchCrashedFile``,
``MinDistance``, ``TopFrameIndex``) can be grouped together.

In training MetaDictSerializer is used to serialize MetaWeight and MetaFeature
to np arrays.



BUG=679097
TBR=stgao@chromium.org
==========

to

==========
[Predator] Add MetaWeight and MetaFeatureValue to allow Feature return a
collection of values.

MetaWeight, MetaFeature and MetaFeatureValue inherit MetaDict, MetaDict is a
dict-like class and its value can be an element or another dict. (e.g {'a':
e(1), 'b': {'c': e(2), 'd': e(3)}}).

MetaFeature can allow features that share code (e.g. ``TouchCrashedFile``,
``MinDistance``, ``TopFrameIndex``) can be grouped together.

In training MetaDictSerializer is used to serialize MetaWeight and MetaFeature
to np arrays.



BUG=679097
TBR=stgao@chromium.org
==========

[Sharu Jiang, 2017-01-23 22:59:21]

Description was changed from

==========
[Predator] Add MetaWeight and MetaFeatureValue to allow Feature return a
collection of values.

MetaWeight, MetaFeature and MetaFeatureValue inherit MetaDict, MetaDict is a
dict-like class and its value can be an element or another dict. (e.g {'a':
e(1), 'b': {'c': e(2), 'd': e(3)}}).

MetaFeature can allow features that share code (e.g. ``TouchCrashedFile``,
``MinDistance``, ``TopFrameIndex``) can be grouped together.

In training MetaDictSerializer is used to serialize MetaWeight and MetaFeature
to np arrays.



BUG=679097
TBR=stgao@chromium.org
==========

to

==========
[Predator] Add MetaWeight and MetaFeatureValue to allow Feature return a
collection of values.

MetaWeight, MetaFeature and MetaFeatureValue inherit MetaDict, MetaDict is a
dict-like class and its value can be an element or another dict. (e.g {'a':
e(1), 'b': {'c': e(2), 'd': e(3)}}).

MetaFeature allows features that share code (e.g. ``TouchCrashedFile``,
``MinDistance``, ``TopFrameIndex``) to be grouped together.

In training MetaDictSerializer is used to serialize MetaWeight and MetaFeature
to np arrays.



BUG=679097
TBR=stgao@chromium.org
==========

[Sharu Jiang, 2017-01-23 23:01:08]

Description was changed from

==========
[Predator] Add MetaWeight and MetaFeatureValue to allow Feature return a
collection of values.

MetaWeight, MetaFeature and MetaFeatureValue inherit MetaDict, MetaDict is a
dict-like class and its value can be an element or another dict. (e.g {'a':
e(1), 'b': {'c': e(2), 'd': e(3)}}).

MetaFeature allows features that share code (e.g. ``TouchCrashedFile``,
``MinDistance``, ``TopFrameIndex``) to be grouped together.

In training MetaDictSerializer is used to serialize MetaWeight and MetaFeature
to np arrays.



BUG=679097
TBR=stgao@chromium.org
==========

to

==========
[Predator] Add MetaWeight and MetaFeatureValue to group multiple weights and
features together.

MetaWeight, MetaFeature and MetaFeatureValue inherit MetaDict, MetaDict is a
dict-like class and its value can be an element or another dict. (e.g {'a':
e(1), 'b': {'c': e(2), 'd': e(3)}}).

MetaFeature allows features that share code (e.g. ``TouchCrashedFile``,
``MinDistance``, ``TopFrameIndex``) to be grouped together.

In training MetaDictSerializer is used to serialize MetaWeight and MetaFeature
to np arrays.



BUG=679097
TBR=stgao@chromium.org
==========

[Sharu Jiang, 2017-01-24 01:37:43]

Patchset #4 (id:140001) has been deleted

[Sharu Jiang, 2017-01-24 01:37:50]

The CQ bit was checked by katesonia@chromium.org

[Sharu Jiang, 2017-01-24 01:37:50]

The patchset sent to the CQ was uploaded after l-g-t-m from
lijeffrey@chromium.org, chanli@chromium.org, mbarbella@chromium.org
Link to the patchset: https://codereview.chromium.org/2625073003/#ps160001
(title: "Rebase.")

[commit-bot: I haz the power, 2017-01-24 01:37:52]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-01-24 01:47:48]

CQ is committing da patch.

Bot data: {"patchset_id": 160001, "attempt_start_ts": 1485221870138640,
"parent_rev": "6cd71384d11e3ccaf871d8695da2df9e0605cf08", "commit_rev":
"6ca03f16713d70f5b1cc3696e556d839209ac0ee"}

[commit-bot: I haz the power, 2017-01-24 01:47:52]

Description was changed from

==========
[Predator] Add MetaWeight and MetaFeatureValue to group multiple weights and
features together.

MetaWeight, MetaFeature and MetaFeatureValue inherit MetaDict, MetaDict is a
dict-like class and its value can be an element or another dict. (e.g {'a':
e(1), 'b': {'c': e(2), 'd': e(3)}}).

MetaFeature allows features that share code (e.g. ``TouchCrashedFile``,
``MinDistance``, ``TopFrameIndex``) to be grouped together.

In training MetaDictSerializer is used to serialize MetaWeight and MetaFeature
to np arrays.



BUG=679097
TBR=stgao@chromium.org
==========

to

==========
[Predator] Add MetaWeight and MetaFeatureValue to group multiple weights and
features together.

MetaWeight, MetaFeature and MetaFeatureValue inherit MetaDict, MetaDict is a
dict-like class and its value can be an element or another dict. (e.g {'a':
e(1), 'b': {'c': e(2), 'd': e(3)}}).

MetaFeature allows features that share code (e.g. ``TouchCrashedFile``,
``MinDistance``, ``TopFrameIndex``) to be grouped together.

In training MetaDictSerializer is used to serialize MetaWeight and MetaFeature
to np arrays.



BUG=679097
TBR=stgao@chromium.org

Review-Url: https://codereview.chromium.org/2625073003
Committed:
https://chromium.googlesource.com/infra/infra/+/6ca03f16713d70f5b1cc3696e556d...
==========

[commit-bot: I haz the power, 2017-01-24 01:47:54]

Committed patchset #4 (id:160001) as
https://chromium.googlesource.com/infra/infra/+/6ca03f16713d70f5b1cc3696e556d...