[Findit & Predator] Code reorg of Findit.

[Findit & Predator] Code reorg of Findit.

In order to avoid confusion between "Findit for Waterfall" and "Findit for Crashes", we plan to split the original Findit project into two parts:
-Findit: the portion of "Findit for Waterfall" in the original Findit.
-Predator: the portion of "Findit for Crashes" in the original Findit.

The containing project is named "Chrome Culprit Finder".
"Chrome Culprit Finder" = Findit + Predator

We will clean up the code first, and then clean up all related docs and site pages to reflect this change.
Hopefully after that, there won't be confusion any more :)


The findit_mock/ here is to avoid touching stuff in the current findit/.

[stgao, 2016-10-26 00:30:30]

Description was changed from

==========
[Findit] Code reorg.

BUG=
==========

to

==========
[Findit & Predator] Code reorg.

BUG=
==========

[stgao, 2016-11-04 02:22:39]

Description was changed from

==========
[Findit & Predator] Code reorg.

BUG=
==========

to

==========
[Findit & Predator] Code reorg of Findit.

In order to avoid confusion between "Findit for Waterfall" and "Findit for
Crashes", we plan to split the original Findit project into two parts:
-Findit: the portion of "Findit for Waterfall" in the original Findit.
-Predator: the portion of "Findit for Crashes" in the original Findit.

The containing project is named "Chrome Culprit Finder".
"Chrome Culprit Finder" = Findit + Predator

BUG=
==========

[stgao, 2016-11-04 02:23:19]

Description was changed from

==========
[Findit & Predator] Code reorg of Findit.

In order to avoid confusion between "Findit for Waterfall" and "Findit for
Crashes", we plan to split the original Findit project into two parts:
-Findit: the portion of "Findit for Waterfall" in the original Findit.
-Predator: the portion of "Findit for Crashes" in the original Findit.

The containing project is named "Chrome Culprit Finder".
"Chrome Culprit Finder" = Findit + Predator

BUG=
==========

to

==========
[Findit & Predator] Code reorg of Findit.

In order to avoid confusion between "Findit for Waterfall" and "Findit for
Crashes", we plan to split the original Findit project into two parts:
-Findit: the portion of "Findit for Waterfall" in the original Findit.
-Predator: the portion of "Findit for Crashes" in the original Findit.

The containing project is named "Chrome Culprit Finder".
"Chrome Culprit Finder" = Findit + Predator

We will clean up the code first, and then clean up all related docs and site
pages to reflect this change.

BUG=
==========

[stgao, 2016-11-04 02:23:44]

Description was changed from

==========
[Findit & Predator] Code reorg of Findit.

In order to avoid confusion between "Findit for Waterfall" and "Findit for
Crashes", we plan to split the original Findit project into two parts:
-Findit: the portion of "Findit for Waterfall" in the original Findit.
-Predator: the portion of "Findit for Crashes" in the original Findit.

The containing project is named "Chrome Culprit Finder".
"Chrome Culprit Finder" = Findit + Predator

We will clean up the code first, and then clean up all related docs and site
pages to reflect this change.

BUG=
==========

to

==========
[Findit & Predator] Code reorg of Findit.

In order to avoid confusion between "Findit for Waterfall" and "Findit for
Crashes", we plan to split the original Findit project into two parts:
-Findit: the portion of "Findit for Waterfall" in the original Findit.
-Predator: the portion of "Findit for Crashes" in the original Findit.

The containing project is named "Chrome Culprit Finder".
"Chrome Culprit Finder" = Findit + Predator

We will clean up the code first, and then clean up all related docs and site
pages to reflect this change.
Hopefully after that, there won't be confusion any more :)
==========

[stgao, 2016-11-04 02:24:28]

Description was changed from

==========
[Findit & Predator] Code reorg of Findit.

In order to avoid confusion between "Findit for Waterfall" and "Findit for
Crashes", we plan to split the original Findit project into two parts:
-Findit: the portion of "Findit for Waterfall" in the original Findit.
-Predator: the portion of "Findit for Crashes" in the original Findit.

The containing project is named "Chrome Culprit Finder".
"Chrome Culprit Finder" = Findit + Predator

We will clean up the code first, and then clean up all related docs and site
pages to reflect this change.
Hopefully after that, there won't be confusion any more :)
==========

to

==========
[Findit & Predator] Code reorg of Findit.

In order to avoid confusion between "Findit for Waterfall" and "Findit for
Crashes", we plan to split the original Findit project into two parts:
-Findit: the portion of "Findit for Waterfall" in the original Findit.
-Predator: the portion of "Findit for Crashes" in the original Findit.

The containing project is named "Chrome Culprit Finder".
"Chrome Culprit Finder" = Findit + Predator

We will clean up the code first, and then clean up all related docs and site
pages to reflect this change.
Hopefully after that, there won't be confusion any more :)


The findit_mock/ here is to avoid touching stuff in the current findit/.
==========

[stgao, 2016-11-04 02:26:06]

stgao@chromium.org changed reviewers:
+ chanli@chromium.org, inferno@chromium.org, katesonia@chromium.org,
lijeffrey@chromium.org, mbarbella@chromium.org, wrengr@chromium.org

[stgao, 2016-11-04 02:26:07]

Hey guys,

Mind a quick look at the code layout as discussed on Tuesday?

Thanks,
Shuotao

[Sharu Jiang, 2016-11-04 18:39:46]

https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/l...
File appengine/findit_mock/libs/cache/cache.py (right):

https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/l...
appengine/findit_mock/libs/cache/cache.py:1: import functools
I think some part of cache is gae related, like *MemCacher, maybe we should
split it to an general interface and MemCachers, and put those MemCachers to
gae_libs?

https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/s...
File appengine/findit_mock/scripts/run.sh (right):

https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/s...
appengine/findit_mock/scripts/run.sh:2: #
We may also have some code in scripts to share between findit and predator,
should we make something like script_util/?

https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
File appengine/predator/app/common/model/crash_analysis.py (right):

https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
appengine/predator/app/common/model/crash_analysis.py:1: from
google.appengine.ext import ndb
So the common/ here is for things shared by fracas/cracas/clusterfuzz? It seems
a little too fine-grained. If we do this to model, should we also do this to
handler and template?

https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
File appengine/predator/app/frontend/handlers/report_crash.py (right):

https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
appengine/predator/app/frontend/handlers/report_crash.py:1: # Copyright 2016 The
Chromium Authors. All rights reserved.
I think we should also have frontend/template/

https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/firs...
File appengine/predator/first_party/gae_libs (right):

https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/firs...
appengine/predator/first_party/gae_libs:1: ../../findit_mock/gae_libs/
The first_party/ means from findit, right? Maybe  we to remove the /first_party/
to be simpler :)

[stgao, 2016-11-08 18:53:32]

If there is no other concern, I will mirror this structure to Findit and we
could start the file moving/splitting/etc.

https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/l...
File appengine/findit_mock/libs/cache/cache.py (right):

https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/l...
appengine/findit_mock/libs/cache/cache.py:1: import functools
On 2016/11/04 18:39:46, Sharu Jiang wrote:
> I think some part of cache is gae related, like *MemCacher, maybe we should
> split it to an general interface and MemCachers, and put those MemCachers to
> gae_libs?

Yes, that's the plan.

https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/s...
File appengine/findit_mock/scripts/run.sh (right):

https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/s...
appengine/findit_mock/scripts/run.sh:2: #
On 2016/11/04 18:39:46, Sharu Jiang wrote:
> We may also have some code in scripts to share between findit and predator,
> should we make something like script_util/?

Good point. Will add.

https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
File appengine/predator/app/common/model/crash_analysis.py (right):

https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
appengine/predator/app/common/model/crash_analysis.py:1: from
google.appengine.ext import ndb
On 2016/11/04 18:39:46, Sharu Jiang wrote:
> So the common/ here is for things shared by fracas/cracas/clusterfuzz? It
seems
> a little too fine-grained. If we do this to model, should we also do this to
> handler and template? 

These are the models/etc that have to be shared between the frontend and
backend.

https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
File appengine/predator/app/frontend/handlers/report_crash.py (right):

https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
appengine/predator/app/frontend/handlers/report_crash.py:1: # Copyright 2016 The
Chromium Authors. All rights reserved.
On 2016/11/04 18:39:46, Sharu Jiang wrote:
> I think we should also have frontend/template/

Yes, we could add when an HTML template is added.

https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/firs...
File appengine/predator/first_party/gae_libs (right):

https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/firs...
appengine/predator/first_party/gae_libs:1: ../../findit_mock/gae_libs/
On 2016/11/04 18:39:46, Sharu Jiang wrote:
> The first_party/ means from findit, right? Maybe  we to remove the
/first_party/
> to be simpler :)

It is more than those from Findit.
We need others like testing_utils that are shared in the infra repo.

[Sharu Jiang, 2016-11-08 19:10:16]

On 2016/11/08 18:53:32, stgao (slow on Monday) wrote:
> If there is no other concern, I will mirror this structure to Findit and we
> could start the file moving/splitting/etc.
> 
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/l...
> File appengine/findit_mock/libs/cache/cache.py (right):
> 
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/l...
> appengine/findit_mock/libs/cache/cache.py:1: import functools
> On 2016/11/04 18:39:46, Sharu Jiang wrote:
> > I think some part of cache is gae related, like *MemCacher, maybe we should
> > split it to an general interface and MemCachers, and put those MemCachers to
> > gae_libs?
> 
> Yes, that's the plan.
> 
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/s...
> File appengine/findit_mock/scripts/run.sh (right):
> 
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/s...
> appengine/findit_mock/scripts/run.sh:2: #
> On 2016/11/04 18:39:46, Sharu Jiang wrote:
> > We may also have some code in scripts to share between findit and predator,
> > should we make something like script_util/?
> 
> Good point. Will add.
> 
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
> File appengine/predator/app/common/model/crash_analysis.py (right):
> 
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
> appengine/predator/app/common/model/crash_analysis.py:1: from
> google.appengine.ext import ndb
> On 2016/11/04 18:39:46, Sharu Jiang wrote:
> > So the common/ here is for things shared by fracas/cracas/clusterfuzz? It
> seems
> > a little too fine-grained. If we do this to model, should we also do this to
> > handler and template? 
> 
> These are the models/etc that have to be shared between the frontend and
> backend.
> 
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
> File appengine/predator/app/frontend/handlers/report_crash.py (right):
> 
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
> appengine/predator/app/frontend/handlers/report_crash.py:1: # Copyright 2016
The
> Chromium Authors. All rights reserved.
> On 2016/11/04 18:39:46, Sharu Jiang wrote:
> > I think we should also have frontend/template/
> 
> Yes, we could add when an HTML template is added.
> 
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/firs...
> File appengine/predator/first_party/gae_libs (right):
> 
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/firs...
> appengine/predator/first_party/gae_libs:1: ../../findit_mock/gae_libs/
> On 2016/11/04 18:39:46, Sharu Jiang wrote:
> > The first_party/ means from findit, right? Maybe  we to remove the
> /first_party/
> > to be simpler :)
> 
> It is more than those from Findit.
> We need others like testing_utils that are shared in the infra repo.

I have a question, why does predator have frontend/ and backend/, however findit
doesn't?

[stgao, 2016-11-08 19:34:34]

On 2016/11/08 19:10:16, Sharu Jiang wrote:
> On 2016/11/08 18:53:32, stgao (slow on Monday) wrote:
> > If there is no other concern, I will mirror this structure to Findit and we
> > could start the file moving/splitting/etc.
> > 
> >
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/l...
> > File appengine/findit_mock/libs/cache/cache.py (right):
> > 
> >
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/l...
> > appengine/findit_mock/libs/cache/cache.py:1: import functools
> > On 2016/11/04 18:39:46, Sharu Jiang wrote:
> > > I think some part of cache is gae related, like *MemCacher, maybe we
should
> > > split it to an general interface and MemCachers, and put those MemCachers
to
> > > gae_libs?
> > 
> > Yes, that's the plan.
> > 
> >
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/s...
> > File appengine/findit_mock/scripts/run.sh (right):
> > 
> >
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/findit_mock/s...
> > appengine/findit_mock/scripts/run.sh:2: #
> > On 2016/11/04 18:39:46, Sharu Jiang wrote:
> > > We may also have some code in scripts to share between findit and
predator,
> > > should we make something like script_util/?
> > 
> > Good point. Will add.
> > 
> >
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
> > File appengine/predator/app/common/model/crash_analysis.py (right):
> > 
> >
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
> > appengine/predator/app/common/model/crash_analysis.py:1: from
> > google.appengine.ext import ndb
> > On 2016/11/04 18:39:46, Sharu Jiang wrote:
> > > So the common/ here is for things shared by fracas/cracas/clusterfuzz? It
> > seems
> > > a little too fine-grained. If we do this to model, should we also do this
to
> > > handler and template? 
> > 
> > These are the models/etc that have to be shared between the frontend and
> > backend.
> > 
> >
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
> > File appengine/predator/app/frontend/handlers/report_crash.py (right):
> > 
> >
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/app/...
> > appengine/predator/app/frontend/handlers/report_crash.py:1: # Copyright 2016
> The
> > Chromium Authors. All rights reserved.
> > On 2016/11/04 18:39:46, Sharu Jiang wrote:
> > > I think we should also have frontend/template/
> > 
> > Yes, we could add when an HTML template is added.
> > 
> >
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/firs...
> > File appengine/predator/first_party/gae_libs (right):
> > 
> >
>
https://codereview.chromium.org/2447253002/diff/60001/appengine/predator/firs...
> > appengine/predator/first_party/gae_libs:1: ../../findit_mock/gae_libs/
> > On 2016/11/04 18:39:46, Sharu Jiang wrote:
> > > The first_party/ means from findit, right? Maybe  we to remove the
> > /first_party/
> > > to be simpler :)
> > 
> > It is more than those from Findit.
> > We need others like testing_utils that are shared in the infra repo.
> 
> I have a question, why does predator have frontend/ and backend/, however
findit
> doesn't?

For Findit, we will clean things up after Predator is split out.
That would make it easier :)

[Sharu Jiang, 2016-11-08 23:27:14]

lgtm

[stgao, 2016-11-10 18:49:00]

As no more concern came in, I went ahead and mirrored this code structure in CL
https://codereview.chromium.org/2488113005/

For sharing util scripts, let's do it separately in a follow CL after figuring
out how much code should be shared.

[stgao, 2016-11-10 18:49:32]

not lgtm to avoid commit of this CL as it is for demo purpose.

[stgao, 2016-11-10 18:50:07]

On 2016/11/10 18:49:32, stgao (slow on Monday) wrote:
> not lgtm to avoid commit of this CL as it is for demo purpose.

seems not work, thus close it.