Split HttpCache miss tracking by Open and Create

Split HttpCache miss tracking by Open and Create

Missing on the HTTP cache requires two blocking operations on the disk cache. This CL splits those out; the idea is to figure out if racing on Create is a worthwile task.

R=rvargas@chromium.org
BUG=None

[gavinp, 2012-12-19 12:38:19]

Also with a bonus bug fix: we were overcounting misses, and measuring them as
too performant.

[rvargas (doing something else), 2012-12-20 03:25:29]

I'm seriously more inclined to remove code (and histograms!) than to add. What
is the new histogram really going to tell us? That create may be expensive? That
open can be expensive? What is it that we don't know? How is that going to
support the argument about racing creates?

[gavinp, 2012-12-20 03:48:26]

On 2012/12/20 03:25:29, rvargas wrote:
> I'm seriously more inclined to remove code (and histograms!) than to add. What
> is the new histogram really going to tell us? That create may be expensive?
That
> open can be expensive? What is it that we don't know? How is that going to
> support the argument about racing creates?

Yes and no.

In the miss case, there's no good reason that we do either open or create. And
so sure, why bother measuring the two since we shouldn't be doing either.

Even worse, let's say open is more expensive than create. Well... The problem is
that open is cache warming; so you can't necessarily save the time from open by
omitting a create. So probably data from the zero latency cache is more
interesting.

OTOH, we are arguing here about five lines of code, post branch, to measure two
very important blocking operations on one of the most common use cases... And
these five lines of code give us that split.

[gavinp, 2012-12-20 04:30:41]

On 2012/12/20 03:25:29, rvargas wrote:
> I'm seriously more inclined to remove code (and histograms!) than to add. What
> is the new histogram really going to tell us? That create may be expensive?
That
> open can be expensive? What is it that we don't know? How is that going to
> support the argument about racing creates?

More directly answering the question: racing creates saves the cost of create.
Measuring create separately from create means we can know the savings.

Ricardo, I'd love to hear you explain the downside of landing a short commit
like this more clearly. You've said you're inclined to remove code, but what's
the argument against this CL? What's the benefit you see? What's the cost?

[rvargas (doing something else), 2012-12-20 07:31:11]

On 2012/12/20 04:30:41, gavinp wrote:
> On 2012/12/20 03:25:29, rvargas wrote:
> > I'm seriously more inclined to remove code (and histograms!) than to add.
What
> > is the new histogram really going to tell us? That create may be expensive?
> That
> > open can be expensive? What is it that we don't know? How is that going to
> > support the argument about racing creates?
> 
> More directly answering the question: racing creates saves the cost of create.
> Measuring create separately from create means we can know the savings.
> 
> Ricardo, I'd love to hear you explain the downside of landing a short commit
> like this more clearly. You've said you're inclined to remove code, but what's
> the argument against this CL? What's the benefit you see? What's the cost?

I have read your first reply three times and I'm still not sure if you are
agreeing that we don't need this CL or not... I read it as "yes I agree", but
then you ask why are we arguing :(

I see no particular issue with the CL... it's just a few extra lines of code
with no evident problem. My issue is more philosophical: I don't see the result
providing any information that we don't know already, or directly influencing a
decision that we have to make. I have the same issue with a lot of the current
histograms, and on aggregate they are not five lines of code, they are more like
lines everywhere that we have to worry about now, and that distract from the
code that actually cares about the "real" logic. That's why I said that I much
rather start cleaning up than adding more histograms... unless of course the new
histogram answers a question that we have no way of answering at the moment.

[gavinp, 2012-12-20 14:14:25]

On 2012/12/20 07:31:11, rvargas wrote:
> On 2012/12/20 04:30:41, gavinp wrote:
> > On 2012/12/20 03:25:29, rvargas wrote:
> > > I'm seriously more inclined to remove code (and histograms!) than to add.
> What
> > > is the new histogram really going to tell us? That create may be
expensive?
> > That
> > > open can be expensive? What is it that we don't know? How is that going to
> > > support the argument about racing creates?
> > 
> > More directly answering the question: racing creates saves the cost of
create.
> > Measuring create separately from create means we can know the savings.
> > 
> > Ricardo, I'd love to hear you explain the downside of landing a short commit
> > like this more clearly. You've said you're inclined to remove code, but
what's
> > the argument against this CL? What's the benefit you see? What's the cost?
> 
> I have read your first reply three times and I'm still not sure if you are
> agreeing that we don't need this CL or not... I read it as "yes I agree", but
> then you ask why are we arguing :(

I agree with all the counterpoints. On balance though, I do think the
investigative benefit outweighs the costs.

> 
> I see no particular issue with the CL... it's just a few extra lines of code
> with no evident problem. My issue is more philosophical: I don't see the
result
> providing any information that we don't know already, or directly influencing
a
> decision that we have to make. I have the same issue with a lot of the current
> histograms, and on aggregate they are not five lines of code, they are more
like
> lines everywhere that we have to worry about now, and that distract from the
> code that actually cares about the "real" logic. That's why I said that I much
> rather start cleaning up than adding more histograms... unless of course the
new
> histogram answers a question that we have no way of answering at the moment.

All I know is that misses take longer than they should; there's two fundamental
operations that slow them down, and this CL splits them in two. I suspect that
Open is the more expensive operation, although yesterday morning I thought it
was Create. I also know that there's two design documents I circulated to
eliminate each of these two operations; it would be nice to have some priority
on those approaches. The reason I don't know why we're arguing: we're still in
the investigative stage of trying to make our cache faster, the bar shouldn't be
very high for adding metrics. The time to remove them is after we've made
decisions and started working on them.

[gavinp, 2012-12-26 15:09:19]

On 2012/12/20 14:14:25, gavinp wrote:
> On 2012/12/20 07:31:11, rvargas wrote:
Ricardo, thanks for gchatting on Friday. I'm now closing this issue as we
discussed then. We agree that there's little cost to adding this measurement,
and as well you have told me that you will not approve this patch.

As discussed above, the first idea motivating this measurement was that it would
be of use perhaps in prioritizing the two current fast miss designs going
around; the bitmap in IO thread proposal and the race on create proposal. You've
indicated that you don't like race on create in the chat (although you haven't
commented on the document: will you do that?), but not the bitmap... But of
course since OpenEntry warms the disk cache for CreateEntry, it's not clear that
time saved by avoiding a blocking OpenEntry actually saves time in a blocking
CreateEntry.

I remain interested in the cost of blocking cache operations; another area we
talked about on Friday. I'll upload CLs that focus on the cost of the operation
queue as time permits.

> > On 2012/12/20 04:30:41, gavinp wrote:
> > > On 2012/12/20 03:25:29, rvargas wrote:
> > > > I'm seriously more inclined to remove code (and histograms!) than to
add.
> > What
> > > > is the new histogram really going to tell us? That create may be
> expensive?
> > > That
> > > > open can be expensive? What is it that we don't know? How is that going
to
> > > > support the argument about racing creates?
> > > 
> > > More directly answering the question: racing creates saves the cost of
> create.
> > > Measuring create separately from create means we can know the savings.
> > > 
> > > Ricardo, I'd love to hear you explain the downside of landing a short
commit
> > > like this more clearly. You've said you're inclined to remove code, but
> what's
> > > the argument against this CL? What's the benefit you see? What's the cost?
> > 
> > I have read your first reply three times and I'm still not sure if you are
> > agreeing that we don't need this CL or not... I read it as "yes I agree",
but
> > then you ask why are we arguing :(
> 
> I agree with all the counterpoints. On balance though, I do think the
> investigative benefit outweighs the costs.
> 
> > 
> > I see no particular issue with the CL... it's just a few extra lines of code
> > with no evident problem. My issue is more philosophical: I don't see the
> result
> > providing any information that we don't know already, or directly
influencing
> a
> > decision that we have to make. I have the same issue with a lot of the
current
> > histograms, and on aggregate they are not five lines of code, they are more
> like
> > lines everywhere that we have to worry about now, and that distract from the
> > code that actually cares about the "real" logic. That's why I said that I
much
> > rather start cleaning up than adding more histograms... unless of course the
> new
> > histogram answers a question that we have no way of answering at the moment.
> 
> All I know is that misses take longer than they should; there's two
fundamental
> operations that slow them down, and this CL splits them in two. I suspect that
> Open is the more expensive operation, although yesterday morning I thought it
> was Create. I also know that there's two design documents I circulated to
> eliminate each of these two operations; it would be nice to have some priority
> on those approaches. The reason I don't know why we're arguing: we're still in
> the investigative stage of trying to make our cache faster, the bar shouldn't
be
> very high for adding metrics. The time to remove them is after we've made
> decisions and started working on them.