added fix that deals with precision in layer sorter

cc/layer_sorter.cc

Index: cc/layer_sorter.cc
diff --git a/cc/layer_sorter.cc b/cc/layer_sorter.cc
index e251e11dc5e8d9764bf5997e6233d01a6b5c5667..ece0afb41fa6b0c9b3b7bec71fce1221864d0f70 100644
--- a/cc/layer_sorter.cc
+++ b/cc/layer_sorter.cc
@@ -20,6 +20,13 @@ using WebKit::WebTransformationMatrix;
 
 namespace cc {
 
+// This epsilon is used to determine if two layers are too close to each other
+// to be able to tell which is in front of the other.  It's a relative epsilon
+// so it is robust to changes in scene scale.  This value was chosen by picking
+// a value near machine epsilon and then increasing it until the flickering on
+// the test scene went away.
+const float LAYER_EPSILON = 1e-4f;

[enne (OOO), 2012/11/01 22:55:23]

style nit: kLayerEpsilon

+
 inline static float perpProduct(const gfx::Vector2dF& u, const gfx::Vector2dF& v)
 {
     return u.x() * v.y() - u.y() * v.x();
@@ -114,10 +121,28 @@ LayerSorter::ABCompareResult LayerSorter::checkOverlap(LayerShape* a, LayerShape
     // which layer is in front.
     float maxPositive = 0;
     float maxNegative = 0;
+
+    // This flag tracks the existance of a numerically accurate seperation
+    // between two layers.  If there is no accurate seperation, the layers
+    // cannot be effectively sorted.
+    bool accurate = false;

[shawnsingh, 2012/11/01 23:03:57]

Could we move the computation of "accurate" and associated comments into a
simple trivial helper function?

+
     for (unsigned o = 0; o < overlapPoints.size(); o++) {
         float za = a->layerZFromProjectedPoint(overlapPoints[o]);
         float zb = b->layerZFromProjectedPoint(overlapPoints[o]);
 
+        // Here we attempt to avoid numeric issues with layers that are too
+        // close together.  If we have 2-sided quads that are very close
+        // together then we will draw them in document order to avoid
+        // flickering.  The correct solution is for the content maker to turn
+        // on back-face culling or move the quads apart (if they're not two
+        // sides of one object).
+        float absDif = fabsf(zb - za);
+        float absMax = fmax(fabs(zb), fabs(za));

[shawnsingh, 2012/11/01 23:03:57]

We probably should be consistent in our usage of fabsf() versus fabs()

[danakj, 2012/11/01 23:06:34]

Why aren't we using std::abs() from <cmath> ?

+        // Check to see if we've got a result with a reasonable amount of error.
+        if (absDif > LAYER_EPSILON * absMax)

[shawnsingh, 2012/11/01 23:03:57]

I'm still not understanding exactly how this checks ulp-level precision?  this
seems like a relative threshold based on "within percent accuracy" where
LAYER_EPSILON is that percentage.  I'm pretty sure that's not exactly the same
as ulp-thresholding.  Can we discuss it offline briefly?

That said, it might be acceptable to use relative thresholding here than
ulp-level thresholding.

+          accurate = true;
+
         float diff = za - zb;
         if (diff > maxPositive)
             maxPositive = diff;
@@ -125,6 +150,10 @@ LayerSorter::ABCompareResult LayerSorter::checkOverlap(LayerShape* a, LayerShape
             maxNegative = diff;
     }
 
+    // If we can't tell which should come first, we use document order.
+    if (!accurate)
+      return ABeforeB;
+
     float maxDiff = (fabsf(maxPositive) > fabsf(maxNegative) ? maxPositive : maxNegative);
 
     // If the results are inconsistent (and the z difference substantial to rule out