Python Tools for Pixel-by-Pixel Image Comparison

chrome/test/functional/ispy/image_tools.py

Index: chrome/test/functional/ispy/image_tools.py
diff --git a/chrome/test/functional/ispy/image_tools.py b/chrome/test/functional/ispy/image_tools.py
new file mode 100644
index 0000000000000000000000000000000000000000..571cad5d6f9fecd734441b33059ec53ac78a3aad
--- /dev/null
+++ b/chrome/test/functional/ispy/image_tools.py
@@ -0,0 +1,295 @@
+# Copyright (c) 2013 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+"""A Module containing useful image tools for use by I Spy."""

[craigdh, 2013/06/13 19:28:31]

"""Utilities for performing pixel-by-pixel image comparisons."""

[cgrimm, 2013/06/14 18:43:58]

Done.

+
+__author__ = 'cgrimm@google.com (Chris Grimm)'

[craigdh, 2013/06/13 19:28:31]

We don't do this in chromium.

[cgrimm, 2013/06/14 18:43:58]

Done.

+
+from itertools import izip

[craigdh, 2013/06/13 19:28:31]

Prefer importing the module and then calling module.funct()

[cgrimm, 2013/06/14 18:43:58]

Done.

+from PIL import Image

[craigdh, 2013/06/13 19:28:31]

ditto

[cgrimm, 2013/06/14 18:43:58]

Done.

+
+
+class Error(Exception):

[craigdh, 2013/06/13 19:28:31]

This adds little additional meaning. Just use Exception directly (or a more
descriptive built-in exception like ValueError). If you want specific errors so
they can be caught, create more specific errors with clear names.

[cgrimm, 2013/06/14 18:43:58]

Done.

+  pass
+
+
+def AreTheSameSize(*images):

[craigdh, 2013/06/13 19:28:31]

Just pass the list by value. The contents of the list will still be references
to the original images.

[cgrimm, 2013/06/14 18:43:58]

Done.

+  """Returns if a set of images are the same size.
+
+  Args:
+    *images: a list of images to compare
+  Returns:
+    boolean
+  Raises:
+    Error: One image or fewer is passed in.
+  """
+  if len(images) > 1:
+    i1 = images[0]
+    for i2 in images:
+      if i1.size == i2.size:

[craigdh, 2013/06/13 19:28:31]

No need for the pass, just do:

if i1.size != i2.size:
  return false

[cgrimm, 2013/06/14 18:43:58]

Done.

+        pass
+      else:
+        return False
+    return True

[craigdh, 2013/06/13 19:28:31]

Actually, you can probably do this whole function using any() instead of a loop.
I think this one-liner should work:

return not any(images[0].size != img.size for img in images[1:])

[cgrimm, 2013/06/14 18:43:58]

Done.

+  else:
+    raise Error('No images passed in')
+
+
+def GetColorDist(px1, px2):

[craigdh, 2013/06/13 19:28:31]

Functions local to the module should be prefixed with an _

[cgrimm, 2013/06/14 18:43:58]

Done.

+  """Returns the normalized color distance between pixels.
+
+  This function gets the color distance between two pixels and
+  returns a number between 0 and 1 where 0 is lowest distance
+  and 1 is the greatest distance.
+
+  Args:
+    px1: a 3-tuple (R,G,B)
+    px2: a 3-tuple (R,G,B)
+  Returns:
+    a number between 0 and 1
+  """
+  dist = 0.
+  for i in range(3):
+    dist += (px1[i]-px2[i])**2

[craigdh, 2013/06/13 19:28:31]

Use sum() for efficiency as this is in the critical inner loop.

[cgrimm, 2013/06/14 18:43:58]

Done.

+  return dist/(3*255**2)

[craigdh, 2013/06/13 19:28:31]

Does python calculate this constant every time? I'm not sure. If so, maybe put
it in a constant at the top of the file.

[cgrimm, 2013/06/14 18:43:58]

Done.

+
+
+def GetColorDistAsColor(px1, px2):
+  """Computes Color Distance as a greyscale pixel.
+
+  This function gets the color distance between two pixels
+  represented as a pixel ranging in color from (0,0,0) to
+  (255,255,255).
+
+  Args:
+    px1: a 3-tuple (R,G,B)
+    px2: a 3-tuple (R,G,B)
+  Returns:
+    a 3-tuple between (0,0,0) and (255,255,255)
+  """
+  n = GetColorDist(px1, px2)
+  return (int(255*n), int(255*n), int(255*n))
+
+
+def Brightness(px):
+  """gets the brightness of a pixel.

[craigdh, 2013/06/13 19:28:31]

g -> G

[cgrimm, 2013/06/14 18:43:58]

Done.

+
+  returns the sum of a given pixel's color channels.

[craigdh, 2013/06/13 19:28:31]

r -> R

[cgrimm, 2013/06/14 18:43:58]

Done.

+
+  Args:
+    px: a 3-tuple (R,G,B)
+  Returns:
+    a number between 0 and 3*255
+  """
+  return px[0]+px[1]+px[2]

[craigdh, 2013/06/13 19:28:31]

Just sum(px)

[cgrimm, 2013/06/14 18:43:58]

Done.

+
+
+def MinPixel(px1, px2):
+  """gets the pixel with the lower brightness.
+
+  Calculates the brightness of each pixel and returns
+  the pixel with the lower brightness.
+
+  Args:
+    px1: a 3-tuple (R,G,B)
+    px2: a 3-tuple (R,G,B)
+  Returns:
+    a 3-tuple (R,G,B)
+  """
+  average_px1 = Brightness(px1)

[craigdh, 2013/06/13 19:28:31]

no need for the intermediate variables here.

[cgrimm, 2013/06/14 18:43:58]

Done.

+  average_px2 = Brightness(px2)
+  if average_px1 < average_px2:
+    return px1
+  else:
+    return px2
+
+
+def MaxPixel(px1, px2):
+  """gets the pixel with the greater brightness.
+
+  Calculates the brightness of each pixel and returns
+  the pixel with the greater brightness.
+
+  Args:
+    px1: a 3-tuple (R,G,B)
+    px2: a 3-tuple (R,G,B)
+  Returns:
+    a 3-tuple (R,G,B)
+  """
+  average_px1 = Brightness(px1)

[craigdh, 2013/06/13 19:28:31]

ditto

[cgrimm, 2013/06/14 18:43:58]

Done.

+  average_px2 = Brightness(px2)
+  if average_px1 < average_px2:
+    return px2
+  else:
+    return px1
+
+
+def CreateMask(is_hard, cutoff, *images):

[craigdh, 2013/06/13 19:28:31]

No need for the *, as mentioned earlier.

[cgrimm, 2013/06/14 18:43:58]

Done.

+  """Computes a mask for a set of images.
+
+  Returns an image that is computed from the images
+  passed in. The mask's values are computed by calculating
+  total differences between the image, and storing them
+  such that (255,255,255) represents great difference,
+  and (0,0,0) represents no difference. If is_hard is
+  True, the resulting mask is put through a
+  thresholding pass where pixel values below the
+  cutoff become (0,0,0) and ones above become (255,255,255).
+
+  Args:
+    is_hard: boolean, whether or not to threshold the mask

[craigdh, 2013/06/13 19:28:31]

is_hard -> threshold

Also, I would pass images first and give cutoff and threshold default values to
mask ANY difference between images.

[cgrimm, 2013/06/14 18:43:58]

Done.

+    cutoff: number, the value to threshold the mask at
+    *images: the images to compute the mask from
+  Returns:
+    an image that is a mask of the passed in images.
+  Raises:
+    Error: if the images passed in are not of the same size
+  """
+  if AreTheSameSize(*images):

[craigdh, 2013/06/13 19:28:31]

Instead of indenting the whole function, just do this at the top:

if not AreTheSameSize(images):
  raise Exception('All images must be the same size')

[cgrimm, 2013/06/14 18:43:58]

Done.

+    mask = Image.new('RGB', images[0].size)
+    mask_data = list(mask.getdata())

[craigdh, 2013/06/13 19:28:31]

The data structure returned is already iterable, it would be much more efficient
to not copy it into a list.

[cgrimm, 2013/06/14 18:43:58]

Done.

+    data_set = [list(x.getdata()) for x in images]

[craigdh, 2013/06/13 19:28:31]

data_set is unnecessary. Just pass images[0].getdata() on line 155 instead of
i1.

[cgrimm, 2013/06/14 18:43:58]

Done.

+    i1 = data_set[0]

[craigdh, 2013/06/13 19:28:31]

more descriptive variable names than i1, i2

[cgrimm, 2013/06/14 18:43:58]

Done.

+    for i2 in data_set[1:]:
+      mask_data = ComputeLargestDifference(mask_data, i1, i2)
+
+    if is_hard:
+      mask_data = [
+          (255, 255, 255) if Brightness(x) > cutoff
+          else (0, 0, 0)
+          for x in mask_data
+          ]

[craigdh, 2013/06/13 19:28:31]

Instead of constructing a list, just create an iterator like so:

mask.putdata((255, 255, 255) if Brightness(x) > cutoff else (0, 0, 0) for x in
mask_data)

And x -> px

[cgrimm, 2013/06/14 18:43:58]

putdata requires a sequence not an iterator.

+
+    mask.putdata(mask_data)
+    return mask
+  else:
+    raise Error('All Images must be the same size')
+
+
+def ComputeLargestDifference(mask_data, image_data1, image_data2):
+  """Modifies a mask based upon a comparision of two images.
+
+  A helper function used to generate masks. The mask, as
+  it exists when the function is called, is compared to two
+  images on a pixel-by-pixel basis. For each pixel, the
+  ColorDistance between the two images is computed as a Color,
+  and compared against the color of the mask at the same
+  pixel. The brightest of these pixels is chosen for pixel
+  in the images/mask, and a new set of pixels is returned.
+
+  Args:
+    mask_data: a list of pixels of the mask
+    image_data1: a list of pixels of an image
+    image_data2: a list of pixels of another image
+  Returns:
+    a list of pixels representing a modified version of the mask.
+  """
+  return [

[craigdh, 2013/06/13 19:28:31]

Should be able to create an iterator instead of a list.

[cgrimm, 2013/06/14 18:43:58]

Done.

+      MaxPixel(m, GetColorDistAsColor(i1, i2))
+      for m, i1, i2 in izip(mask_data, image_data1, image_data2)
+      ]

[craigdh, 2013/06/13 19:28:31]

Also, this indentation doesn't look like chromium style.

[cgrimm, 2013/06/14 18:43:58]

Done.

+
+
+def ThresholdColorDiff(px1, px2, cutoff):
+  """Thresholds the color distance of two pixels.
+
+  Computes a Threshold of the color distance of two pixels,
+  if the normalized color distance of px1 and px2 is greater than
+  cutoff, returns 1. otherwise returns 0.
+
+  Args:
+    px1: a 3-tuple (R,G,B)
+    px2: a 3-tuple (R,G,B)
+    cutoff: a number used as the threshold limit
+  Returns:
+    either 1. or 0. depending on the distance of px1 and px2.
+  """
+  diff = GetColorDist(px1, px2)
+  if diff > cutoff:
+    return 1.
+  else:
+    return 0.
+
+
+def MaskToValue(px):
+  """Converts a black or white pixel to 1. or 0.
+
+  Args:
+    px: a 3-tuple (R,G,B)
+  Returns:
+    1. if the pixel is (0,0,0), 0. if the pixel is (255,255,255)
+  Raises:
+    Error: if the pixel is not (0,0,0) or (255,255,255)
+  """
+  if px == (0, 0, 0):
+    return 1.
+  if px == (255, 255, 255):
+    return 0.
+  else:
+    raise Error('Mask may only contain black or white pixels')
+
+
+def Similarity(image1, image2, mask=None):
+  """computes the similarity of two images.
+
+  Computes the similarity of two images taking into account
+  an optional mask. the similarity is returned as a number
+  between 0 and 1, where 1 is the same image, and 0 is a

[craigdh, 2013/06/13 19:28:31]

We're doing pixel-by-pixel comparisons, so this would probably be better
reported as the number of non-identical pixels.

[cgrimm, 2013/06/14 18:43:58]

Done.

+  dramatically different image.
+
+  Args:
+    image1: the first Image to be compared
+    image2: the second Image to be compared
+    mask: an optional mask to occlude parts of the images
+     from calculation
+  Returns:
+    a number representing the similarity of the images.
+  Raises:
+    Error: if the images to be compared and mask are not the same size.
+  """
+  if mask:
+    if AreTheSameSize(image1, image2, mask):
+      return 1. - sum([
+          MaskToValue(m)*ThresholdColorDiff(px1, px2, 0.001)
+          for m, px1, px2 in izip(mask.getdata(),
+                                  image1.getdata(),
+                                  image2.getdata())
+          ]) / (image1.size[0]*image1.size[1])
+    else:
+      raise Error('images and mask must be the same size')
+  else:
+    if AreTheSameSize(image1, image2):
+      return 1. - sum([
+          ThresholdColorDiff(px1, px2, 0.001)
+          for px1, px2 in izip(image1.getdata(), image2.getdata())
+          ]) / (image1.size[0]*image1.size[1])
+    else:
+      raise Error('images must be the same size')
+
+
+def SameImage(image1, image2, certainty, mask=None):
+  """Returns a boolean representing if the images are the Same.
+
+  Returns a boolean relating to whether or not two images
+  are similar enough to be considered the same. Essentially
+  wraps the Similarity function and adds a certainty
+  parameter: a number between 0 and 1 that represents the

[craigdh, 2013/06/13 19:28:31]

As above, take a difference in pixels for the cutoff.

[cgrimm, 2013/06/14 18:43:58]

Done.

+  cutoff in Similarity for Sameness.
+
+  Args:
+    image1: an Image to compare
+    image2: an Image to compare
+    certainty: a number that is the cutoff for image sameness
+    mask: an optional Image that occludes parts of the images from
+     same-ness calculation
+  Returns:
+    a boolean representing if the images are the same.
+  Raises:
+    Error: if the images (and mask) are different sizes.
+  """
+  similar = Similarity(image1, image2, mask)
+  if similar >= certainty:

[craigdh, 2013/06/13 19:28:31]

just:

return similar >= certainty

[cgrimm, 2013/06/14 18:43:58]

Done.

+    return True
+  else:
+    return False