Expose InlineTextBoxes in the accessibility tree.

Source/testing/runner/AccessibilityUIElementChromium.cpp

Index: Source/testing/runner/AccessibilityUIElementChromium.cpp
diff --git a/Source/testing/runner/AccessibilityUIElementChromium.cpp b/Source/testing/runner/AccessibilityUIElementChromium.cpp
index e48d75c71f9b12f4821fb8d9766f186ba26a7ef3..747f431f54dff8bf4606711f19699d32573277c4 100644
--- a/Source/testing/runner/AccessibilityUIElementChromium.cpp
+++ b/Source/testing/runner/AccessibilityUIElementChromium.cpp
@@ -132,6 +132,8 @@ string roleToString(WebAXRole role)
         return result.append("Image");
     case WebAXRoleIncrementor:
         return result.append("Incrementor");
+    case WebAXRoleInlineTextBox:
+        return result.append("InlineTextBox");
     case WebAXRoleLabel:
         return result.append("Label");
     case WebAXRoleLegend:
@@ -345,6 +347,80 @@ string getAttributes(const WebAccessibilityObject& object)
     return attributes;
 }
 
+WebRect boundsForCharacter(const WebAccessibilityObject& object, int characterIndex)
+{
+    WEBKIT_ASSERT(object.role() == WebAXRoleStaticText);
+    int end = 0;
+    for (unsigned i = 0; i < object.childCount(); i++) {
+        WebAccessibilityObject child = object.childAt(i);
+        WEBKIT_ASSERT(child.role() == WebAXRoleInlineTextBox);
+        int start = end;
+        end += child.stringValue().length();
+        if (end <= characterIndex)
+            continue;
+        WebRect childRect = child.boundingBoxRect();
+        int index = characterIndex - start;
+        WebVector<int> characterOffsets;
+        child.characterOffsets(characterOffsets);
+        WEBKIT_ASSERT(characterOffsets.size() > 0 && characterOffsets.size() == child.stringValue().length());

[aboxhall, 2013/09/06 17:19:21]

Is checking characterOffsets.size() > 0 redundant?

[dmazzoni, 2013/09/06 20:25:27]

I want to assert that they're both not zero. It'd be just as surprising if
child.stringValue() was empty.

+        if (characterOffsets[characterOffsets.size() - 1] < 0) {

[aboxhall, 2013/09/06 17:19:21]

Why last offset rather than first?

[dmazzoni, 2013/09/06 20:25:27]

I was worried about a corner case where the first character's width happens to
be zero (especially since it's rounded). Since the offsets are accumulative,
checking the last offset is better.

+            // RTL.
+            int right = childRect.x + childRect.width;
+
+            if (index)
+                return WebRect(right + characterOffsets[index], childRect.y, characterOffsets[index - 1] - characterOffsets[index], childRect.height);

[aboxhall, 2013/09/06 17:19:21]

This is breaking my brain... would it be a problem to pull it out something like
int left = right + characterOffsets[index];
int width = characterOffsets[index - 1] - characterOffsets[index];
return WebRect(left, childRect.y, width, childRect.height);
...? All the double negatives are pretty hard to follow when you're also keeping
track of what argument you're up to.

(It might also be helpful to do something similar in the if(index) case for LTR
below as well, just for comparison.)

[aboxhall, 2013/09/06 20:51:12]

Ping? No worries if this isn't worth the effort for layout test-only code.

[dmazzoni, 2013/09/09 07:03:14]

Sorry, meant to do this. Done - I like your idea, it's hopefully more readable
now.

+            return WebRect(right + characterOffsets[0], childRect.y, -characterOffsets[0], childRect.height);
+        }
+
+        if (index)
+            return WebRect(childRect.x + characterOffsets[index - 1], childRect.y, characterOffsets[index] - characterOffsets[index - 1], childRect.height);
+        return WebRect(childRect.x, childRect.y, characterOffsets[0], childRect.height);
+    }
+
+    WEBKIT_ASSERT(false);
+    return WebRect();
+}
+
+void getWordBoundaries(const WebAccessibilityObject& object, int characterIndex, int& wordStart, int& wordEnd)
+{
+    int end = 0;
+    for (unsigned i = 0; i < object.childCount(); i++) {
+        WebAccessibilityObject child = object.childAt(i);
+        WEBKIT_ASSERT(child.role() == WebAXRoleInlineTextBox);
+        int start = end;
+        end += child.stringValue().length();
+        if (end <= characterIndex)
+            continue;
+        int index = characterIndex - start;
+
+        WebVector<int> starts;
+        WebVector<int> ends;
+        child.wordBoundaries(starts, ends);
+        size_t wordCount = starts.size();
+        WEBKIT_ASSERT(ends.size() == wordCount);
+
+        // If there are no words, use the InlineTextBox boundaries.

[aboxhall, 2013/09/06 17:19:21]

Why would there be no words? 

[dmazzoni, 2013/09/06 20:25:27]

An inline text box could have just whitespace, for example. The word break
iterator would detect no complete words.

+        if (!wordCount) {
+            wordStart = start;
+            wordEnd = end;
+            return;
+        }
+
+        // Look for a character within any word other than the last.

[aboxhall, 2013/09/06 17:19:21]

Why not the last?

[dmazzoni, 2013/09/06 20:25:27]

Only because that case is handled below - any character past the last word is
treated as part of the last word.

Remember, this code is ONLY to enable layout tests. This implementation is
somewhat arbitrary.

The equivalent Mac and Win code on the Chromium side will
have to do the right thing, but I think those APIs are
more specific, too.

+        for (size_t j = 0; j < wordCount - 1; j++) {
+            if (characterIndex <= ends[j]) {

[aboxhall, 2013/09/06 17:19:21]

Should this be index?

[dmazzoni, 2013/09/06 20:25:27]

You're totally right. Fixed.

I renamed it inlineBoxRelativeIndex to be totally clear.

[aboxhall, 2013/09/06 20:51:12]

I'm curious why this wasn't caught by the layout test, I would have thought the
multi-line case would be broken by this.

BTW, what about renaming index -> inlineBoxRelativeIndex in the
boundsForCharacter method too?

[dmazzoni, 2013/09/09 07:03:14]

Totally fair point. It wasn't caught because I didn't have an actual "word" on
the
second line of text. I fixed the test so that there's a word on the second line
and now the test fails without this fix.
Done. How about "localIndex" instead - it's basically the same index, but in the
local coordinates of that inline box.

+                wordStart = start + starts[j];
+                wordEnd = start + ends[j];
+                return;
+            }
+        }
+
+        // Return the last word by default.
+        wordStart = start + starts[wordCount - 1];
+        wordEnd = start + ends[wordCount - 1];
+        return;
+    }
+}
 
 // Collects attributes into a string, delimited by dashes. Used by all methods
 // that output lists of attributes: attributesOfLinkedUIElementsCallback,
@@ -458,6 +534,8 @@ AccessibilityUIElement::AccessibilityUIElement(const WebAccessibilityObject& obj
     bindMethod("scrollToMakeVisible", &AccessibilityUIElement::scrollToMakeVisibleCallback);
     bindMethod("scrollToMakeVisibleWithSubFocus", &AccessibilityUIElement::scrollToMakeVisibleWithSubFocusCallback);
     bindMethod("scrollToGlobalPoint", &AccessibilityUIElement::scrollToGlobalPointCallback);
+    bindMethod("wordStart", &AccessibilityUIElement::wordStartCallback);
+    bindMethod("wordEnd", &AccessibilityUIElement::wordEndCallback);
 
     bindFallbackMethod(&AccessibilityUIElement::fallbackCallback);
 }
@@ -747,9 +825,36 @@ void AccessibilityUIElement::lineForIndexCallback(const CppArgumentList& argumen
     result->set(line);
 }
 
-void AccessibilityUIElement::boundsForRangeCallback(const CppArgumentList&, CppVariant* result)
+void AccessibilityUIElement::boundsForRangeCallback(const CppArgumentList& arguments, CppVariant* result)
 {
     result->setNull();
+
+    if (arguments.size() != 2 || !arguments[0].isNumber() || !arguments[1].isNumber())
+        return;
+
+    if (accessibilityObject().role() != WebAXRoleStaticText)
+        return;
+
+    int start = arguments[0].toInt32();
+    int end = arguments[1].toInt32();
+    int len = end - start;
+
+    // Get the bounds for each character and union them into one large rectangle.
+    // This is just for testing so it doesn't need to be efficient.
+    WebRect bounds = boundsForCharacter(accessibilityObject(), start);
+    for (int i = 1; i < len; i++) {
+        WebRect next = boundsForCharacter(accessibilityObject(), start + i);
+        int right = std::max(bounds.x + bounds.width, next.x + next.width);
+        int bottom = std::max(bounds.y + bounds.height, next.y + next.height);
+        bounds.x = std::min(bounds.x, next.x);
+        bounds.y = std::min(bounds.y, next.y);
+        bounds.width = right - bounds.x;
+        bounds.height = bottom - bounds.y;
+    }
+
+    char buffer[100];
+    snprintf(buffer, sizeof(buffer), "{x: %d, y: %d, width: %d, height: %d}", bounds.x, bounds.y, bounds.width, bounds.height);
+    result->set(string(buffer));
 }
 
 void AccessibilityUIElement::stringForRangeCallback(const CppArgumentList&, CppVariant* result)
@@ -1035,6 +1140,38 @@ void AccessibilityUIElement::scrollToGlobalPointCallback(const CppArgumentList&
     result->setNull();
 }
 
+void AccessibilityUIElement::wordStartCallback(const CppArgumentList& arguments, CppVariant* result)

[aboxhall, 2013/09/06 17:19:21]

Are we likely to mostly be calling this and wordEnd() in rapid succession? Is
there any way to cache the result of getWordBoundaries() for a given character
so we don't need to recalculate it if that is the case? Alternatively, is there
any way we could combine these two methods and return a start, end pair instead?

[dmazzoni, 2013/09/06 20:25:27]

This code is only for enabling layout tests, so this seemed simpler. For the
'actual' implementation of this method on the Chromium side
we should worry about efficiency.

[aboxhall, 2013/09/06 20:51:12]

Makes sense.

+{
+    result->setNull();
+
+    if (arguments.size() != 1 || !arguments[0].isNumber())
+        return;
+
+    if (accessibilityObject().role() != WebAXRoleStaticText)
+        return;
+
+    int characterIndex = arguments[0].toInt32();
+    int wordStart, wordEnd;
+    getWordBoundaries(accessibilityObject(), characterIndex, wordStart, wordEnd);
+    result->set(wordStart);
+}
+
+void AccessibilityUIElement::wordEndCallback(const CppArgumentList& arguments, CppVariant* result)
+{
+    result->setNull();
+
+    if (arguments.size() != 1 || !arguments[0].isNumber())
+        return;
+
+    if (accessibilityObject().role() != WebAXRoleStaticText)
+        return;
+
+    int characterIndex = arguments[0].toInt32();
+    int wordStart, wordEnd;
+    getWordBoundaries(accessibilityObject(), characterIndex, wordStart, wordEnd);
+    result->set(wordEnd);
+}
+
 void AccessibilityUIElement::fallbackCallback(const CppArgumentList &, CppVariant* result)
 {
     // FIXME: Implement this.