UTF-8 detector for pages missing encoding info

third_party/WebKit/Source/wtf/text/UTF8.cpp

 /*
  * Copyright (C) 2007 Apple Inc.  All rights reserved.
  * Copyright (C) 2010 Patrick Gansterer <paroga@paroga.com>
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
  * are met:
  * 1. Redistributions of source code must retain the above copyright
  *    notice, this list of conditions and the following disclaimer.
  * 2. Redistributions in binary form must reproduce the above copyright
@@ skipping 424 matching lines @@
 bool equalUTF16WithUTF8(const UChar* a, const UChar* aEnd, const char* b, const char* bEnd)
 {
     return equalWithUTF8Internal(a, aEnd, b, bEnd);
 }
 
 bool equalLatin1WithUTF8(const LChar* a, const LChar* aEnd, const char* b, const char* bEnd)
 {
     return equalWithUTF8Internal(a, aEnd, b, bEnd);
 }
 
+bool isUTF8Encoded(const char* data, size_t length)

[jungshik at Google, 2016/04/03 00:52:04]

Without looking at the header file or the function body, it'd be all but
impossible to know what this function does (i.e. returns true ONLY if |data| is
UTF-8 and contains characters beyond ASCII).  How about changing the name to
'isNonAsciiUTF8'? Hmm, that sounds awkward.  Well, I wish I could come up with a
better name.   'isUTF8andNonASCII' better?  Or, how about 'isUTF8andNotASCII' ? 

[Jinsuk Kim, 2016/04/06 04:34:11]

Chose isUTF8andNotASCII 

+{
+    // This cast is necessary because U8_NEXT uses int32_ts.
+    int32_t srcLen = static_cast<int32_t>(length);
+    int32_t charIndex = 0;
+    bool markDetected = false;

[jungshik at Google, 2016/04/03 00:52:04]

At first, it took me a while to figure out what this variable is for. How about
|isNonAsciiFound| or just |nonAsciiFound|?  Or, you can go with |isASCIIOnly|
and flip the value. 

[Jinsuk Kim, 2016/04/06 04:34:11]

Done.

+
+    while (charIndex < srcLen) {
+        int32_t codePoint;
+        if (static_cast<uint8_t>(data[charIndex]) >= 0x80)
+            markDetected = true;
+        U8_NEXT(data, charIndex, srcLen, codePoint);
+        if (!U_IS_UNICODE_CHAR(codePoint))
+            return false;
+    }
+    return markDetected;
+}
+
 } // namespace Unicode
 } // namespace WTF