Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(312)

Unified Diff: third_party/WebKit/Source/wtf/text/UTF8.cpp

Issue 1721373002: UTF-8 detector for pages missing encoding info (Closed) Base URL: https://chromium.googlesource.com/chromium/src.git@master
Patch Set: left out test files that should be landed manually Created 4 years, 8 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View side-by-side diff with in-line comments
Download patch
« no previous file with comments | « third_party/WebKit/Source/wtf/text/UTF8.h ('k') | third_party/WebKit/Source/wtf/text/UTF8Test.cpp » ('j') | no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
Index: third_party/WebKit/Source/wtf/text/UTF8.cpp
diff --git a/third_party/WebKit/Source/wtf/text/UTF8.cpp b/third_party/WebKit/Source/wtf/text/UTF8.cpp
index 0beca10497c8e8411d13883305a172c28b264b45..a258db81b99a483309b80078de146aed8053d9c3 100644
--- a/third_party/WebKit/Source/wtf/text/UTF8.cpp
+++ b/third_party/WebKit/Source/wtf/text/UTF8.cpp
@@ -442,5 +442,23 @@ bool equalLatin1WithUTF8(const LChar* a, const LChar* aEnd, const char* b, const
return equalWithUTF8Internal(a, aEnd, b, bEnd);
}
+bool isUTF8andNotASCII(const char* data, size_t length)
+{
+ // This cast is necessary because U8_NEXT uses int32_ts.
+ int32_t srcLen = static_cast<int32_t>(length);
+ int32_t charIndex = 0;
+ bool isASCIIOnly = true;
+
+ while (charIndex < srcLen) {
+ int32_t codePoint;
+ if (static_cast<uint8_t>(data[charIndex]) >= 0x80)
+ isASCIIOnly = false;
+ U8_NEXT(data, charIndex, srcLen, codePoint);
+ if (!U_IS_UNICODE_CHAR(codePoint))
+ return false;
+ }
+ return !isASCIIOnly;
+}
+
} // namespace Unicode
} // namespace WTF
« no previous file with comments | « third_party/WebKit/Source/wtf/text/UTF8.h ('k') | third_party/WebKit/Source/wtf/text/UTF8Test.cpp » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698