Avoid using language hint in encoding detection

third_party/WebKit/Source/core/html/parser/TextResourceDecoder.cpp

Index: third_party/WebKit/Source/core/html/parser/TextResourceDecoder.cpp
diff --git a/third_party/WebKit/Source/core/html/parser/TextResourceDecoder.cpp b/third_party/WebKit/Source/core/html/parser/TextResourceDecoder.cpp
index 0e20dbebc89de3228330e72fc4d3512658779694..0a4d454814eaa56b8ce4d66dc3e6a932a5e6671f 100644
--- a/third_party/WebKit/Source/core/html/parser/TextResourceDecoder.cpp
+++ b/third_party/WebKit/Source/core/html/parser/TextResourceDecoder.cpp
@@ -159,9 +159,13 @@ TextResourceDecoder::TextResourceDecoder(
   if (m_encodingDetectionOption == AlwaysUseUTF8ForText) {
     DCHECK(m_contentType == PlainTextContent && m_encoding == UTF8Encoding());
   } else if (m_encodingDetectionOption == UseAllAutoDetection) {
-    // Checking empty URL helps unit testing. Providing defaultLanguage() is
+    // Obtain language hint from system locale:
+    // 1) In general, do not use language hint. This helps get more
+    // deterministic encoding detection results across devices. Note that local
+    // file resources can still benefit from the hint.
+    // 2) Checking empty URL helps unit testing. Providing defaultLanguage() is
     // sometimes difficult in tests.
-    if (!hintUrl.isEmpty()) {
+    if (hintUrl.protocol() == "file" || !hintUrl.isEmpty()) {

[tkent, 2017/04/06 07:29:13]

The code is incorrect.  http:// URLs are not isEmpty(). So any URL passes this
condition.

BTW, I'd like to add file:// check to detectTextEncoding() rather than
TextResourceDecoder.  It would make unit testing easy.

[Jinsuk Kim, 2017/04/06 08:10:25]

My bad.

Moved to TextEncodingDetector and updated tests.

       // This object is created in the main thread, but used in another thread.
       // We should not share an AtomicString.
       AtomicString locale = defaultLanguage();