Unpunycode search keywords and short names.

components/search_engines/template_url.cc

Index: components/search_engines/template_url.cc
diff --git a/components/search_engines/template_url.cc b/components/search_engines/template_url.cc
index 8202fa4483c0d7d0569b9bf5da797ff44b1c2f26..2be8e778502e655c6fc9e2da3a2d86892112d859 100644
--- a/components/search_engines/template_url.cc
+++ b/components/search_engines/template_url.cc
@@ -1217,7 +1217,8 @@ base::string16 TemplateURL::GenerateKeyword(const GURL& url) {
   // Special case: if the host was exactly "www." (not sure this can happen but
   // perhaps with some weird intranet and custom DNS server?), ensure we at
   // least don't return the empty string.
-  base::string16 keyword(net::StripWWWFromHost(url));
+  // We do not want any punycode in any language.
+  base::string16 keyword(net::StripWWW(net::IDNToUnicode(url.host(), "")));

[Peter Kasting, 2015/07/14 18:05:33]

I think we should pass the user's accept-languages here, so we'll un-punycode
more cases that the user should understand.

This should probably be passed to this function as a second arg, and so on up
the call chain until you get to a chrome-side caller that can compute this as:

  profile_->GetPrefs()->GetString(prefs::kAcceptLanguages)

Another route in some cases would be to add an embedder client method to get
these languages, then have the components-side code that needs to call the
client method get the embedder to instantiate such a client instance for it. 
Yet another route would be to add this information, or accessors that compute
it, to SearchTermsData.

[alshabalin, 2015/07/15 14:18:01]

My thought was that a string in a language user does not read and a string in
IDN form are equally incomprehensible but the first one looks nicer. After all,
punycode is there only because hostname must be ASCII and as such is an
implementation detail.

Or is there some guideline that we should unpunycode only to the languages
the user understands?

[Peter Kasting, 2015/07/15 21:19:12]

It seems like you're misunderstanding how the conversion works.  If you pass no
languages, we will fail to render as Unicode any hostnames that mix characters
from different scripts.  If you pass the languages the user understands, then
we'll be willing to show as Unicode hostnames which use characters from those
scripts.  Passing the empty string doesn't mean "un-punycode everything", it
means something closer to the opposite.

In other words, I'm trying to get your code to un-punycode more hostnames, not
fewer.

[alshabalin, 2015/07/16 15:13:07]

I thought script-mixing in hostnames is not allowed anyway because it opens up
more ways to spoof. But apparently I'm wrong
https://tools.ietf.org/html/rfc5890#section-4.4
My mistake, sorry, will rewrite to pass accepted languages then.

[alshabalin, 2015/07/20 14:06:37]

Done.

   return keyword.empty() ? base::ASCIIToUTF16("www") : keyword;
 }