FTP: use ICU to parse month abbreviations.

net/ftp/ftp_util.cc

 // Copyright (c) 2009 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 #include "net/ftp/ftp_util.h"
 
 #include <vector>
 
 #include "base/logging.h"
 #include "base/string_number_conversions.h"
 #include "base/string_tokenizer.h"
 #include "base/string_util.h"
 #include "base/time.h"
 #include "base/utf_string_conversions.h"
+#include "unicode/dtfmtsym.h"
 
 // For examples of Unix<->VMS path conversions, see the unit test file. On VMS
 // a path looks differently depending on whether it's a file or directory.
 
 namespace net {
 
 // static
 std::string FtpUtil::UnixFilePathToVMS(const std::string& unix_path) {
   if (unix_path.empty())
     return std::string();
@@ skipping 76 matching lines @@
   std::replace(result.begin(), result.end(), ']', '/');
 
   // Make sure the result doesn't end with a slash.
   if (result.length() && result[result.length() - 1] == '/')
     result = result.substr(0, result.length() - 1);
 
   return result;
 }
 
 // static
-bool FtpUtil::ThreeLetterMonthToNumber(const string16& text, int* number) {
-  const static char* months[] = { "jan", "feb", "mar", "apr", "may", "jun",
-                                  "jul", "aug", "sep", "oct", "nov", "dec" };
+bool FtpUtil::AbbreviatedMonthToNumber(const string16& text, int* number) {
+  icu::UnicodeString unicode_text(text.data(), text.size());
 
-  for (size_t i = 0; i < arraysize(months); i++) {
-    if (LowerCaseEqualsASCII(text, months[i])) {
-      *number = i + 1;
-      return true;
+  int32_t locales_count;
+  const icu::Locale* locales = icu::Locale::getAvailableLocales(locales_count);

[jungshik at Google, 2011/03/17 22:59:08]

This will return a lot more entries than we have datetime format symbols for. 

You can use DateFormat::getAvailableLocale(), instead. 

[Paweł Hajdan Jr., 2011/03/18 09:04:41]

Good idea, done.

+
+  // Some FTP servers localize the date listings. To guess the locale,
+  // we loop over all available ones.
+  for (int32_t locale = 0; locale < locales_count; locale++) {

[jungshik at Google, 2011/03/17 22:59:08]

Every time this function is called, we have to go through this loop. How about
splitting this function into two functions? The first one will be used to
determine the locale. The second one will convert an abbreviated month name to a
month index in the locale determined in the first. 

In addition, we can try 'en' first before trying other locales in the first
function.

[Paweł Hajdan Jr., 2011/03/18 09:04:41]

I was considering some ways to make it faster, but I don't consider this code
performance-critical. Downloading the listing from FTP will take much more time,
especially because of multiple TCP connections.

Also, if we "lock into" a locale too easily, we can run into a variation of
http://code.google.com/p/chromium/issues/detail?id=76171 .
I was also thinking about that, but I'm not sure whether a special case is
really worth it. Also, I'd like the ICU code to be exercised a lot here, at
least for a few dev channel releases after this change lands.

+    UErrorCode status(U_ZERO_ERROR);
+
+    icu::DateFormatSymbols format_symbols(locales[locale], status);
+
+    // If we cannot get format symbols for some locale, it's not a fatal error.
+    // Just try another one.
+    if (U_FAILURE(status))
+      continue;
+
+    int32_t months_count;
+    const icu::UnicodeString* months =
+        format_symbols.getShortMonths(months_count);
+
+    // Loop over all abbreviated month names in given locale.
+    // An alternative solution (to parse |text| in given locale) is more
+    // lenient, and may accept more than we want even with setLenient(false).

[jungshik at Google, 2011/03/17 22:59:08]

Have you tried parsing a whole 'date string' instead of just a month name? Hmm,
in that case, you have to detect the language first. Perhaps, it's slower
because you need to go through supported locales. 

[Paweł Hajdan Jr., 2011/03/18 09:04:41]

Parsing a whole date string may accept more than what's valid in an "ls -l"
style listing. Moreover, there are no guarantees it _will_ accept the date
format that is valid in an "ls -l" style listing.

Because of lack of those guarantees, I'd like to not go that route even if it
would work for some version of ICU.

+    for (int32_t month = 0; month < months_count; month++) {
+      if (months[month].caseCompare(unicode_text, 0) == 0) {
+        *number = month + 1;
+        return true;
+      }
     }
   }
 
-  // Special cases for directory listings in German (other three-letter month
-  // abbreviations are the same as in English). Note that we don't need to do
-  // a case-insensitive compare here. Only "ls -l" style listings may use
-  // localized month names, and they will always start capitalized. Also,
-  // converting non-ASCII characters to lowercase would be more complicated.
-  if (text == UTF8ToUTF16("M\xc3\xa4r")) {
-    // The full month name is M-(a-umlaut)-rz (March), which is M-(a-umlaut)r
-    // when abbreviated.
-    *number = 3;
-    return true;
-  }
-  if (text == ASCIIToUTF16("Mai")) {
-    *number = 5;
-    return true;
-  }
-  if (text == ASCIIToUTF16("Okt")) {
-    *number = 10;
-    return true;
-  }
-  if (text == ASCIIToUTF16("Dez")) {
-    *number = 12;
-    return true;
-  }
-
   return false;
 }
 
 // static
 bool FtpUtil::LsDateListingToTime(const string16& month, const string16& day,
                                   const string16& rest,
                                   const base::Time& current_time,
                                   base::Time* result) {

[jungshik at Google, 2011/03/17 22:59:08]

Hmm... Do all the ftp servers use 'day month year' order? Callers of this
function assume that. Perhaps, they do, but ...

[Paweł Hajdan Jr., 2011/03/18 09:04:41]

They do. I consider that a part of the listing format definition. We actually
rely on this ordering to detect multiple variants of the "ls -l" listing. I've
found that to be the most reliable method.

   base::Time::Exploded time_exploded = { 0 };
 
-  if (!ThreeLetterMonthToNumber(month, &time_exploded.month))
+  if (!AbbreviatedMonthToNumber(month, &time_exploded.month))
     return false;
 
   if (!base::StringToInt(day, &time_exploded.day_of_month))
     return false;
 
   if (!base::StringToInt(rest, &time_exploded.year)) {
     // Maybe it's time. Does it look like time (HH:MM)?
     if (rest.length() == 5 && rest[2] == ':') {
       if (!base::StringToInt(rest.begin(),
                              rest.begin() + 2,
@@ skipping 52 matching lines @@
     while (pos < text.length() && !isspace(text[pos]))
       pos++;
   }
 
   string16 result(text.substr(pos));
   TrimWhitespace(result, TRIM_ALL, &result);
   return result;
 }
 
 }  // namespace