Adds a feature to MHTML serialization that omits subframes and subresources marked no-store.

content/browser/download/mhtml_generation_browsertest.cc

 // Copyright (c) 2012 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 #include <stdint.h>
 
 #include "base/bind.h"
 #include "base/callback.h"
 #include "base/files/file_path.h"
 #include "base/files/file_util.h"
@@ skipping 181 matching lines @@
   // We expect that there was an error (file size -1 indicates an error.)
   EXPECT_EQ(-1, file_size());
 
   std::string mhtml;
   ASSERT_TRUE(base::ReadFileToString(path, &mhtml));
 
   // Make sure the contents are missing.
   EXPECT_THAT(mhtml, Not(HasSubstr("test body")));
 }
 
+IN_PROC_BROWSER_TEST_F(MHTMLGenerationTest,

[Łukasz Anforowicz, 2016/05/18 17:27:25]

Could you please also add a test that tries to open / navigate to the generated
mhtml file?  I am worried that your current solution results in dangling cid
URIs as described in https://crbug.com/570353.

I think it would be great if you could somehow reuse before-vs-after tests from
SavePageOriginalVsSavedComparisonTest test suite in
chrome/browser/download/save_page_browsertest.cc, but I understand that some
things won't work as-is:

1) TestOriginalVsSavedPage shuts down embedded_test_server before opening the
saved file.  This is done to make sure that the saved file is self-contained,
but in your scenario, you *do* want the no-store resources to be fetched again
from the network.

2) save_page_type parameter of TestOriginalVsSavedPage is not covering your
no-store flag.

I am not sure how to combine the various test flags nicely.  I think that maybe
introducing a test-only enum SaveTypeForTesting (MHTML, COMPLETE_HTML,
MHTML_WITH_NO_STORE) might work.  For the last enum value you wouldn't stop the
http test server + you could set MHTMLCacheControlPolicy as appropriate.

[dewittj, 2016/05/19 04:13:17]

Done.
It does indeed result in dangling CIDs.  I don't think this is a problem that
needs be fixed in this patch, since dangling CIDs are also present in other
MHTML generated by Chrome, and since this path for MHTML is only on Android and
is not being shared at present.  MHTML does not cause external protocol handler
requests for cid: on Android.
In my particular scenario, I don't actually want the no-store resources to be
fetched at all, since I'm trying to create an archive independent of the
network.  It's not clear that fetching a no-store iframe anew if you are just
given this file is the right thing to do anyway. (cookie jar could be completely
different, for example.)

[Łukasz Anforowicz, 2016/05/19 17:00:13]

Thanks for checking.  Since right now the no-store serialization flag is only
used by offline pages on Android, then I guess we can deal with dangling cid
URIs in a follow-up CL.
Ok - thanks for explaining why we might still want to turn off the test server. 
Makes sense.

Thanks for adding the new tests.

[dewittj, 2016/05/19 18:18:35]

Acknowledged.

+                       GenerateMHTMLIgnoreNoStoreSubFrame) {
+  base::FilePath path(temp_dir_.path());
+  path = path.Append(FILE_PATH_LITERAL("test.mht"));
+
+  GURL url(embedded_test_server()->GetURL("/page_with_nostore_iframe.html"));
+
+  // Generate MHTML, specifying the FAIL_FOR_NO_STORE_MAIN_FRAME policy.
+  MHTMLGenerationParams params(path);
+  params.cache_control_policy =
+      content::MHTMLCacheControlPolicy::FAIL_FOR_NO_STORE_MAIN_FRAME;
+
+  GenerateMHTML(params, url);
+  // We expect that there was no error (file size -1 indicates an error.)
+  EXPECT_LT(0, file_size());
+
+  std::string mhtml;
+  ASSERT_TRUE(base::ReadFileToString(path, &mhtml));
+
+  // Make sure that no-store subresources exist in this mode.
+  EXPECT_THAT(mhtml, HasSubstr("no-store test body"));
+  EXPECT_THAT(mhtml, ContainsRegex("Content-Location:.*nostore.jpg"));
+}
+
+IN_PROC_BROWSER_TEST_F(MHTMLGenerationTest, GenerateMHTMLObeyNoStoreSubFrame) {
+  base::FilePath path(temp_dir_.path());
+  path = path.Append(FILE_PATH_LITERAL("test.mht"));
+
+  GURL url(embedded_test_server()->GetURL("/page_with_nostore_iframe.html"));
+
+  // Generate MHTML, specifying the FAIL_FOR_NO_STORE_MAIN_FRAME policy.
+  MHTMLGenerationParams params(path);
+  params.cache_control_policy = content::MHTMLCacheControlPolicy::
+      SKIP_ANY_FRAME_OR_RESOURCE_MARKED_NO_STORE;
+
+  GenerateMHTML(params, url);
+  // We expect that there was no error (file size -1 indicates an error.)
+  EXPECT_LT(0, file_size());
+
+  std::string mhtml;
+  ASSERT_TRUE(base::ReadFileToString(path, &mhtml));
+
+  // Make sure the contents are missing.
+  EXPECT_THAT(mhtml, Not(HasSubstr("no-store test body")));
+  // This image comes from a resource marked no-store.
+  EXPECT_THAT(mhtml, ContainsRegex("Content-Location:.*nostore.jpg"));
+}
+
 // Test suite that allows testing --site-per-process against cross-site frames.
 // See http://dev.chromium.org/developers/design-documents/site-isolation.
 class MHTMLGenerationSitePerProcessTest : public MHTMLGenerationTest {
  public:
   MHTMLGenerationSitePerProcessTest() {}
 
  protected:
   void SetUpCommandLine(base::CommandLine* command_line) override {
     MHTMLGenerationTest::SetUpCommandLine(command_line);
 
@@ skipping 32 matching lines @@
 
   // Make sure that URLs of both frames are present
   // (note that these are single-line regexes).
   EXPECT_THAT(
       mhtml,
       ContainsRegex("Content-Location:.*/frame_tree/page_with_one_frame.html"));
   EXPECT_THAT(mhtml, ContainsRegex("Content-Location:.*/title1.html"));
 }
 
 }  // namespace content