Remove the EV Certs Whitelist

Remove the EV Certs Whitelist

Introduced as part of the 2015/01/01 requirement that all EV
certificates should be accompanied by Certificate Transparency
information, the EVCertWhitelist contained the set of publicly
logged EV certificates issued prior to that date, to ensure they
maintained their EV status.

As an EV certificate is only valid for 27 months, the whitelist has
been shrinking over time, with the most recent update trimming it to
around 100 certificates.

However, as 27 months have passed since 2015/01/01, the whitelist is
no longer needed, and as such, the entire supporting infrastructure is
also no longer needed.

This rewinds the code by:
  - Removing the EVCertsWhitelist from //net
  - Removing the distinct EV CT policy from CTPolicyEnforcer
  - Unwinding the EV CT status from the CTVerifyResult and SSLInfo
  - Removing the specific Golomb-coded compressed CT EV whitelist logic
  - Removing the Component Updater version of the EV whitelist
  - Removing all metrics related to the EV whitelist

BUG=732427
TBR=lcwu@chromium.org,sergeyu@chromium.org,isherman@chromium.org

Review-Url: https://codereview.chromium.org/2937563002
Cr-Commit-Position: refs/heads/master@{#479343}
Committed: https://chromium.googlesource.com/chromium/src/+/cd7390ed559bc7101d0c29b2bc2e5b71b01c8eb1

[Ryan Sleevi, 2017-06-12 19:07:36]

The CQ bit was checked by rsleevi@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-06-12 19:08:29]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Ryan Sleevi, 2017-06-12 20:22:15]

rsleevi@chromium.org changed reviewers:
+ estark@chromium.org, jochen@chromium.org, mattm@chromium.org,
waffles@chromium.org

[Ryan Sleevi, 2017-06-12 20:22:17]

estark: IPC files (and overall CT knowledge)
mattm: //net files
jochen: components and Chrome, broadly speaking - although it's mostly just
nuking
waffles: Component updater. In particular, this is removing a component - does
anything need to be done to clean up the component and/or its artifacts as part
of this? (Happy to break it into a separate CL if that's easier, since that'd
presumably be _new_ code)

[waffles, 2017-06-12 20:39:36]

component_updater lgtm

> does
anything need to be done to clean up the component and/or its artifacts as part
of this?

If it is OK to leave the final version of the component on the disk (~684 KB
leaked), no more work is required. If not, there are two options:
1 • Publish an empty sth set as version 8 of the component, which will cut the
leak down to ~2 KB.
2 • Put code in Chrome to recursively delete the contents of
USER_DATA_DIR/EVWhitelist. Presumably such code could be removed after a
milestone or two.

[Ryan Sleevi, 2017-06-12 20:44:31]

On 2017/06/12 20:39:36, waffles wrote:
> component_updater lgtm
> 
> > does
> anything need to be done to clean up the component and/or its artifacts as
part
> of this?
> 
> If it is OK to leave the final version of the component on the disk (~684 KB
> leaked), no more work is required. If not, there are two options:

This isn't the STHSet - just the EV whitelist. How'd you come up with the
~684KB, since the current set should only be ~107 certificates at 8 bytes each,
then compressed.

> 2 • Put code in Chrome to recursively delete the contents of
> USER_DATA_DIR/EVWhitelist. Presumably such code could be removed after a
> milestone or two.

Presumably, this would be something similar to line 508 in
https://codereview.chromium.org/2937563002/diff/1/chrome/browser/chrome_brows...
and approximately line 1240-ish of
https://codereview.chromium.org/2937563002/diff/1/chrome/browser/chromeos/log...
right?

[commit-bot: I haz the power, 2017-06-12 20:54:50]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-06-12 20:54:51]

Dry run: This issue passed the CQ dry run.

[waffles, 2017-06-12 21:14:28]

On 2017/06/12 20:44:31, Ryan Sleevi wrote:
> On 2017/06/12 20:39:36, waffles wrote:
> > component_updater lgtm
> > 
> > > does
> > anything need to be done to clean up the component and/or its artifacts as
> part
> > of this?
> > 
> > If it is OK to leave the final version of the component on the disk (~684 KB
> > leaked), no more work is required. If not, there are two options:
> 
> This isn't the STHSet - just the EV whitelist. How'd you come up with the
> ~684KB, since the current set should only be ~107 certificates at 8 bytes
each,
> then compressed.

I just looked on my local disk:
`du -sh ~/.config/google-chrome/EVWhitelist/7/`
688K ~/.config/google-chrome/EVWhitelist/7/

(In comparison, STH Set is at .../CertificateTransparency and is 2.3M)

> > 2 • Put code in Chrome to recursively delete the contents of
> > USER_DATA_DIR/EVWhitelist. Presumably such code could be removed after a
> > milestone or two.
> 
> Presumably, this would be something similar to line 508 in
>
https://codereview.chromium.org/2937563002/diff/1/chrome/browser/chrome_brows...
> and approximately line 1240-ish of
>
https://codereview.chromium.org/2937563002/diff/1/chrome/browser/chromeos/log...
> right?

Not sure about the second link, but CRLSetFetcher::DeleteFromDisk is a good
model to follow.

[mattm, 2017-06-12 22:23:59]

net lgtm w/nit

https://codereview.chromium.org/2937563002/diff/1/net/ssl/ssl_info.h
File net/ssl/ssl_info.h (right):

https://codereview.chromium.org/2937563002/diff/1/net/ssl/ssl_info.h#newcode133
net/ssl/ssl_info.h:133: // (|ev_policy_compliance|) will contain information
about whether
update comment

[estark, 2017-06-13 05:12:38]

ipc files lgtm

[jochen (gone - plz use gerrit), 2017-06-13 07:23:13]

lgtm

[Eran Messeri, 2017-06-13 09:24:55]

eranm@chromium.org changed reviewers:
+ eranm@chromium.org

[Eran Messeri, 2017-06-13 09:24:56]

lgtm!
Cannot endorse this CL enough :)

[Ryan Sleevi, 2017-06-13 21:18:42]

The CQ bit was checked by rsleevi@chromium.org

[Ryan Sleevi, 2017-06-13 21:18:43]

The patchset sent to the CQ was uploaded after l-g-t-m from
waffles@chromium.org, mattm@chromium.org
Link to the patchset: https://codereview.chromium.org/2937563002/#ps20001
(title: "Update comment")

[commit-bot: I haz the power, 2017-06-13 21:19:34]

CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-06-13 21:31:33]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-06-13 21:31:35]

Try jobs failed on following builders:
  chromium_presubmit on master.tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

[Ryan Sleevi, 2017-06-14 10:10:21]

Description was changed from

==========
Remove the EV Certs Whitelist

Introduced as part of the 2015/01/01 requirement that all EV
certificates should be accompanied by Certificate Transparency
information, the EVCertWhitelist contained the set of publicly
logged EV certificates issued prior to that date, to ensure they
maintained their EV status.

As an EV certificate is only valid for 27 months, the whitelist has
been shrinking over time, with the most recent update trimming it to
around 100 certificates.

However, as 27 months have passed since 2015/01/01, the whitelist is
no longer needed, and as such, the entire supporting infrastructure is
also no longer needed.

This rewinds the code by:
  - Removing the EVCertsWhitelist from //net
  - Removing the distinct EV CT policy from CTPolicyEnforcer
  - Unwinding the EV CT status from the CTVerifyResult and SSLInfo
  - Removing the specific Golomb-coded compressed CT EV whitelist logic
  - Removing the Component Updater version of the EV whitelist
  - Removing all metrics related to the EV whitelist

BUG=732427
==========

to

==========
Remove the EV Certs Whitelist

Introduced as part of the 2015/01/01 requirement that all EV
certificates should be accompanied by Certificate Transparency
information, the EVCertWhitelist contained the set of publicly
logged EV certificates issued prior to that date, to ensure they
maintained their EV status.

As an EV certificate is only valid for 27 months, the whitelist has
been shrinking over time, with the most recent update trimming it to
around 100 certificates.

However, as 27 months have passed since 2015/01/01, the whitelist is
no longer needed, and as such, the entire supporting infrastructure is
also no longer needed.

This rewinds the code by:
  - Removing the EVCertsWhitelist from //net
  - Removing the distinct EV CT policy from CTPolicyEnforcer
  - Unwinding the EV CT status from the CTVerifyResult and SSLInfo
  - Removing the specific Golomb-coded compressed CT EV whitelist logic
  - Removing the Component Updater version of the EV whitelist
  - Removing all metrics related to the EV whitelist

BUG=732427
TBR=lcwu@chromium.org,sergeyu@chromium.org,isherman@chromium.org
==========

[Ryan Sleevi, 2017-06-14 10:10:21]

rsleevi@chromium.org changed reviewers:
+ isherman@chromium.org, lcwu@chromium.org, sergeyu@chromium.org

[Ryan Sleevi, 2017-06-14 10:11:28]

lcwu, sergeyu: TBR because mechanical
isherman: TBR because obsoleting old metrics

I'll follow-up with a separate CL to clean up the component, based on waffles'
suggestion.

[Ryan Sleevi, 2017-06-14 10:11:36]

The CQ bit was checked by rsleevi@chromium.org

[commit-bot: I haz the power, 2017-06-14 10:11:53]

CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-06-14 10:18:29]

CQ is committing da patch.

Bot data: {"patchset_id": 20001, "attempt_start_ts": 1497435096083120,
"parent_rev": "7fa075e8c7635e26f35041b62cd34da83c575e72", "commit_rev":
"cd7390ed559bc7101d0c29b2bc2e5b71b01c8eb1"}

[commit-bot: I haz the power, 2017-06-14 10:18:47]

Description was changed from

==========
Remove the EV Certs Whitelist

Introduced as part of the 2015/01/01 requirement that all EV
certificates should be accompanied by Certificate Transparency
information, the EVCertWhitelist contained the set of publicly
logged EV certificates issued prior to that date, to ensure they
maintained their EV status.

As an EV certificate is only valid for 27 months, the whitelist has
been shrinking over time, with the most recent update trimming it to
around 100 certificates.

However, as 27 months have passed since 2015/01/01, the whitelist is
no longer needed, and as such, the entire supporting infrastructure is
also no longer needed.

This rewinds the code by:
  - Removing the EVCertsWhitelist from //net
  - Removing the distinct EV CT policy from CTPolicyEnforcer
  - Unwinding the EV CT status from the CTVerifyResult and SSLInfo
  - Removing the specific Golomb-coded compressed CT EV whitelist logic
  - Removing the Component Updater version of the EV whitelist
  - Removing all metrics related to the EV whitelist

BUG=732427
TBR=lcwu@chromium.org,sergeyu@chromium.org,isherman@chromium.org
==========

to

==========
Remove the EV Certs Whitelist

Introduced as part of the 2015/01/01 requirement that all EV
certificates should be accompanied by Certificate Transparency
information, the EVCertWhitelist contained the set of publicly
logged EV certificates issued prior to that date, to ensure they
maintained their EV status.

As an EV certificate is only valid for 27 months, the whitelist has
been shrinking over time, with the most recent update trimming it to
around 100 certificates.

However, as 27 months have passed since 2015/01/01, the whitelist is
no longer needed, and as such, the entire supporting infrastructure is
also no longer needed.

This rewinds the code by:
  - Removing the EVCertsWhitelist from //net
  - Removing the distinct EV CT policy from CTPolicyEnforcer
  - Unwinding the EV CT status from the CTVerifyResult and SSLInfo
  - Removing the specific Golomb-coded compressed CT EV whitelist logic
  - Removing the Component Updater version of the EV whitelist
  - Removing all metrics related to the EV whitelist

BUG=732427
TBR=lcwu@chromium.org,sergeyu@chromium.org,isherman@chromium.org

Review-Url: https://codereview.chromium.org/2937563002
Cr-Commit-Position: refs/heads/master@{#479343}
Committed:
https://chromium.googlesource.com/chromium/src/+/cd7390ed559bc7101d0c29b2bc2e...
==========

[commit-bot: I haz the power, 2017-06-14 10:18:50]

Committed patchset #2 (id:20001) as
https://chromium.googlesource.com/chromium/src/+/cd7390ed559bc7101d0c29b2bc2e...

[Ilya Sherman, 2017-06-15 02:24:16]

histograms.xml lgtm