Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(457)

Issue 1228543002: Translate ONC ProxySettings <-> Shill ProxyConfig (Closed)

Created:
5 years, 5 months ago by stevenjb
Modified:
5 years, 5 months ago
CC:
cbentzel+watch_chromium.org, chromium-reviews, davemoore+watch_chromium.org, oshima+watch_chromium.org, stevenjb+watch_chromium.org
Base URL:
https://chromium.googlesource.com/chromium/src.git@master
Target Ref:
refs/pending/heads/master
Project:
chromium
Visibility:
Public.

Description

Translate ONC ProxySettings <-> Shill ProxyConfig This CL adds translation of the ProxySettings property in an ONC dictionary to the ProxyConfig property of a Shill dictionary (which previously was not being handled). BUG=470445 TBR=abarth@chromium.org for DEPS change (+url) Committed: https://crrev.com/8336888d95e672e8642889cade066aed7bb1f595 Cr-Commit-Position: refs/heads/master@{#338352}

Patch Set 1 #

Total comments: 23

Patch Set 2 : Use net/ parsing, update OWNERS #

Total comments: 16

Patch Set 3 : Feedback #

Patch Set 4 : Compile fix #

Patch Set 5 : Move proxy_config.json #

Patch Set 6 : Add Proxy PAC tests #

Total comments: 15

Patch Set 7 : Fix assumptions in ShillToONC #

Total comments: 5

Patch Set 8 : Add SchemeToString #

Total comments: 2
Unified diffs Side-by-side diffs Delta from patch set Stats (+546 lines, -291 lines) Patch
M chrome/browser/chromeos/net/onc_utils.h View 1 chunk +0 lines, -9 lines 0 comments Download
M chrome/browser/chromeos/net/onc_utils.cc View 3 chunks +0 lines, -116 lines 0 comments Download
D chrome/browser/chromeos/net/onc_utils_unittest.cc View 1 chunk +0 lines, -48 lines 0 comments Download
M chrome/browser/chromeos/net/proxy_config_handler.cc View 1 chunk +1 line, -0 lines 0 comments Download
M chrome/browser/chromeos/ui_proxy_config.h View 1 chunk +0 lines, -5 lines 0 comments Download
M chrome/browser/chromeos/ui_proxy_config.cc View 1 2 3 3 chunks +17 lines, -25 lines 0 comments Download
M chrome/chrome_tests_unit.gypi View 1 chunk +0 lines, -1 line 0 comments Download
D chrome/test/data/chromeos/net/proxy_config.json View 1 2 3 4 1 chunk +0 lines, -84 lines 0 comments Download
M chromeos/BUILD.gn View 2 chunks +2 lines, -0 lines 0 comments Download
M chromeos/chromeos.gyp View 3 chunks +3 lines, -0 lines 0 comments Download
M chromeos/network/DEPS View 1 2 3 4 1 chunk +3 lines, -0 lines 0 comments Download
M chromeos/network/onc/onc_translation_tables.cc View 2 chunks +1 line, -3 lines 0 comments Download
M chromeos/network/onc/onc_translator_onc_to_shill.cc View 2 chunks +13 lines, -0 lines 0 comments Download
M chromeos/network/onc/onc_translator_shill_to_onc.cc View 1 2 3 4 5 6 2 chunks +17 lines, -0 lines 0 comments Download
M chromeos/network/onc/onc_translator_unittest.cc View 1 2 3 4 5 2 chunks +6 lines, -0 lines 0 comments Download
M chromeos/network/onc/onc_utils.h View 1 chunk +12 lines, -0 lines 0 comments Download
M chromeos/network/onc/onc_utils.cc View 1 2 3 4 5 6 7 2 chunks +246 lines, -0 lines 2 comments Download
M chromeos/network/onc/onc_utils_unittest.cc View 1 2 3 4 2 chunks +82 lines, -0 lines 0 comments Download
A + chromeos/test/data/network/proxy_config.json View 1 2 3 4 0 chunks +-1 lines, --1 lines 0 comments Download
M chromeos/test/data/network/shill_openvpn_clientcert.json View 1 chunk +1 line, -0 lines 0 comments Download
M chromeos/test/data/network/shill_wifi_clientcert.json View 1 chunk +1 line, -0 lines 0 comments Download
M chromeos/test/data/network/shill_wifi_clientref.json View 1 chunk +1 line, -0 lines 0 comments Download
A chromeos/test/data/network/shill_wifi_proxy.json View 1 chunk +6 lines, -0 lines 0 comments Download
A chromeos/test/data/network/shill_wifi_proxy_pac.json View 1 2 3 4 5 1 chunk +6 lines, -0 lines 0 comments Download
M chromeos/test/data/network/shill_wifi_psk.json View 1 chunk +1 line, -0 lines 0 comments Download
M chromeos/test/data/network/translation_of_shill_wifi_clientcert.onc View 1 chunk +4 lines, -0 lines 0 comments Download
A chromeos/test/data/network/translation_of_shill_wifi_proxy.onc View 1 chunk +32 lines, -0 lines 0 comments Download
A chromeos/test/data/network/translation_of_shill_wifi_proxy_pac.onc View 1 2 3 4 5 1 chunk +11 lines, -0 lines 0 comments Download
A chromeos/test/data/network/wifi_proxy.onc View 1 chunk +32 lines, -0 lines 0 comments Download
A chromeos/test/data/network/wifi_proxy_pac.onc View 1 2 3 4 5 1 chunk +11 lines, -0 lines 0 comments Download
M components/OWNERS View 1 1 chunk +4 lines, -0 lines 0 comments Download
M components/proxy_config.gypi View 1 chunk +1 line, -0 lines 0 comments Download
M components/proxy_config/BUILD.gn View 1 chunk +1 line, -0 lines 0 comments Download
A + components/proxy_config/DEPS View 1 chunk +1 line, -0 lines 0 comments Download
M components/proxy_config/proxy_config_dictionary.h View 2 chunks +12 lines, -1 line 0 comments Download
M components/proxy_config/proxy_config_dictionary.cc View 2 chunks +19 lines, -0 lines 0 comments Download

Dependent Patchsets:

Messages

Total messages: 46 (16 generated)
stevenjb
jochen@chromium.org for components/proxy_config.gypi and chrome/chrome_tests_unit.gypi
5 years, 5 months ago (2015-07-07 01:42:02 UTC) #2
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1228543002/1
5 years, 5 months ago (2015-07-07 01:42:40 UTC) #4
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: chromium_presubmit on tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presubmit/builds/76293)
5 years, 5 months ago (2015-07-07 01:50:49 UTC) #6
pneubeck (no reviews)
the referenced bug number is wrong. overall lgtm, the most fragile part is the added ...
5 years, 5 months ago (2015-07-07 10:18:48 UTC) #7
jochen (gone - plz use gerrit)
i don't get what this CL is about. please provide a better description
5 years, 5 months ago (2015-07-07 11:26:32 UTC) #8
stevenjb
On 2015/07/07 11:26:32, jochen wrote: > i don't get what this CL is about. please ...
5 years, 5 months ago (2015-07-07 18:30:28 UTC) #9
stevenjb
https://codereview.chromium.org/1228543002/diff/1/chromeos/network/onc/onc_utils.cc File chromeos/network/onc/onc_utils.cc (right): https://codereview.chromium.org/1228543002/diff/1/chromeos/network/onc/onc_utils.cc#newcode831 chromeos/network/onc/onc_utils.cc:831: << "PAC field is invalid for this ProxySettings.Type"; On ...
5 years, 5 months ago (2015-07-07 18:30:36 UTC) #10
pneubeck (no reviews)
https://codereview.chromium.org/1228543002/diff/20001/chromeos/network/onc/onc_utils.cc File chromeos/network/onc/onc_utils.cc (right): https://codereview.chromium.org/1228543002/diff/20001/chromeos/network/onc/onc_utils.cc#newcode822 chromeos/network/onc/onc_utils.cc:822: proxy_list = proxy_rules.MapUrlSchemeToProxyList(scheme); according to the documentation, this should ...
5 years, 5 months ago (2015-07-08 06:52:48 UTC) #11
jochen (gone - plz use gerrit)
+eroman, do those changes make sense from a net PoV? https://codereview.chromium.org/1228543002/diff/20001/chrome/browser/chromeos/ui_proxy_config.cc File chrome/browser/chromeos/ui_proxy_config.cc (right): https://codereview.chromium.org/1228543002/diff/20001/chrome/browser/chromeos/ui_proxy_config.cc#newcode124 ...
5 years, 5 months ago (2015-07-08 13:20:15 UTC) #13
stevenjb
https://codereview.chromium.org/1228543002/diff/20001/chrome/browser/chromeos/ui_proxy_config.cc File chrome/browser/chromeos/ui_proxy_config.cc (right): https://codereview.chromium.org/1228543002/diff/20001/chrome/browser/chromeos/ui_proxy_config.cc#newcode124 chrome/browser/chromeos/ui_proxy_config.cc:124: "http", http_proxy.server, &spec); On 2015/07/08 13:20:15, jochen wrote: > ...
5 years, 5 months ago (2015-07-08 16:26:15 UTC) #15
stevenjb
PTAL
5 years, 5 months ago (2015-07-08 16:34:24 UTC) #18
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1228543002/60001
5 years, 5 months ago (2015-07-08 16:35:19 UTC) #19
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1228543002/80001
5 years, 5 months ago (2015-07-08 16:45:46 UTC) #23
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: chromium_presubmit on tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presubmit/builds/77050)
5 years, 5 months ago (2015-07-08 16:56:44 UTC) #25
eroman
Looks fine from net point of view. My only concern was around a seeming lack ...
5 years, 5 months ago (2015-07-08 21:26:38 UTC) #26
stevenjb
https://codereview.chromium.org/1228543002/diff/120001/chromeos/network/onc/onc_translator_shill_to_onc.cc File chromeos/network/onc/onc_translator_shill_to_onc.cc (right): https://codereview.chromium.org/1228543002/diff/120001/chromeos/network/onc/onc_translator_shill_to_onc.cc#newcode572 chromeos/network/onc/onc_translator_shill_to_onc.cc:572: ReadDictionaryFromJson(proxy_config_str)); On 2015/07/08 21:26:37, eroman wrote: > This can ...
5 years, 5 months ago (2015-07-08 22:13:08 UTC) #27
pneubeck (no reviews)
https://codereview.chromium.org/1228543002/diff/20001/chromeos/network/onc/onc_utils.cc File chromeos/network/onc/onc_utils.cc (right): https://codereview.chromium.org/1228543002/diff/20001/chromeos/network/onc/onc_utils.cc#newcode830 chromeos/network/onc/onc_utils.cc:830: if (server.scheme() == net::ProxyServer::SCHEME_SOCKS5) On 2015/07/08 16:26:14, stevenjb wrote: ...
5 years, 5 months ago (2015-07-09 07:16:12 UTC) #28
pneubeck (no reviews)
https://codereview.chromium.org/1228543002/diff/120001/chromeos/network/onc/onc_utils.cc File chromeos/network/onc/onc_utils.cc (right): https://codereview.chromium.org/1228543002/diff/120001/chromeos/network/onc/onc_utils.cc#newcode766 chromeos/network/onc/onc_utils.cc:766: static_cast<uint16>(port))); On 2015/07/08 22:13:08, stevenjb wrote: > On 2015/07/08 ...
5 years, 5 months ago (2015-07-09 07:20:32 UTC) #29
stevenjb
Trying the SchemeToString suggestion this time. If this isn't satisfactory (at least for now), then ...
5 years, 5 months ago (2015-07-09 18:24:29 UTC) #30
pneubeck (no reviews)
Thanks for addressing the scheme prefixing. lgtm You see now why I originally only translated ...
5 years, 5 months ago (2015-07-10 07:20:43 UTC) #31
jochen (gone - plz use gerrit)
lgtm
5 years, 5 months ago (2015-07-10 09:08:47 UTC) #32
stevenjb
https://codereview.chromium.org/1228543002/diff/160001/chromeos/network/onc/onc_utils.cc File chromeos/network/onc/onc_utils.cc (right): https://codereview.chromium.org/1228543002/diff/160001/chromeos/network/onc/onc_utils.cc#newcode860 chromeos/network/onc/onc_utils.cc:860: if (server.scheme() != default_scheme) On 2015/07/10 07:20:42, pneubeck wrote: ...
5 years, 5 months ago (2015-07-10 16:16:59 UTC) #33
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1228543002/160001
5 years, 5 months ago (2015-07-10 16:17:33 UTC) #35
commit-bot: I haz the power
Try jobs failed on following builders: chromium_presubmit on tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presubmit/builds/77862)
5 years, 5 months ago (2015-07-10 16:27:20 UTC) #37
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1228543002/160001
5 years, 5 months ago (2015-07-10 20:22:52 UTC) #39
commit-bot: I haz the power
Try jobs failed on following builders: chromium_presubmit on tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presubmit/builds/77948)
5 years, 5 months ago (2015-07-10 20:34:45 UTC) #41
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1228543002/160001
5 years, 5 months ago (2015-07-10 20:45:04 UTC) #43
commit-bot: I haz the power
Committed patchset #8 (id:160001)
5 years, 5 months ago (2015-07-10 20:51:55 UTC) #44
commit-bot: I haz the power
Patchset 8 (id:??) landed as https://crrev.com/8336888d95e672e8642889cade066aed7bb1f595 Cr-Commit-Position: refs/heads/master@{#338352}
5 years, 5 months ago (2015-07-10 20:52:54 UTC) #45
eroman
5 years, 5 months ago (2015-07-13 20:08:38 UTC) #46
Message was sent while issue was closed.
https://codereview.chromium.org/1228543002/diff/20001/chromeos/network/onc/on...
File chromeos/network/onc/onc_utils.cc (right):

https://codereview.chromium.org/1228543002/diff/20001/chromeos/network/onc/on...
chromeos/network/onc/onc_utils.cc:830: if (server.scheme() ==
net::ProxyServer::SCHEME_SOCKS5)
On 2015/07/10 07:20:42, pneubeck wrote:
> On 2015/07/09 18:24:28, stevenjb wrote:
> > On 2015/07/09 07:16:12, pneubeck wrote:
> > > On 2015/07/08 16:26:14, stevenjb wrote:
> > > > On 2015/07/08 06:52:48, pneubeck wrote:
> > > > > i think you should handle all non-default schemes here. as the default
> for
> > > > > non-socks is HTTP, this could look like (analogous to lines 780, 789
and
> > > 760):
> > > > > 
> > > > > // For all proxy types except SOCKS, the default scheme of the proxy
> host
> > is
> > > > > HTTP.
> > > > > net::ProxyServer::Scheme default_scheme =
net::ProxyServer::SCHEME_HTTP;
> > > > > if (onc_scheme == ::onc::proxy::kSocks) {
> > > > >   default_scheme = net::ProxyServer::SCHEME_SOCKS4;
> > > > > }
> > > > > // Only prefix the host with a non-default scheme.
> > > > > if (server.scheme() != default_scheme)
> > > > >   host = SchemeToString(server.scheme()) + "://" + host;
> > > > 
> > > > So, there is no "SchemeToString" function, 
> > > sadly... the net/proxy code hides this conversion in the .cc .
> > > 
> > > > and effectively if I wrote one here
> > > > it would be "if (scheme == net::ProxyServer::SCHEME_SOCKS5) return
> "socks5",
> > > no, that's not right. you're adding the prefix for every _non_-default
> scheme.
> > > 
> > > > unless you are suggesting we handle SCHEME_DIRECT and SCEME_QUIC?
> > > at least SCHEME_DIRECT should not be necessary, i guess.
> > > 
> > > > I went ahead and updated the comment.
> > > 
> > > The current code will not be able to handle:
> > > 
> > > http=socks4://example.com;https=...
> > > 
> > > which must be translated to
> > > 
> > > Manual: {
> > >   "HTTP": {
> > >      host: "socks4://example.com",
> > >      port: 123
> > >   },
> > >   "HTTPS":
> > >    ....
> > > }
> > > 
> > > instead i think you're currently dropping the scheme 'socks4'.
> > > 
> > > 
> > > e.g. a similar configuration is mentioned here:
> > >
> https://www.chromium.org/developers/design-documents/network-stack/socks-proxy
> > 
> > My head is starting to hurt. I am looking at the code in
> ProxyConfig/ProxyServer
> > and am now wondering if we should even be using it here. Sigh.
> > 
> > This example seems to contradict your previous suggestion.
> I'm not sure where you see a contradiction.
> 
> > If we set
> > default_scheme == SCHEME_SOCKS4 for ::onc::proxy::kSocks, then both socks://
> and
> > socks4:// will skip the scheme since they both translate to SCHEME_SOCKS4 in
> > ProxyConfig.
> 
> As socks:// == socks4://, skipping these two in a host url seems the intended
> behavior for socks proxies.
> 
> Thinking more about it, the whole skipping the default scheme is only an
> optimization and might not be worth it. Neither skipping the default nor
always
> adding the scheme will make the round trip (ONC -> Shill -> ONC) an identity
> operation. There are always cases where we get back a different result than
what
> was originally set.
> e.g. set ONC to
>    HTTPProxy: {
>      Host: "http://1.2.3.4"
>      Port: 1234
>    }
>  => "http://" will be dropped if we skip the default scheme
> 
>  or set ONC to
>    HTTPProxy: {
>      Host: "1.2.3.4"
>      Port: 1234
>    }
>  =>  Scheme prefix "http://" will be added if we always add the prefix
> 
> 
> > I went ahead and added a SchemeToString() function to make this explicit and
> to cover QUIC
> 

There are a few factors at play here with regards to the SOCKS default
protocol...

(1) In PAC scripts, one indicates use of a SOCKS (v4) proxy by using the the
string format "SOCKS proxy:port" rather than "PROXY proxy:port". When SOCKS v5
was added, browsers required specifying it instead as "SOCKS5 proxy:port" to
disambiguate. So in this domain, "SOCKS" refers to v4.

(2) The serialization format you are using by calling
ProxyRules::ParseFromString() is actually the Windows WinInet serialization
format (with more allowances). I guess you decided to use it because there was
already code for it, but it has its own idiosyncrasies. In this format,
"socks=host:port" indicates the use of a socks v4 proxy, because that is how
Windows apps interpreted it. In fact, strictly speaking there is no support for
SOCKS v5 in this windows format, however Chrome hacks it in by interpret leading
protocol designators, such as "socks=socks5://host:port"

(3) Command line flags. This is where it gets interesting. In Chrome you can
specify proxy servers using
--proxy-server="<proxy-protocol>:://<proxy-host>:<proxy-port>". This 3 tuple
expressing the proxy protocol, host, port is represented by ProxyServer class.
Unlike the other representations this is unambiguous about he role of the proxy.
This representation looks like a URL, but isn't, it is just a convenient
serialization. It allows expressing things like "HTTPS" proxies which were a
Chrome invention (not to be confused with an HTTP proxy for https://* URLs,
which is what you are configuring with https=...)

OK, so in this format, "socks://" actually means "socks5://". We invented this
format, so for consistency we could have made socks:// mean socks4://. And in
fact is originally what we did. But because so many people were filing bugs due
to incorrectly specifying a socks 4 proxy when they mean to specify socks 5, we
switched the default here. Socks v4 is practically never what you want to be
using in Chrome. Socks v5 dates back to the 1996 and is widely supported

So to sum it up, if you want to be using ProxyRules::ParseFromString() using the
Windows semantics, then:

   socks=foobar     --> means socks v4 proxy
   socks=socks://foobar --> means socks v5 proxy
   socks=socks4://foobar --> means socks v4 proxy
   socks=socks5://foobar --> means socks v5 proxy

Where the addition of a scheme in front of the host:port is a Chrome invention
and not something that Windows would actually interpret (If you loaded these in
Internet Explorer would fail).

Hope that helps clarify the confusion.

Unfortunately the "standard" way of expressing proxy settings as a series of
host:port per URL scheme, and then SOCKS, is really quite terrible. SOCKS is not
a URL scheme like the others, but rather a proxy protocol (with implicit version
depending on who sets it). And this socks proxy is generally used as a fallback
for something not matched by the URL specific ones (even though you could
interpret it as a proxy for the transport layer in addition to the other proxy
usages).

Also include plenty of tests on how your serialized settings get interpreted,
since if ProxyRules::ParseFromString() were ever changed to match some Windows
specific behavior (which it occasionally does) you wouldn't want your
serializations to break.

</rant>

https://codereview.chromium.org/1228543002/diff/120001/chromeos/network/onc/o...
File chromeos/network/onc/onc_utils.cc (right):

https://codereview.chromium.org/1228543002/diff/120001/chromeos/network/onc/o...
chromeos/network/onc/onc_utils.cc:765:
net::HostPortPair(proxy_server.host_port_pair().host(),
On 2015/07/08 22:13:08, stevenjb wrote:
> On 2015/07/08 21:26:38, eroman wrote:
> > proxy_server may be invalid, in which case calling host_port_pair() is not
> > valid.
> 
> Since the scheme is invalid, the host_port_pair value shouldn't matter here.

It will hit a DCHECK in ProxyServer::host_port_pair()

Powered by Google App Engine
This is Rietveld 408576698