Add additional CT metrics |
|||||||
Issue descriptionSome things we should record (if we're not already): - % of connections with EV certificates that lose their EV status due to lack of CT compliance - SCTs by log and source (TLS extension/OCSP/embedded) - SCTs by cross-product of logs - % of tls connections that are CT compliant
,
Oct 18 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/0d9809ed8636ad3d07d58fae201bfd347dcd6319 commit 0d9809ed8636ad3d07d58fae201bfd347dcd6319 Author: Emily Stark <estark@google.com> Date: Wed Oct 18 08:29:15 2017 Add two additional CT histograms - What percentage of connections with EV certificates are CT-compliant? We used to have this histogram but removed it when the EV whitelist was removed. I think it's still useful to have around, as it helps us gauge how often connections that are supposed to be CT-compliant are in fact CT-compliant. For example, a high rate of non-compliant EV certs might indicate that a CA is logging improperly. - What percentage of connections where CT is required are actually CT-compliant? As above, this helps us judge whether servers that are supposed to send CT actually are. The existing count that we have of the ERR_CERTIFICATE_TRANSPARENCY_REQUIRED error code is not enough to tell us this, because it doesn't tell us how many connections were actually supposed to have CT, only how many were supposed to but didn't. The latter histogram requires a modification to the return value of TransportSecurityState::CheckCTRequirements; instead of returning early when a connection is compliant and telling the caller that CT requirements were met, it now differentiates "CT requirements met" from "CT wasn't required". (An alternative approach is to record the histogram inside CheckCTRequirements itself and not modify its return value, but I thought it was preferable to minimize side effects inside that method [even though it does already have other side effects, unfortunately].) Bug: 772534 Change-Id: Ifd3d6f72475e07e1470b0d0b171f9ce0a325807e Reviewed-on: https://chromium-review.googlesource.com/707994 Commit-Queue: Emily Stark <estark@chromium.org> Reviewed-by: Steven Holte <holte@chromium.org> Reviewed-by: Matt Mueller <mattm@chromium.org> Cr-Commit-Position: refs/heads/master@{#509716} [modify] https://crrev.com/0d9809ed8636ad3d07d58fae201bfd347dcd6319/net/http/transport_security_state.cc [modify] https://crrev.com/0d9809ed8636ad3d07d58fae201bfd347dcd6319/net/http/transport_security_state.h [modify] https://crrev.com/0d9809ed8636ad3d07d58fae201bfd347dcd6319/net/http/transport_security_state_unittest.cc [modify] https://crrev.com/0d9809ed8636ad3d07d58fae201bfd347dcd6319/net/quic/chromium/crypto/proof_verifier_chromium.cc [modify] https://crrev.com/0d9809ed8636ad3d07d58fae201bfd347dcd6319/net/quic/chromium/crypto/proof_verifier_chromium_test.cc [modify] https://crrev.com/0d9809ed8636ad3d07d58fae201bfd347dcd6319/net/socket/ssl_client_socket_impl.cc [modify] https://crrev.com/0d9809ed8636ad3d07d58fae201bfd347dcd6319/net/socket/ssl_client_socket_unittest.cc [modify] https://crrev.com/0d9809ed8636ad3d07d58fae201bfd347dcd6319/net/spdy/chromium/spdy_session.cc [modify] https://crrev.com/0d9809ed8636ad3d07d58fae201bfd347dcd6319/tools/metrics/histograms/histograms.xml
,
Nov 21 2017
,
Nov 23 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8b411de0034dd83ccd909207895b4044f15c4066 commit 8b411de0034dd83ccd909207895b4044f15c4066 Author: Emily Stark <estark@google.com> Date: Thu Nov 23 20:21:27 2017 Add request-level CT metrics We already have per-connection CT metrics for percent of connections that are CT-compliant (both overall and of connections which are required to be CT-compliant). This CL adds per-request metrics, which may give a different picture: for example, it's possible that overall compliance might look high as a percentage of overall connections but low as a percentage of requests, because one non-compliant connection might be responsible for a disproportionate number of requests. Bug: 772534 Cq-Include-Trybots: master.tryserver.chromium.android:android_cronet_tester;master.tryserver.chromium.mac:ios-simulator-cronet Change-Id: I7c54dc7621588505ac5c30cf9d8d704cb6b59b59 Reviewed-on: https://chromium-review.googlesource.com/769295 Reviewed-by: Jesse Doherty <jwd@chromium.org> Reviewed-by: Ryan Sleevi <rsleevi@chromium.org> Reviewed-by: Matt Mueller <mattm@chromium.org> Commit-Queue: Emily Stark <estark@chromium.org> Cr-Commit-Position: refs/heads/master@{#519015} [modify] https://crrev.com/8b411de0034dd83ccd909207895b4044f15c4066/net/cert/ct_verify_result.cc [modify] https://crrev.com/8b411de0034dd83ccd909207895b4044f15c4066/net/cert/ct_verify_result.h [modify] https://crrev.com/8b411de0034dd83ccd909207895b4044f15c4066/net/quic/chromium/crypto/proof_verifier_chromium.cc [modify] https://crrev.com/8b411de0034dd83ccd909207895b4044f15c4066/net/quic/chromium/crypto/proof_verifier_chromium_test.cc [modify] https://crrev.com/8b411de0034dd83ccd909207895b4044f15c4066/net/socket/ssl_client_socket_impl.cc [modify] https://crrev.com/8b411de0034dd83ccd909207895b4044f15c4066/net/socket/ssl_client_socket_unittest.cc [modify] https://crrev.com/8b411de0034dd83ccd909207895b4044f15c4066/net/ssl/ssl_info.cc [modify] https://crrev.com/8b411de0034dd83ccd909207895b4044f15c4066/net/ssl/ssl_info.h [modify] https://crrev.com/8b411de0034dd83ccd909207895b4044f15c4066/net/url_request/url_request_http_job.cc [modify] https://crrev.com/8b411de0034dd83ccd909207895b4044f15c4066/net/url_request/url_request_http_job_unittest.cc [modify] https://crrev.com/8b411de0034dd83ccd909207895b4044f15c4066/tools/metrics/histograms/histograms.xml
,
Dec 1 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4e63e1a3e1822929140bbff850dfc3d8a530599c commit 4e63e1a3e1822929140bbff850dfc3d8a530599c Author: Emily Stark <estark@google.com> Date: Fri Dec 01 06:08:11 2017 Add UseCounters for subresources with CT errors When a subresource fails to load with net::ERR_CERTIFICATE_TRANSPARENCY_REQUIRED, increment a UseCounter for the document. This gives an upper-bound estimate on the percent of page loads with user-visible CT-induced breakage. (A subresource failing to load might or might not be user-visible breakage, which is why it's an upper bound.) This metric will be useful for evaluating user impact as we roll out more Certificate Transparency enforcement. Note that this does not yet count subframe errors, due to some larger PlzNavigate-related problems with how iframe errors are sent to renderers. ( https://crbug.com/750901 ) Bug: 772534 Change-Id: Ib2367aea4556e355c9c4b3406199c177b0a6be79 Reviewed-on: https://chromium-review.googlesource.com/798639 Commit-Queue: Emily Stark <estark@chromium.org> Reviewed-by: Mike West <mkwst@chromium.org> Cr-Commit-Position: refs/heads/master@{#520868} [modify] https://crrev.com/4e63e1a3e1822929140bbff850dfc3d8a530599c/third_party/WebKit/Source/core/loader/FrameFetchContext.cpp [modify] https://crrev.com/4e63e1a3e1822929140bbff850dfc3d8a530599c/third_party/WebKit/Source/core/loader/WorkerFetchContext.cpp [modify] https://crrev.com/4e63e1a3e1822929140bbff850dfc3d8a530599c/third_party/WebKit/Source/platform/network/NetworkUtils.cpp [modify] https://crrev.com/4e63e1a3e1822929140bbff850dfc3d8a530599c/third_party/WebKit/Source/platform/network/NetworkUtils.h [modify] https://crrev.com/4e63e1a3e1822929140bbff850dfc3d8a530599c/third_party/WebKit/public/platform/web_feature.mojom [modify] https://crrev.com/4e63e1a3e1822929140bbff850dfc3d8a530599c/tools/metrics/histograms/enums.xml
,
Dec 2 2017
Requesting a merge to M64 for the commit in comment 5. This is a simple UseCounter addition that just missed branch and would be really nice to have in M64 due to the upcoming Certificate Transparency enforcement date in April (https://groups.google.com/a/chromium.org/forum/#!topic/ct-policy/sz_3W_xKBNY)
,
Dec 3 2017
Your change meets the bar and is auto-approved for M64. Please go ahead and merge the CL to branch 3282 manually. Please contact milestone owner if you have questions. Owners: cmasso@(Android), cmasso@(iOS), kbleicher@(ChromeOS), abdulsyed@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Dec 4 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ca6444ba4211d0c522c2ccd8d24a9338fce63cc9 commit ca6444ba4211d0c522c2ccd8d24a9338fce63cc9 Author: Emily Stark <estark@google.com> Date: Mon Dec 04 19:21:11 2017 Add UseCounters for subresources with CT errors When a subresource fails to load with net::ERR_CERTIFICATE_TRANSPARENCY_REQUIRED, increment a UseCounter for the document. This gives an upper-bound estimate on the percent of page loads with user-visible CT-induced breakage. (A subresource failing to load might or might not be user-visible breakage, which is why it's an upper bound.) This metric will be useful for evaluating user impact as we roll out more Certificate Transparency enforcement. Note that this does not yet count subframe errors, due to some larger PlzNavigate-related problems with how iframe errors are sent to renderers. ( https://crbug.com/750901 ) Bug: 772534 Change-Id: Ib2367aea4556e355c9c4b3406199c177b0a6be79 Reviewed-on: https://chromium-review.googlesource.com/798639 Commit-Queue: Emily Stark <estark@chromium.org> Reviewed-by: Mike West <mkwst@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#520868}(cherry picked from commit 4e63e1a3e1822929140bbff850dfc3d8a530599c) Reviewed-on: https://chromium-review.googlesource.com/806316 Reviewed-by: Emily Stark <estark@chromium.org> Cr-Commit-Position: refs/branch-heads/3282@{#16} Cr-Branched-From: 5fdc0fab22ce7efd32532ee989b223fa12f8171e-refs/heads/master@{#520840} [modify] https://crrev.com/ca6444ba4211d0c522c2ccd8d24a9338fce63cc9/third_party/WebKit/Source/core/loader/FrameFetchContext.cpp [modify] https://crrev.com/ca6444ba4211d0c522c2ccd8d24a9338fce63cc9/third_party/WebKit/Source/core/loader/WorkerFetchContext.cpp [modify] https://crrev.com/ca6444ba4211d0c522c2ccd8d24a9338fce63cc9/third_party/WebKit/Source/platform/network/NetworkUtils.cpp [modify] https://crrev.com/ca6444ba4211d0c522c2ccd8d24a9338fce63cc9/third_party/WebKit/Source/platform/network/NetworkUtils.h [modify] https://crrev.com/ca6444ba4211d0c522c2ccd8d24a9338fce63cc9/third_party/WebKit/public/platform/web_feature.mojom [modify] https://crrev.com/ca6444ba4211d0c522c2ccd8d24a9338fce63cc9/tools/metrics/histograms/enums.xml
,
Dec 8 2017
,
Jan 18 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2819badcc28625556abfe9f4c704072eb563b230 commit 2819badcc28625556abfe9f4c704072eb563b230 Author: Emily Stark <estark@google.com> Date: Thu Jan 18 15:03:11 2018 Add CT compliance histogram for main frame navigation resources This histogram will be used to help evaluate user-visible breakage for when we turn on CT enforcement universally. Bug: 772534 Change-Id: Ie4382cad1e70d19b3d008546a009f13e00683d2f Reviewed-on: https://chromium-review.googlesource.com/872450 Reviewed-by: Camille Lamy <clamy@chromium.org> Reviewed-by: Jesse Doherty <jwd@chromium.org> Commit-Queue: Emily Stark <estark@chromium.org> Cr-Commit-Position: refs/heads/master@{#530143} [modify] https://crrev.com/2819badcc28625556abfe9f4c704072eb563b230/chrome/browser/ssl/security_state_tab_helper.cc [modify] https://crrev.com/2819badcc28625556abfe9f4c704072eb563b230/chrome/browser/ssl/security_state_tab_helper_browsertest.cc [modify] https://crrev.com/2819badcc28625556abfe9f4c704072eb563b230/content/public/browser/ssl_status.cc [modify] https://crrev.com/2819badcc28625556abfe9f4c704072eb563b230/content/public/browser/ssl_status.h [modify] https://crrev.com/2819badcc28625556abfe9f4c704072eb563b230/tools/metrics/histograms/histograms.xml
,
Feb 4 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d617b2faad73e49eede9b9e5b8d341153f865e0d commit d617b2faad73e49eede9b9e5b8d341153f865e0d Author: Emily Stark <estark@google.com> Date: Sun Feb 04 01:34:47 2018 Plumb CT compliance status into Blink and record with a UseCounter For now, this is only used for metrics. We want to record the number of page loads that contain at least one non-CT-compliant resource; this acts as an upper bound on the number of pages that would break when we turn on CT enforcement. (This requires plumbing into Blink for lack of a way to associate a subresource with a document/page visit in the browser process.) In future, we probably want to use this information to display in the DevTools security panel, to help developers identify non-CT-compliant subresources on their websites. Bug: 772534 Cq-Include-Trybots: master.tryserver.chromium.linux:linux_mojo Change-Id: Ifaa00d7a37e1a46240fac7bce543edfc31202df0 Reviewed-on: https://chromium-review.googlesource.com/895055 Commit-Queue: Emily Stark <estark@chromium.org> Reviewed-by: Mike West <mkwst@chromium.org> Reviewed-by: Yutaka Hirano <yhirano@chromium.org> Cr-Commit-Position: refs/heads/master@{#534288} [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/content/browser/loader/resource_loader.cc [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/content/renderer/loader/web_url_loader_impl.cc [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/services/network/public/cpp/network_param_ipc_traits.h [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/services/network/public/cpp/resource_response.cc [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/services/network/public/cpp/resource_response_info.cc [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/services/network/public/cpp/resource_response_info.h [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/services/network/url_loader.cc [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/third_party/WebKit/Source/core/loader/FrameFetchContext.cpp [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/third_party/WebKit/Source/platform/exported/WebURLResponse.cpp [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/third_party/WebKit/Source/platform/loader/fetch/ResourceResponse.cpp [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/third_party/WebKit/Source/platform/loader/fetch/ResourceResponse.h [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/third_party/WebKit/public/platform/DEPS [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/third_party/WebKit/public/platform/WebURLResponse.h [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/third_party/WebKit/public/platform/web_feature.mojom [modify] https://crrev.com/d617b2faad73e49eede9b9e5b8d341153f865e0d/tools/metrics/histograms/enums.xml
,
May 1 2018
@estark: Fixed/closable?
,
May 2 2018
Yep! |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by bugdroid1@chromium.org
, Oct 10 2017