Collect data to help determine a pruning policy for the HSTS preload list |
|||||
Issue description
I have a basic scanning script [1] that can output error codes.
Of the 2768 sites, there are many that are no longer eligible despite the requirements stated at [2]
Errors:
800 redirects.http.www_first
391 redirects.http.no_redirect
180 redirects.http.first_redirect.insecure
130 response.no_header
110 redirects.insecure.initial
86 redirects.follow_error
80 header.preloadable.include_sub_domains.missing
67 domain.tls.cannot_connect
39 header.preloadable.preload.missing
33 redirects.insecure.subsequent
21 response.multiple_headers
13 redirects.too_many
9 domain.www.no_tls
9 header.preloadable.max_age.too_low
7 redirects.http.first_redirect.no_hsts
5 header.preloadable.max_age.zero
4 redirects.http.first_redirect.invalid
Warnings:
1816 header.parse.spelling.include_sub_domains
56 header.parse.empty_directive
3 header.parse.unknown_directive
3 header.preloadable.max_age.over_10_years
(Note that some of these conditions are stricter than the current submission script, but will be enforced for new submissions soon. Notably, the top 3 errors were previously either not enforced or not correctly enforced.)
Based on a discussion today with agl@, I want to run a version of this script daily to see how many sites continue to have various errors. If a site is consistently offline, we may stop preloading.
[1] https://github.com/chromium/hstspreload/commit/ea737a2e0ab938bd23ca22c54699cbc9d7539f08
[2] https://hstspreload.appspot.com/
,
May 20 2016
I just did a scan of all 12341 current HSTS entries. Here are counts of issues with the entries:
Errors:
{
"domain.format.only_one_label": 1,
"domain.is_subdomain": 373,
"domain.tls.cannot_connect": 764,
"domain.tls.invalid_cert_chain": 362,
"domain.tls.sha1": 22,
"domain.www.no_tls": 164,
"header.parse.max_age.non_digit_characters": 1,
"header.preloadable.include_sub_domains.missing": 669,
"header.preloadable.max_age.missing": 3,
"header.preloadable.max_age.too_low": 69,
"header.preloadable.max_age.zero": 41,
"header.preloadable.preload.missing": 578,
"internal.domain.name.cannot_compute_etld1": 4,
"internal.redirects.http.first_probe_failed": 215,
"redirects.follow_error": 345,
"redirects.http.first_redirect.insecure": 549,
"redirects.http.first_redirect.invalid": 54,
"redirects.http.first_redirect.no_hsts": 73,
"redirects.http.no_redirect": 1231,
"redirects.http.www_first": 2626,
"redirects.insecure.initial": 324,
"redirects.insecure.subsequent": 137,
"redirects.too_many": 84,
"response.multiple_headers": 75,
"response.no_header": 950
}
Warnings:
{
"header.parse.empty_directive": 311,
"header.parse.invalid.preload": 5,
"header.parse.spelling.include_sub_domains": 5433,
"header.parse.unknown_directive": 22,
"header.preloadable.max_age.over_10_years": 61,
"redirects.http.useless_header": 2778
}
Note that this scan includes manual preload entries that would not be subject to all normal submission requirements, such as entries without includeSubDomains, as well as the eTLD "google".
In total, 5421/12600 of the domains in the scan would be eligible for automated submission under the newly tightened requirements.
In particular, note that just under 10% are currently inaccessible in Chrome (domain.tls.cannot_connect) or (domain.tls.invalid_cert_chain).
,
May 20 2016
* 5421/12341
,
Oct 26 2016
,
Jan 5 2017
,
Dec 1 2017
,
Dec 8 2017
,
May 1 2018
Hey, Nick-- This is one of a few ambiguous workitems related to evaluating Chrome's long-term plans with regard to the preload list and its scalability. |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by lgar...@chromium.org
, May 12 2016