Companion security  bug 787894  bug can be merged here

Ooops typo: first line should read
As a workaround users can install the root cert for Google Internet Authority G3 directly to Trusted Root Certificates.

Unable to reproduce this issue on reported version 63.0.3239.52 using windows. Naviagated to www.gmail.com and observed no error.

As per comment#0 this issue is seen only with some users. Hence adding appropriate label.

Could someone from Internals>Network>Certificate team take a look into this.

Thanks!

I have deleted the comment advising users to install those certificates, as advice to install such certificates represents a security risk to users, as well as long-term interoperability and compatibility risks from the manual installation.

Users who encounter such an issue are generally symptomatic of a system configuration (noted because of the similar failure in Microsoft Browsers). I'm marking this as WontFix as it's an OS-level issue, not a Chrome-specific issue.

General steps to resolve include running Windows Update to install the latest patches, which may also correct system corruption. Alternatively, running the command in a command console:

certutil -f -verifyCTL AuthRootWU

May also correct it.

rsleevi@ scoo335@: thanks for checking in.
The failure is only reproducible on affected/damaged user Windows machines.

This is a Windows cert update failure, affecting the Edge, IE, Chrome equally.
Won't fix is correct in that there is nothing to fix in Chrome, but I'm trying to gain some visibility about how to repair the user windows configuration.

The problem is well documented in Eric Lawrence's G3 blog post
https://textslashplain.com/2017/10/23/google-internet-authority-g3/

Agreed, posting the root certs is a security risk, but the affected users are in a catch-22 situation.  
Their mail.google.com connection fails.
The instructions for fixing the problem (Eric's G3 post) are not reachable for the same reason.
Attachments at the chrome help forum are not accessible for the same reason.

I was hoping to use this CR as another resource.

The posted certs can be vetted by comparing (thumbprints) to the known valid instances.  With chrome, visit mail.google.com, compare the root cert to the   attached GlobalSign-R2 cert.  Ditto for Eric's blog and the DST X3 root cert.  See next comment for attachments.

re #5 certutil -f -verifyCTL AuthRootWU

-verifyCTL is not available in Win7, where many of the failures are being reported.

In my Win10 home machine (otherwise healthy), the command loads a lot of unneeded root certs and then dies before adding GlobalSign R2.

See attached log.  The command was run from an admin cmd session.

Deleted: certutilAdmin.log 86.9 KB

certutilAdmin.log 86.9 KB View Download

rsleeve@ #5: Can you explain: "as well as long-term interoperability and compatibility risks from the manual installation"?

The root update failure affects only isolated users.  I'm looking for local (historic) scenarios that prevent automatic root updates.

Windows update does not fix the problem.  I have at least 2 users with regular win updts that still see the problem.

FYI: the corresponding intermediate certs also not being cached to the cert store (inspect with certmgr.msc).  For GMail the intermediate G3 is not being cached.  Since the intermediate is always dynamically loaded the impact is only a cache miss.

I again want to stress that advising users to install these intermediates as trusted roots is actively harmful to users stability and security. These comments will be deleted and, if continued, may be necessary to suspend posting privileges, much like it would be if commentors were advising users to install 3rd party software that would undermine their security.

rsleevi@ #11 - Thanks for being respectful of my intent.  I think we're in a grey area here, but I understand your responsibilities too, to maintain a safe environment.  Until we can establish an acceptable approach, I will defer further suggestions to install root certs.

Re: 'to install these intermediates as trusted roots' - The certs I referenced are root certs, not intermediates.  They are demonstrably in the list of Microsoft recognized root certs, and are routinely auto-updated.  I gave instructions on how to validate their authenticity.  Installing them manually does not change the security level.  Normally they are automatically installed.

Can you describe how manually installing these root certs, that, except for an unknown reason, would normally be auto-updated, creates a long-term risk for the user?  This may help explain, what users may have done in the past that is blocking the current auto-update.  I'm trying to uncover why these few users have a problem receiving root updates.

Sorry for interrupting the holiday.

Installing intermediates-as-roots can cause security enforcement to be bypassed.
Installing these roots-as-roots means that if/when they are removed/restricted, the locally installed settings will override any Microsoft-supplied settings. This has happened with several CAs in the past.
Installing these roots can also introduce non-determinism in path-building by adding new variations of possible certification paths and stores consulted, causing less-than-reliable behaviour.

The overall goal should be to restore the Windows functionality. Windows update (and notably, the security rollups) should do this as a matter of course, but I'm also trying to find an alternative suggestion (the old way was manually running 'rootsupd.exe', which was the Authroot Rollup, but that's no longer distributed.)

rsleevi@ - Thanks.  

Since the manually installed roots, in this case, are taken from the failed cert path, I don't see a new variations, non-deterministic impact.

I will ask Erik about impacts for the manually installed GlobalSign R2 certs, when R2 phases out.  I see there is a GlobalSign R3 and R5.

Agreed, restoring Windows functionality is the goal. Still looking..

I can track the failure point in net-export to the last windows request.
I think I can see the failure event in the Windows event log, but
I'm not sure what to do about it.

Most of the reports seem to be from win7 and a few win8 users, which means these are older machines with a lot of history.  The depth of the history, makes trying to isolate the config error cause difficult.

All of the cases are on private home networks - no admin or policy issues.

Nearly all are for Windows Home - gpedit and policies are rarely available or an issue.

I realize that it may not be obvious why installing roots from the trust path is harmful - after all, it seems like a logically appropriate thing - it is not the same. Installing certificates manually - whether intermediates or roots - go into a different store and trigger a host of different behaviours, including preventing future updates. This is why it should not be done, particularly with 'publicly trusted' root certificates - you should limit it to those truly under enterprise control.

KB 931125 is the correct fix for Windows 7, 2008 R2, Vista, and earlier. https://support.microsoft.com/en-us/help/931125/how-to-get-a-root-certificate-update-for-windows 

Unfortunately, this is no longer provided by Microsoft as far as I can tell (or it's triggering redirects for macOS to not download)

Microsoft Help/931125 was one of the first articles I read. The first part concerns repairs for a November 2013 problem. I stopped there last time.

The tail has re-sync tools (Easy Fix downloads) through Win8.1, i.e. past Win7 but not for Win10.

Description: This Easy fix solution deletes the Certificates folder and everything inside it, and it also deletes the LastSyncTime registry key. 

Is that a reasonable manual approach for Win10?

I'll post back when I have results on Win7, Win8x machines.

The 931125 article has a weird/storied history - Microsoft updated the executable with every root store update, but did not update the article.

Since then, the root store update is no longer available (AFACT), including via WSUS - but even the last fix (from 2014/2015) is sufficient to 'reset' the internal corruption and get updates flowing again.

The 'Easy Fix' solution did more than just that (including resetting WinHTTP proxy settings in the event of proxy misconfig, since AuthRoot uses the WinHTTP/BITS proxy config, not the user-proxy-config), but that's hard to summarize succintly.

The functionality of rootsupd.exe (such as extracting the CAB and sideloading the SST) have been folded into certutil.exe's functionality, but if you have an older machine, the rootsupd.exe approach - as long as it's ~2 years - is sufficient to fix crypt32.dll corruption or registry redirecting.

Per the 9/11 Google Security Blog, Chrome 66 will distrust Symantec and partner certs issued before 6/1/17.  Partners include GeoTrust.  

Does distrust include the GeoTrust Global CA valid 5/20/0202 - 5/202022
root cert used by many *.google.com services 
with the intermediate Google Internet Authority G2 valid 5/22/17 - 12/31/18?
Examples: {productforums, myaccount, support}.google.com 

Per the article, DevTools is already providing alerts.  
What do these look like?
Is there a demo server, like badSSL, that will trigger the alerts?

BTW I saw your name on earlier Google security blog posts for Symantec issues. Well done.

9/11 Security Blog: 
https://security.googleblog.com/2017/09/chromes-plan-to-distrust-symantec.html

Dev-tools warnings appear as !Warnings on the console top taskbar, and look like:
The SSL certificate used to load resources from  https://… will be distrusted in M70. Once distrusted, users will be prevented from loading these resources. See https://g.co/chrome/symantecpkicerts for more information.

For Windows Dev-tools the messages differentiate between M66 and M70.
Linux omits the Mxx ref.

Look here for the Dev-tools discussion:
https://groups.google.com/forum/#!topic/google-chrome-developer-tools/QWgY65oGH64

rseelvi@:
The Help/931125 tool link is stale and goes to a generic EasyFix page.
931125 contains a ref near the top to Help/2801679, which does contain an active tool(50974), and per the help page deletes
  HKLM\SOFTWARE\Microsoft\SystemCertificates\AuthRoot\Certificates
which will then auto-refresh as needed.

I don't know if it does more internally.  The tool fix was originally targeted to Windows Server 2008, 20012 but will probably work for Win7/8/10 too.

TBD if it clears the config problem blocking root cert updates.
Is the 50974 EasyFix tool worth pursuing?

Look here: 
https://support.microsoft.com/en-us/help/2801679/ssl-tls-communication-problems-after-you-install-kb-931125

I tried the easyFix 50974 (see Help/2801679).  It won't run for win10, doubtful if will run for other (client, non-server) win versions.

The G3 failure (for damaged users) is a symptom of a broader problem:

-The 'Connection not private' HSTS text is misleading. This is a config problem - the site will never be accessible later as the text states.

-The diag check for root cert auto update failure is trivial, but no tool is provided - to distinguish this failure mode from other bad cert problems

-The CTL approach does not work for .mil.  The militaryCAC service provides their own root cert update fix: InstallRoot (.msi package).  See:
https://militarycac.com/dodcerts.htm

I mention the .mil issue because it is another example of a root cert update fix.  With a decent help page, this type of fix should be available as a general solution, but that's a topic I can't discuss here...