Issue metadata
Sign in to add a comment
|
Headless: SSL Intermediate Certificates aren't downloaded with AIA
Reported by
beebw...@gmail.com,
Aug 23
|
||||||||||||||||||||||||
Issue descriptionUserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36 Steps to reproduce the problem: /usr/bin/google-chrome --headless --dump-dom https://incomplete-chain.badssl.com What is the expected behavior? DOM returned (with Chrome downloading the missing SSL intermediate certificate as defined in the certificate AIA in the background). This works as expected on Macos 10.13.6 (High Sierra) using Google Chrome 68.0.3440.106:. What went wrong? [0823/115256.905080:ERROR:gpu_process_transport_factory.cc(1016)] Lost UI shared context. [0823/115257.126541:ERROR:nss_ocsp.cc(601)] No URLRequestContext for NSS HTTP handler. host: cacerts.digicert.com [0823/115257.126667:ERROR:cert_verify_proc_nss.cc(981)] CERT_PKIXVerifyCert for incomplete-chain.badssl.com failed err=-8179 <html><head></head><body></body></html> Did this work before? N/A Chrome version: 68.0.3440.106 Channel: stable OS Version: Ubuntu 14.04.5 LTS Flash Version: Only seems to be happening on Linux - Mac OS works as expected. Launching Chrome with the arguments --ignore-certificate-errors --enable-features=NetworkService does appear to work - until you turn on request interception (which is needed to handle htauth/htpass username/password authentication or to block/inspect urls). See also https://github.com/GoogleChrome/puppeteer/issues/2377 / https://groups.google.com/a/chromium.org/forum/#!msg/headless-dev/CbVEqMc7Rlk/nElqclg1DgAJ
,
Aug 23
Given the below regression tests fail, I am assume that it has never worked on headless without the --ignore-certificate-errors command line flag: however, that was recently removed in Chrome and it looks like it has triggered knock on effects when interception is enabled. I still believe that headless should operate as identically to 'headed' Chrome as possible: and if Chrome normally fetches the AIA certificates then headless should do as well. Many developers have perhaps ignored this discrepancy until now by using the ignore-certificates-errors argument/flag, but removal of that flag has highlighted issues. Regression tests going back to November 2017: all failed. All on Ubuntu 14.04.5 - Chrome downloaded from https://storage.googleapis.com/chromium-browser-snapshots/Linux_x64/%d/chrome-linux.zip (%d = revision number) For the following revisions 579032 (Chromium 70.0.3208.0), 574897 (Chromium 69.0.3491.0), 571375 (Chromium 69.0.3477.0), 557152 (Chromium 68.0.3426.0), I received the the same/very similar core dump data (actual dump is from 579032): ubuntu@host:~/chrome-linux$ ./chrome --headless --dump-dom https://incomplete-chain.badssl.com Fontconfig warning: "/etc/fonts/fonts.conf", line 86: unknown element "blank" [0823/150302.318220:ERROR:gpu_process_transport_factory.cc(1007)] Lost UI shared context. [0823/150302.525605:ERROR:nss_ocsp.cc(582)] No URLRequestContext for NSS HTTP handler. host: cacerts.digicert.com [0823/150302.525749:ERROR:cert_verify_proc_nss.cc(977)] CERT_PKIXVerifyCert for incomplete-chain.badssl.com failed err=-8179 [0823/150302.586608:FATAL:window_proxy.cc(105)] Check failed: global_proxy_.IsEmpty(). #0 0x7f977bbafdfc base::debug::StackTrace::StackTrace() #1 0x7f977bb2f150 logging::LogMessage::~LogMessage() #2 0x7f977e1957e4 blink::WindowProxy::SetGlobalProxy() #3 0x7f977e6471a7 blink::WindowProxyManager::SetGlobalProxies() #4 0x7f977e77b2a1 blink::WebFrame::Swap() #5 0x7f977f0c20c9 content::RenderFrameImpl::SwapIn() #6 0x7f977f0cdb9d content::RenderFrameImpl::DidCommitProvisionalLoad() #7 0x7f977e86c56f blink::LocalFrameClientImpl::DispatchDidCommitLoad() #8 0x7f977ec3f599 blink::DocumentLoader::DidCommitNavigation() #9 0x7f977ec3e99e blink::DocumentLoader::InstallNewDocument() #10 0x7f977ec3e6f8 blink::DocumentLoader::CommitNavigation() #11 0x7f977ec3d8a6 blink::DocumentLoader::CommitData() #12 0x7f977ec3d48f blink::DocumentLoader::FinishedLoading() #13 0x7f977b67a1c6 blink::Resource::DidAddClient() #14 0x7f977b67410b blink::RawResource::DidAddClient() #15 0x7f977b67a65c blink::Resource::FinishPendingClients() #16 0x7f977e178790 blink::TaskHandle::Runner::Run() #17 0x7f977bb345c9 base::debug::TaskAnnotator::RunTask() #18 0x7f977bb6fdd2 base::sequence_manager::internal::ThreadControllerImpl::DoWork() #19 0x7f977bb345c9 base::debug::TaskAnnotator::RunTask() #20 0x7f977bb33852 base::MessageLoop::RunTask() #21 0x7f977bb33d02 base::MessageLoop::DoWork() #22 0x7f977bb363ba base::MessagePumpDefault::Run() #23 0x7f977bb52995 base::RunLoop::Run() #24 0x7f977f945c4f content::RendererMain() #25 0x7f977b819ba9 content::RunZygote() #26 0x7f977b81afa9 content::ContentMainRunnerImpl::Run() #27 0x7f977b84f22b service_manager::Main() #28 0x7f977b8191a1 content::ContentMain() #29 0x7f977fc54e9d headless::(anonymous namespace)::RunContentMain() #30 0x7f977fc54d4e headless::RunChildProcessIfNeeded() #31 0x7f977b84b585 headless::HeadlessShellMain() #32 0x7f9779c991ac ChromeMain #33 0x7f977133bf45 __libc_start_main #34 0x7f9779c9902a _start Received signal 6 #0 0x7f977bbafdfc base::debug::StackTrace::StackTrace() #1 0x7f977bbaf961 base::debug::(anonymous namespace)::StackDumpSignalHandler() #2 0x7f9776fc6330 <unknown> #3 0x7f9771350c37 gsignal #4 0x7f9771354028 abort #5 0x7f977bbae7b5 base::debug::BreakDebugger() #6 0x7f977bb2f5b9 logging::LogMessage::~LogMessage() #7 0x7f977e1957e4 blink::WindowProxy::SetGlobalProxy() #8 0x7f977e6471a7 blink::WindowProxyManager::SetGlobalProxies() #9 0x7f977e77b2a1 blink::WebFrame::Swap() #10 0x7f977f0c20c9 content::RenderFrameImpl::SwapIn() #11 0x7f977f0cdb9d content::RenderFrameImpl::DidCommitProvisionalLoad() #12 0x7f977e86c56f blink::LocalFrameClientImpl::DispatchDidCommitLoad() #13 0x7f977ec3f599 blink::DocumentLoader::DidCommitNavigation() #14 0x7f977ec3e99e blink::DocumentLoader::InstallNewDocument() #15 0x7f977ec3e6f8 blink::DocumentLoader::CommitNavigation() #16 0x7f977ec3d8a6 blink::DocumentLoader::CommitData() #17 0x7f977ec3d48f blink::DocumentLoader::FinishedLoading() #18 0x7f977b67a1c6 blink::Resource::DidAddClient() #19 0x7f977b67410b blink::RawResource::DidAddClient() #20 0x7f977b67a65c blink::Resource::FinishPendingClients() #21 0x7f977e178790 blink::TaskHandle::Runner::Run() #22 0x7f977bb345c9 base::debug::TaskAnnotator::RunTask() #23 0x7f977bb6fdd2 base::sequence_manager::internal::ThreadControllerImpl::DoWork() #24 0x7f977bb345c9 base::debug::TaskAnnotator::RunTask() #25 0x7f977bb33852 base::MessageLoop::RunTask() #26 0x7f977bb33d02 base::MessageLoop::DoWork() #27 0x7f977bb363ba base::MessagePumpDefault::Run() #28 0x7f977bb52995 base::RunLoop::Run() #29 0x7f977f945c4f content::RendererMain() #30 0x7f977b819ba9 content::RunZygote() #31 0x7f977b81afa9 content::ContentMainRunnerImpl::Run() #32 0x7f977b84f22b service_manager::Main() #33 0x7f977b8191a1 content::ContentMain() #34 0x7f977fc54e9d headless::(anonymous namespace)::RunContentMain() #35 0x7f977fc54d4e headless::RunChildProcessIfNeeded() #36 0x7f977b84b585 headless::HeadlessShellMain() #37 0x7f9779c991ac ChromeMain #38 0x7f977133bf45 __libc_start_main #39 0x7f9779c9902a _start r8: 00007f97775caa40 r9: 000029dd03807800 r10: 0000000000000008 r11: 0000000000000202 r12: 00007ffd0fdd7c18 r13: 0000000000000057 r14: 00007ffd0fdd7c20 r15: 00007ffd0fdd7c28 di: 0000000000000001 si: 0000000000000001 bp: 00007ffd0fdd7570 bx: 00007ffd0fdd75e0 dx: 0000000000000006 ax: 0000000000000000 cx: ffffffffffffffff sp: 00007ffd0fdd7438 ip: 00007f9771350c37 efl: 0000000000000202 cgf: 0000000000000033 erf: 0000000000000000 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000 [end of stack trace] Calling _exit(1). Core file will not be generated. For revision 568432 (Chromium 69.0.3466.0), 553380 (Chromium 68.0.3406.0), 546920 (Chromium 67.0.3384.0) and 513435 (Chromium 64.0.3257.0): Fontconfig warning: "/etc/fonts/fonts.conf", line 146: blank doesn't take any effect anymore. please remove it from your fonts.conf [0823/151108.598903:ERROR:gpu_process_transport_factory.cc(1007)] Lost UI shared context. [0823/151108.844671:ERROR:nss_ocsp.cc(582)] No URLRequestContext for NSS HTTP handler. host: cacerts.digicert.com [0823/151108.844806:ERROR:cert_verify_proc_nss.cc(981)] CERT_PKIXVerifyCert for incomplete-chain.badssl.com failed err=-8179 <html><head></head><body></body></html>
,
Aug 23
See also https://github.com/GoogleChrome/puppeteer/issues/1159 and https://bugs.chromium.org/p/chromium/issues/detail?id=801426 which all seem related.
,
Aug 23
,
Aug 23
Thanks for the report! I can confirm that's not working. It looks like the issue is that headless mode doesn't call into SetURLRequestContextForNSSHttpIO. These days that's done by creating a network::NetworkContext with primary_network_context toggled to true. That's the "system NetworkContext" (SystemNetworkContextManager). Probably headless mode needs one of those too. Or if it's only got the one NetworkContext, that one can probably be set to primary? +mmenke since I'm not sure what all the implications are.
,
Aug 23
I don't think anyone's even looked at what's needed to get headless working with the network service. I'd just call SetURLRequestContextForNSSHttpIO directly for now.
,
Aug 24
If it's going to be a while to fix, would it be possible to re-introduce the ignore-certificate-errors flag functionality which allowed this to work (using the devtools protocol Security.setIgnoreCertificateErrors doesn't function the same in respect to AIA intermediate certificates as I'm guessing it operates on a different context/thread).
,
Aug 24
,
Aug 27
,
Aug 27
While using the network service will fix this, I don't think this is really associated with the network service? It was broken before I moved configuration of the setting over to the network service, too, since Headless doesn't use IOThread (At least I assume it doesn't depend on chrome/).
,
Aug 28
+Pavel, who's looking into turning on NS for headless to fix some other bugs
,
Aug 28
,
Sep 7
Do we have a timeline to get NS/Intermediate Certificates working ? I'm just trying to figure out I should move on to another tech until this is fixed or wait for the fix.
,
Sep 18
Ping.
,
Sep 18
,
Sep 21
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/455dc67aff774c2dd920212092bd470a1cfe2e8e commit 455dc67aff774c2dd920212092bd470a1cfe2e8e Author: Andrey Kosyakov <caseq@chromium.org> Date: Fri Sep 21 18:07:51 2018 Support fetching missing intermediate certificates in headless Drive-by: remove references to NetLog. Bug: 877075 Change-Id: I9fa4df67d89793754f8502a4e756f86c84571129 Reviewed-on: https://chromium-review.googlesource.com/1232616 Reviewed-by: Dmitry Gozman <dgozman@chromium.org> Reviewed-by: Ryan Sleevi <rsleevi@chromium.org> Commit-Queue: Andrey Kosyakov <caseq@chromium.org> Cr-Commit-Position: refs/heads/master@{#593256} [modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/browser/headless_browser_context_impl.cc [modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/browser/headless_browser_context_impl.h [modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/browser/headless_browser_impl.cc [modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/browser/headless_browser_impl.h [modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/browser/headless_url_request_context_getter.cc [modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/browser/headless_url_request_context_getter.h [modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/headless_browser_browsertest.cc
,
Sep 21
|
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by anje0...@student.miun.se
, Aug 23