New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 877075 link

Starred by 6 users

Issue metadata

Status: Fixed
Owner:
Closed: Sep 21
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Bug
Proj-Servicification



Sign in to add a comment

Headless: SSL Intermediate Certificates aren't downloaded with AIA

Reported by beebw...@gmail.com, Aug 23

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36

Steps to reproduce the problem:
/usr/bin/google-chrome --headless --dump-dom https://incomplete-chain.badssl.com

What is the expected behavior?
DOM returned (with Chrome downloading the missing SSL intermediate certificate as defined in the certificate AIA in the background).

This works as expected on Macos 10.13.6 (High Sierra) using Google Chrome 68.0.3440.106:.

What went wrong?
[0823/115256.905080:ERROR:gpu_process_transport_factory.cc(1016)] Lost UI shared context.
[0823/115257.126541:ERROR:nss_ocsp.cc(601)] No URLRequestContext for NSS HTTP handler. host: cacerts.digicert.com
[0823/115257.126667:ERROR:cert_verify_proc_nss.cc(981)] CERT_PKIXVerifyCert for incomplete-chain.badssl.com failed err=-8179
<html><head></head><body></body></html>

Did this work before? N/A 

Chrome version: 68.0.3440.106  Channel: stable
OS Version: Ubuntu 14.04.5 LTS
Flash Version: 

Only seems to be happening on Linux - Mac OS works as expected.

Launching Chrome with the arguments --ignore-certificate-errors --enable-features=NetworkService does appear to work - until you turn on request interception (which is needed to handle htauth/htpass username/password authentication or to block/inspect urls).

See also https://github.com/GoogleChrome/puppeteer/issues/2377 / https://groups.google.com/a/chromium.org/forum/#!msg/headless-dev/CbVEqMc7Rlk/nElqclg1DgAJ
 
Confirming. Works without --headless, doesn't work with --headless.

Chrome version 68.0.3440.106 (stable channel) on Ubuntu 18.04.1:

/usr/bin/google-chrome --headless --dump-dom https://incomplete-chain.badssl.com
[0823/163049.151538:ERROR:gpu_process_transport_factory.cc(1016)] Lost UI shared context.
[0823/163049.540431:ERROR:nss_ocsp.cc(601)] No URLRequestContext for NSS HTTP handler. host: cacerts.digicert.com
[0823/163049.540548:ERROR:cert_verify_proc_nss.cc(981)] CERT_PKIXVerifyCert for incomplete-chain.badssl.com failed err=-8179
<html><head></head><body></body></html>

Chromium nightly (70.0.3531.0, build revision 585476) on Ubuntu 18.04.1:

chrome-linux/chrome --headless --dump-dom https://incomplete-chain.badssl.com
[0823/163209.683403:ERROR:gpu_process_transport_factory.cc(984)] Lost UI shared context.
[0823/163210.023419:ERROR:nss_ocsp.cc(582)] No URLRequestContext for NSS HTTP handler. host: cacerts.digicert.com
[0823/163210.023527:ERROR:cert_verify_proc_nss.cc(975)] CERT_PKIXVerifyCert for incomplete-chain.badssl.com failed err=-8179
[0823/163210.055469:FATAL:window_proxy.cc(105)] Check failed: global_proxy_.IsEmpty(). 
#0 0x55668585e2ac base::debug::StackTrace::StackTrace()
#1 0x5566857d9a70 logging::LogMessage::~LogMessage()
#2 0x556687e9bb14 blink::WindowProxy::SetGlobalProxy()
#3 0x55668834f2f7 blink::WindowProxyManager::SetGlobalProxies()
#4 0x556688481661 blink::WebFrame::Swap()
#5 0x556688ddc2c9 content::RenderFrameImpl::SwapIn()
#6 0x556688de7e8d content::RenderFrameImpl::DidCommitProvisionalLoad()
#7 0x556688572d2f blink::LocalFrameClientImpl::DispatchDidCommitLoad()
#8 0x55668894cea9 blink::DocumentLoader::DidCommitNavigation()
#9 0x55668894c26a blink::DocumentLoader::InstallNewDocument()
#10 0x55668894bfc8 blink::DocumentLoader::CommitNavigation()
#11 0x55668894b176 blink::DocumentLoader::CommitData()
#12 0x55668894ad5f blink::DocumentLoader::FinishedLoading()
#13 0x556685328f76 blink::Resource::DidAddClient()
#14 0x556685322d7b blink::RawResource::DidAddClient()
#15 0x55668532940c blink::Resource::FinishPendingClients()
#16 0x556687e7ef90 blink::TaskHandle::Runner::Run()
#17 0x5566857df1d9 base::debug::TaskAnnotator::RunTask()
#18 0x55668581c182 base::sequence_manager::internal::ThreadControllerImpl::DoWork()
#19 0x5566857df1d9 base::debug::TaskAnnotator::RunTask()
#20 0x5566857de58e base::MessageLoop::RunTask()
#21 0x5566857de922 base::MessageLoop::DoWork()
#22 0x5566857e0efa base::MessagePumpDefault::Run()
#23 0x5566857fbf95 base::RunLoop::Run()
#24 0x55668966e041 content::RendererMain()
#25 0x5566854ac8e7 content::RunZygote()
#26 0x5566854ada2d content::ContentMainRunnerImpl::Run()
#27 0x5566854e12fb service_manager::Main()
#28 0x5566854abef1 content::ContentMain()
#29 0x55668998f3ad headless::(anonymous namespace)::RunContentMain()
#30 0x55668998f25e headless::RunChildProcessIfNeeded()
#31 0x5566854dd575 headless::HeadlessShellMain()
#32 0x5566838ca1ac ChromeMain
#33 0x7fc312b0db97 __libc_start_main
#34 0x5566838ca02a _start

Received signal 6
#0 0x55668585e2ac base::debug::StackTrace::StackTrace()
#1 0x55668585de11 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#2 0x7fc318b52890 <unknown>
#3 0x7fc312b2ae97 gsignal
#4 0x7fc312b2c801 abort
#5 0x55668585cc65 base::debug::BreakDebugger()
#6 0x5566857d9ed9 logging::LogMessage::~LogMessage()
#7 0x556687e9bb14 blink::WindowProxy::SetGlobalProxy()
#8 0x55668834f2f7 blink::WindowProxyManager::SetGlobalProxies()
#9 0x556688481661 blink::WebFrame::Swap()
#10 0x556688ddc2c9 content::RenderFrameImpl::SwapIn()
#11 0x556688de7e8d content::RenderFrameImpl::DidCommitProvisionalLoad()
#12 0x556688572d2f blink::LocalFrameClientImpl::DispatchDidCommitLoad()
#13 0x55668894cea9 blink::DocumentLoader::DidCommitNavigation()
#14 0x55668894c26a blink::DocumentLoader::InstallNewDocument()
#15 0x55668894bfc8 blink::DocumentLoader::CommitNavigation()
#16 0x55668894b176 blink::DocumentLoader::CommitData()
#17 0x55668894ad5f blink::DocumentLoader::FinishedLoading()
#18 0x556685328f76 blink::Resource::DidAddClient()
#19 0x556685322d7b blink::RawResource::DidAddClient()
#20 0x55668532940c blink::Resource::FinishPendingClients()
#21 0x556687e7ef90 blink::TaskHandle::Runner::Run()
#22 0x5566857df1d9 base::debug::TaskAnnotator::RunTask()
#23 0x55668581c182 base::sequence_manager::internal::ThreadControllerImpl::DoWork()
#24 0x5566857df1d9 base::debug::TaskAnnotator::RunTask()
#25 0x5566857de58e base::MessageLoop::RunTask()
#26 0x5566857de922 base::MessageLoop::DoWork()
#27 0x5566857e0efa base::MessagePumpDefault::Run()
#28 0x5566857fbf95 base::RunLoop::Run()
#29 0x55668966e041 content::RendererMain()
#30 0x5566854ac8e7 content::RunZygote()
#31 0x5566854ada2d content::ContentMainRunnerImpl::Run()
#32 0x5566854e12fb service_manager::Main()
#33 0x5566854abef1 content::ContentMain()
#34 0x55668998f3ad headless::(anonymous namespace)::RunContentMain()
#35 0x55668998f25e headless::RunChildProcessIfNeeded()
#36 0x5566854dd575 headless::HeadlessShellMain()
#37 0x5566838ca1ac ChromeMain
#38 0x7fc312b0db97 __libc_start_main
#39 0x5566838ca02a _start
  r8: 0000000000000000  r9: 00007ffdbbe6d920 r10: 0000000000000008 r11: 0000000000000246
 r12: 00007ffdbbe6e338 r13: 0000000000000057 r14: 00007ffdbbe6e340 r15: 00007ffdbbe6e348
  di: 0000000000000002  si: 00007ffdbbe6d920  bp: 00007ffdbbe6db70  bx: 00007ffdbbe6dbd0
  dx: 0000000000000000  ax: 0000000000000000  cx: 00007fc312b2ae97  sp: 00007ffdbbe6d920
  ip: 00007fc312b2ae97 efl: 0000000000000246 cgf: 002b000000000033 erf: 0000000000000000
 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Given the below regression tests fail, I am assume that it has never worked on headless without the --ignore-certificate-errors command line flag: however, that was recently removed in Chrome and it looks like it has triggered knock on effects when interception is enabled.

I still believe that headless should operate as identically to 'headed' Chrome as possible: and if Chrome normally fetches the AIA certificates then headless should do as well. Many developers have perhaps ignored this discrepancy until now by using the ignore-certificates-errors argument/flag, but removal of that flag has highlighted issues.

Regression tests going back to November 2017: all failed.

All on Ubuntu 14.04.5 - Chrome downloaded from https://storage.googleapis.com/chromium-browser-snapshots/Linux_x64/%d/chrome-linux.zip (%d = revision number)

For the following revisions 579032 (Chromium 70.0.3208.0), 574897 (Chromium 69.0.3491.0), 571375 (Chromium 69.0.3477.0), 557152 (Chromium 68.0.3426.0), I received the the same/very similar core dump data (actual dump is from 579032):

ubuntu@host:~/chrome-linux$ ./chrome --headless --dump-dom https://incomplete-chain.badssl.com
Fontconfig warning: "/etc/fonts/fonts.conf", line 86: unknown element "blank"
[0823/150302.318220:ERROR:gpu_process_transport_factory.cc(1007)] Lost UI shared context.
[0823/150302.525605:ERROR:nss_ocsp.cc(582)] No URLRequestContext for NSS HTTP handler. host: cacerts.digicert.com
[0823/150302.525749:ERROR:cert_verify_proc_nss.cc(977)] CERT_PKIXVerifyCert for incomplete-chain.badssl.com failed err=-8179
[0823/150302.586608:FATAL:window_proxy.cc(105)] Check failed: global_proxy_.IsEmpty(). 
#0 0x7f977bbafdfc base::debug::StackTrace::StackTrace()
#1 0x7f977bb2f150 logging::LogMessage::~LogMessage()
#2 0x7f977e1957e4 blink::WindowProxy::SetGlobalProxy()
#3 0x7f977e6471a7 blink::WindowProxyManager::SetGlobalProxies()
#4 0x7f977e77b2a1 blink::WebFrame::Swap()
#5 0x7f977f0c20c9 content::RenderFrameImpl::SwapIn()
#6 0x7f977f0cdb9d content::RenderFrameImpl::DidCommitProvisionalLoad()
#7 0x7f977e86c56f blink::LocalFrameClientImpl::DispatchDidCommitLoad()
#8 0x7f977ec3f599 blink::DocumentLoader::DidCommitNavigation()
#9 0x7f977ec3e99e blink::DocumentLoader::InstallNewDocument()
#10 0x7f977ec3e6f8 blink::DocumentLoader::CommitNavigation()
#11 0x7f977ec3d8a6 blink::DocumentLoader::CommitData()
#12 0x7f977ec3d48f blink::DocumentLoader::FinishedLoading()
#13 0x7f977b67a1c6 blink::Resource::DidAddClient()
#14 0x7f977b67410b blink::RawResource::DidAddClient()
#15 0x7f977b67a65c blink::Resource::FinishPendingClients()
#16 0x7f977e178790 blink::TaskHandle::Runner::Run()
#17 0x7f977bb345c9 base::debug::TaskAnnotator::RunTask()
#18 0x7f977bb6fdd2 base::sequence_manager::internal::ThreadControllerImpl::DoWork()
#19 0x7f977bb345c9 base::debug::TaskAnnotator::RunTask()
#20 0x7f977bb33852 base::MessageLoop::RunTask()
#21 0x7f977bb33d02 base::MessageLoop::DoWork()
#22 0x7f977bb363ba base::MessagePumpDefault::Run()
#23 0x7f977bb52995 base::RunLoop::Run()
#24 0x7f977f945c4f content::RendererMain()
#25 0x7f977b819ba9 content::RunZygote()
#26 0x7f977b81afa9 content::ContentMainRunnerImpl::Run()
#27 0x7f977b84f22b service_manager::Main()
#28 0x7f977b8191a1 content::ContentMain()
#29 0x7f977fc54e9d headless::(anonymous namespace)::RunContentMain()
#30 0x7f977fc54d4e headless::RunChildProcessIfNeeded()
#31 0x7f977b84b585 headless::HeadlessShellMain()
#32 0x7f9779c991ac ChromeMain
#33 0x7f977133bf45 __libc_start_main
#34 0x7f9779c9902a _start

Received signal 6
#0 0x7f977bbafdfc base::debug::StackTrace::StackTrace()
#1 0x7f977bbaf961 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#2 0x7f9776fc6330 <unknown>
#3 0x7f9771350c37 gsignal
#4 0x7f9771354028 abort
#5 0x7f977bbae7b5 base::debug::BreakDebugger()
#6 0x7f977bb2f5b9 logging::LogMessage::~LogMessage()
#7 0x7f977e1957e4 blink::WindowProxy::SetGlobalProxy()
#8 0x7f977e6471a7 blink::WindowProxyManager::SetGlobalProxies()
#9 0x7f977e77b2a1 blink::WebFrame::Swap()
#10 0x7f977f0c20c9 content::RenderFrameImpl::SwapIn()
#11 0x7f977f0cdb9d content::RenderFrameImpl::DidCommitProvisionalLoad()
#12 0x7f977e86c56f blink::LocalFrameClientImpl::DispatchDidCommitLoad()
#13 0x7f977ec3f599 blink::DocumentLoader::DidCommitNavigation()
#14 0x7f977ec3e99e blink::DocumentLoader::InstallNewDocument()
#15 0x7f977ec3e6f8 blink::DocumentLoader::CommitNavigation()
#16 0x7f977ec3d8a6 blink::DocumentLoader::CommitData()
#17 0x7f977ec3d48f blink::DocumentLoader::FinishedLoading()
#18 0x7f977b67a1c6 blink::Resource::DidAddClient()
#19 0x7f977b67410b blink::RawResource::DidAddClient()
#20 0x7f977b67a65c blink::Resource::FinishPendingClients()
#21 0x7f977e178790 blink::TaskHandle::Runner::Run()
#22 0x7f977bb345c9 base::debug::TaskAnnotator::RunTask()
#23 0x7f977bb6fdd2 base::sequence_manager::internal::ThreadControllerImpl::DoWork()
#24 0x7f977bb345c9 base::debug::TaskAnnotator::RunTask()
#25 0x7f977bb33852 base::MessageLoop::RunTask()
#26 0x7f977bb33d02 base::MessageLoop::DoWork()
#27 0x7f977bb363ba base::MessagePumpDefault::Run()
#28 0x7f977bb52995 base::RunLoop::Run()
#29 0x7f977f945c4f content::RendererMain()
#30 0x7f977b819ba9 content::RunZygote()
#31 0x7f977b81afa9 content::ContentMainRunnerImpl::Run()
#32 0x7f977b84f22b service_manager::Main()
#33 0x7f977b8191a1 content::ContentMain()
#34 0x7f977fc54e9d headless::(anonymous namespace)::RunContentMain()
#35 0x7f977fc54d4e headless::RunChildProcessIfNeeded()
#36 0x7f977b84b585 headless::HeadlessShellMain()
#37 0x7f9779c991ac ChromeMain
#38 0x7f977133bf45 __libc_start_main
#39 0x7f9779c9902a _start
  r8: 00007f97775caa40  r9: 000029dd03807800 r10: 0000000000000008 r11: 0000000000000202
 r12: 00007ffd0fdd7c18 r13: 0000000000000057 r14: 00007ffd0fdd7c20 r15: 00007ffd0fdd7c28
  di: 0000000000000001  si: 0000000000000001  bp: 00007ffd0fdd7570  bx: 00007ffd0fdd75e0
  dx: 0000000000000006  ax: 0000000000000000  cx: ffffffffffffffff  sp: 00007ffd0fdd7438
  ip: 00007f9771350c37 efl: 0000000000000202 cgf: 0000000000000033 erf: 0000000000000000
 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.

For revision 568432 (Chromium 69.0.3466.0), 553380 (Chromium 68.0.3406.0), 546920 (Chromium 67.0.3384.0) and 513435 (Chromium 64.0.3257.0):
Fontconfig warning: "/etc/fonts/fonts.conf", line 146: blank doesn't take any effect anymore. please remove it from your fonts.conf
[0823/151108.598903:ERROR:gpu_process_transport_factory.cc(1007)] Lost UI shared context.
[0823/151108.844671:ERROR:nss_ocsp.cc(582)] No URLRequestContext for NSS HTTP handler. host: cacerts.digicert.com
[0823/151108.844806:ERROR:cert_verify_proc_nss.cc(981)] CERT_PKIXVerifyCert for incomplete-chain.badssl.com failed err=-8179
<html><head></head><body></body></html>
Labels: Needs-Triage-M68
Cc: mmenke@chromium.org
Components: -Platform>DevTools Internals>Network>Certificate Internals>Headless
Status: Untriaged (was: Unconfirmed)
Thanks for the report! I can confirm that's not working.

It looks like the issue is that headless mode doesn't call into SetURLRequestContextForNSSHttpIO. These days that's done by creating a network::NetworkContext with primary_network_context toggled to true. That's the "system NetworkContext" (SystemNetworkContextManager).

Probably headless mode needs one of those too. Or if it's only got the one NetworkContext, that one can probably be set to primary? +mmenke since I'm not sure what all the implications are.
I don't think anyone's even looked at what's needed to get headless working with the network service.  I'd just call SetURLRequestContextForNSSHttpIO directly for now.
If it's going to be a while to fix, would it be possible to re-introduce the ignore-certificate-errors flag functionality which allowed this to work (using the devtools protocol Security.setIgnoreCertificateErrors doesn't function the same in respect to AIA intermediate certificates as I'm guessing it operates on a different context/thread).
Summary: Headless: SSL Intermediate Certificates aren't downloaded with AIA (was: Linux: SSL Intermediate Certificates aren't downloaded with AIA)
Components: Internals>Services>Network
While using the network service will fix this, I don't think this is really associated with the network service?  It was broken before I moved configuration of the setting over to the network service, too, since Headless doesn't use IOThread (At least I assume it doesn't depend on chrome/).
Cc: pfeldman@chromium.org
+Pavel, who's looking into turning on NS for headless to fix some other bugs
Owner: caseq@chromium.org
Status: Assigned (was: Untriaged)
Do we have a timeline to get NS/Intermediate Certificates working ?

I'm just trying to figure out I should move on to another tech until this is fixed or wait for the fix.
Ping.
Status: Started (was: Assigned)
Project Member

Comment 16 by bugdroid1@chromium.org, Sep 21

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/455dc67aff774c2dd920212092bd470a1cfe2e8e

commit 455dc67aff774c2dd920212092bd470a1cfe2e8e
Author: Andrey Kosyakov <caseq@chromium.org>
Date: Fri Sep 21 18:07:51 2018

Support fetching missing intermediate certificates in headless

Drive-by: remove references to NetLog.

Bug:  877075 
Change-Id: I9fa4df67d89793754f8502a4e756f86c84571129
Reviewed-on: https://chromium-review.googlesource.com/1232616
Reviewed-by: Dmitry Gozman <dgozman@chromium.org>
Reviewed-by: Ryan Sleevi <rsleevi@chromium.org>
Commit-Queue: Andrey Kosyakov <caseq@chromium.org>
Cr-Commit-Position: refs/heads/master@{#593256}
[modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/browser/headless_browser_context_impl.cc
[modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/browser/headless_browser_context_impl.h
[modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/browser/headless_browser_impl.cc
[modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/browser/headless_browser_impl.h
[modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/browser/headless_url_request_context_getter.cc
[modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/browser/headless_url_request_context_getter.h
[modify] https://crrev.com/455dc67aff774c2dd920212092bd470a1cfe2e8e/headless/lib/headless_browser_browsertest.cc

Cc: lushnikov@chromium.org
Status: Fixed (was: Started)

Sign in to add a comment