New issue
Advanced search Search tips

Issue 922318 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Today
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Android
Pri: 2
Type: Bug-Regression

Blocking:
issue 638435
issue 914587
issue 922463



Sign in to add a comment

GPU bot "Linux Release (NVIDIA)" is failing multiple test suites

Project Member Reported by senorblanco@chromium.org, Jan 16

Issue description

Many test suite steps (see below) are failing (purple) with a "missing shard #0"

Regression range is a single commit (r622873).

List of failing suites:
context_lost_tests
depth_capture_tests
pixel_test
hardware_accelerated_feature_tests
info_collection_tests
screenshot_sync_tests
gpu_process_launch_tests
trace_test
webgl_conformance_tests
tab_capture_end2end_tests
maps_pixel_test
 
Sample stack trace:

[1:1:0115/161106.985916:FATAL:document.h(465)] Check failed: style_engine_.Get(). 
#0 0x7fa534bc267f base::debug::StackTrace::StackTrace()
#1 0x7fa534b074da logging::LogMessage::~LogMessage()
#2 0x7fa537dec3b6 blink::Element::RecalcOwnStyle()
#3 0x7fa537debac9 blink::Element::RecalcStyle()
#4 0x7fa537cfa676 blink::StyleEngine::RecalcStyle()
#5 0x7fa537ca427d blink::Document::UpdateStyle()
#6 0x7fa537c9f2f0 blink::Document::UpdateStyleAndLayoutTree()
#7 0x7fa537cb6f60 blink::Document::FinishedParsing()
#8 0x7fa538838cea blink::HTMLConstructionSite::FinishedParsing()
#9 0x7fa538894a74 blink::HTMLTreeBuilder::Finished()
#10 0x7fa538827fe6 blink::HTMLDocumentParser::end()

Failures on "Android Release (Nexus 5X)" started on the same commit (purple shard steps), although the stack trace looks different:

	Stack Trace:
	  RELADDR   FUNCTION                                                                                                                                                                                                                                                                                                                                                                                                               FILE:LINE
	  0000000000210089  __stack_chk_fail                                                                                                                                                                                                                                                                                                                                                                                                       ??:0:0
	  v------>  blink::(anonymous namespace)::DispatchEventsOnWindowAndFocusedElement(blink::Document*, bool)                                                                                                                                                                                                                                                                                                                          ../../third_party/blink/renderer/core/page/focus_controller.cc:410:22
	  00000000068e8be8  blink::FocusController::FocusHasChanged()                                                                                                                                                                                                                                                                                                                                                                              ../../third_party/blink/renderer/core/page/focus_controller.cc:911:0
	  00000000064860b0  blink::WebViewImpl::SetFocus(bool)                                                                                                                                                                                                                                                                                                                                                                                     ../../third_party/blink/renderer/core/exported/web_view_impl.cc:1840:39
	  000000000700d9dc  content::RenderWidget::SetFocus(bool)                                                                                                                                                                                                                                                                                                                                                                                  ../../content/renderer/render_widget.cc:899:21
	  000000000233787c  base::OnceCallback<void ()>::Run() &&                                                                                                                                                                                                                                                                                                                                                                                  ../../base/callback.h:99:12
	  000000000492c4e4  void base::internal::FunctorTraits<void (*)(base::WeakPtr<net::CookieMonster>, base::OnceCallback<void ()>), void>::Invoke<void (*)(base::WeakPtr<net::CookieMonster>, base::OnceCallback<void ()>), base::WeakPtr<net::CookieMonster>, base::OnceCallback<void ()> >(void (*&&)(base::WeakPtr<net::CookieMonster>, base::OnceCallback<void ()>), base::WeakPtr<net::CookieMonster>&&, base::OnceCallback<void ()>&&)  ../../base/bind_internal.h:416:12
	  000000000233787c  base::OnceCallback<void ()>::Run() &&                                                                                                                                                                                                                                                                                                                                                                                  ../../base/callback.h:99:12
	  0000000006f1b608  content::MainThreadEventQueue::DispatchEvents()                                                                                                       

Labels: -Pri-3 OS-Android OS-Linux Pri-1
Summary: GPU bot "Linux Release (NVIDIA)" is failing multiple test suites (was: GPU bot "Linux Release (NVIDIA)" is failing multiple test suites (sharding failure?))
Blocking: 914587 638435
Cc: kmarshall@chromium.org
Components: -Blink Blink>Loader
Labels: -Pri-1 -Type-Bug Pri-2 Type-Bug-Regression
Owner: japhet@chromium.org
Status: Assigned (was: Untriaged)
senorblanco@: thanks for filing this. This was severe enough breakage that it warranted an immediate revert. In the future please go ahead and do this. I'm not sure how the original CL made it through the commit queue.

kmarshall@ , thanks for reverting in https://chromium.googlesource.com/chromium/src/+/98b0b75172690b363de3975c5540b94ea666ba07 . https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20FYI%20Release%20%28NVIDIA%29/7731 is the first green build containing that revert.

japhet@, I'm mystified how your original CL https://chromium-review.googlesource.com/1387448 passed the CQ. Is it possible it collided with some other CL that landed just before yours did? Could you please figure out what caused this breakage before relanding, and run the "optional" GPU trybots (win_optional_gpu_tests_rel, linux_optional_gpu_tests_rel, mac_optional_gpu_tests_rel) on your revised CL? Thanks.

Downgrading to P2 now that the immediate regression has been addressed.

Comment 6 by senorblanco@chromium.org, Jan 16 (6 days ago)

Yeah, normally I would've reverted right away, but this was the first commit after the tree had been closed all day, so I thought it might be bot issues. Anyway glad to see the revert cleared it up.

Comment 7 by japhet@chromium.org, Jan 16 (6 days ago)

Started running the optional test with my CL as reverted on https://chromium-review.googlesource.com/c/chromium/src/+/1415312. They're green, in spite of nothing having changed.

I have to say, I'm a bit confused about the stack trace in comment #1. style_engine_ is initialized in the constructor and never nulled out. It's unclear to me how that DCHECK would ever be tripped. I guess I'll try to reland as-is?

Comment 8 by kbr@chromium.org, Jan 16 (6 days ago)

Components: Infra>Client>Chrome
I'm also not sure how the CL made it through the CQ but interestingly the very next build started failing catastrophically again:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20FYI%20Release%20%28NVIDIA%29/7732
https://chromium-swarm.appspot.com/task?id=426d9ec3e6d1ce10&refresh=10&show_raw=1

[1636:1636:0115/195949.958352:WARNING:personal_data_manager.cc(487)] 0x150a4ba6a180 refresh is done, notifying PersonalDataChanged
[1:1:0115/195949.816287:FATAL:hash_table.h(764)] Check failed: !AccessForbidden(). 
#0 0x7f461045a50f base::debug::StackTrace::StackTrace()
#1 0x7f461039f9aa logging::LogMessage::~LogMessage()
#2 0x7f4613e390da blink::LocalDOMWindow::DispatchLoadEvent()
#3 0x7f4613e38d64 blink::LocalDOMWindow::DispatchWindowLoadEvent()
#4 0x7f4613e3923e blink::LocalDOMWindow::DocumentWasClosed()
#5 0x7f461353527d blink::Document::ImplicitClose()
#6 0x7f4613535725 blink::Document::CheckCompletedInternal()
#7 0x7f461352333e blink::Document::CheckCompleted()
#8 0x7f46144c41e9 blink::FrameLoader::FinishedParsing()
#9 0x7f46135424f4 blink::Document::FinishedParsing()
#10 0x7f46140c3f9a blink::HTMLConstructionSite::FinishedParsing()
#11 0x7f461411f754 blink::HTMLTreeBuilder::Finished()
#12 0x7f46140b3296 blink::HTMLDocumentParser::end()
#13 0x7f46140ad538 blink::HTMLDocumentParser::AttemptToRunDeferredScriptsAndEnd()
#14 0x7f46140ad3f1 blink::HTMLDocumentParser::PrepareToStopParsing()
#15 0x7f46140b361b blink::HTMLDocumentParser::Finish()
#16 0x7f46144b18e6 blink::DocumentLoader::FinishedLoading()
#17 0x7f46144b4c9b blink::DocumentLoader::MaybeLoadEmpty()
#18 0x7f46144b4dfa blink::DocumentLoader::StartLoading()
#19 0x7f46144c739e blink::FrameLoader::CommitNavigation()
#20 0x7f4613ec9699 blink::WebLocalFrameImpl::CommitNavigation()
#21 0x7f4614bc2b3b content::RenderFrameImpl::CommitNavigation()
#22 0x7f460d91716b content::mojom::FrameNavigationControlStubDispatch::AcceptWithResponder()
#23 0x7f4614bdc7e6 content::mojom::FrameNavigationControlStub<>::AcceptWithResponder()
#24 0x7f46104c1b3d mojo::InterfaceEndpointClient::HandleValidatedMessage()
#25 0x7f46104c4386 mojo::FilterChain::Accept()
#26 0x7f46104c3062 mojo::InterfaceEndpointClient::HandleIncomingMessage()
#27 0x7f4610b0f1ef IPC::(anonymous namespace)::ChannelAssociatedGroupController::AcceptOnProxyThread()
#28 0x7f4610b0c531 base::internal::Invoker<>::Run()
#29 0x7f46103a8f29 base::debug::TaskAnnotator::RunTask()
#30 0x7f461040285a base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWorkImpl()
#31 0x7f4610402dd4 base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork()
#32 0x7f46103aac4a base::MessagePumpDefault::Run()
#33 0x7f4610403179 base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::Run()
#34 0x7f46103d2160 base::RunLoop::Run()
#35 0x7f46156a07f4 content::RendererMain()
#36 0x7f460fed4ed1 content::RunZygote()
#37 0x7f460fed622e content::ContentMainRunnerImpl::Run()
#38 0x7f460ff09886 service_manager::Main()
#39 0x7f460fed4421 content::ContentMain()
#40 0x7f460cdd11b3 ChromeMain
#41 0x7f4603b69f45 __libc_start_main
#42 0x7f460cdd102a _start

and all runs were failing until this one:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20FYI%20Release%20%28NVIDIA%29/7737

I don't see which CL in the blamelist broke these tests again and fixed it. Getting nervous that the blamelists may not be correct or complete; there used to be a 50-item limit and these builds are running up against it. Adding Infra>Client>Chrome to Components.

Not sure how to proceed. japhet@, if you reland please watch this bot and make sure it's still green afterward:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20FYI%20Release%20%28NVIDIA%29

Comment 9 by kbr@chromium.org, Jan 17 (6 days ago)

Note that there was a widespread infra failure that may have caused corrupt object files to show up on some builders. It seems unlikely to me, since there is a single builder which drivers this bot, but perhaps that could be the cause of these widespread and mysterious failures.

Comment 10 by kbr@chromium.org, Jan 17 (6 days ago)

Blocking: 922463
Project Member

Comment 11 by bugdroid1@chromium.org, Jan 18 (4 days ago)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0

commit ebbcef1f17bc37f5380bd7e87a17016defd1fdd0
Author: Nate Chapin <japhet@chromium.org>
Date: Fri Jan 18 08:21:24 2019

Run Javascript urls through the document, attempt #2

Put them on a timer and have them block the load event. This guarantees
that JS urls will always run asynchronously, improving spec compliance.
It also will allow us to depend far less on NavigationScheduler for
our async navigation needs, hopefully allow us to delete that class
someday soon.

Bug: 638435, 914587,  922318 ,  922330 

Change-Id: I9b72af88009115a55167afa42e7206d54bb5c2c1
Reviewed-on: https://chromium-review.googlesource.com/c/1415312
Commit-Queue: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Dmitry Gozman <dgozman@chromium.org>
Auto-Submit: Nate Chapin <japhet@chromium.org>
Cr-Commit-Position: refs/heads/master@{#624035}
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/chrome/test/data/extensions/api_test/content_scripts/about_blank_iframes/end.js
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/chrome/test/data/extensions/api_test/content_scripts/about_blank_iframes/manifest.json
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/chrome/test/data/extensions/api_test/content_scripts/about_blank_iframes/start.js
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/chrome/test/data/extensions/api_test/content_scripts/about_blank_iframes/test.js
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/renderer/bindings/core/v8/script_controller.cc
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/renderer/core/dom/document.cc
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/renderer/core/dom/document.h
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/renderer/core/exported/web_view_test.cc
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/renderer/core/html/forms/html_form_element.cc
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/renderer/core/html/html_frame_owner_element.cc
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/renderer/core/html/lazy_load_frame_observer_test.cc
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/renderer/core/loader/frame_loader.cc
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/renderer/core/loader/navigation_scheduler.cc
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/TestExpectations
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/editing/inserting/insert-with-javascript-protocol-crash-expected.txt
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/external/wpt/dom/nodes/Document-contentType/contentType/contenttype_javascripturi-expected.txt
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/external/wpt/html/browsers/browsing-the-web/navigating-across-documents/013-expected.txt
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/external/wpt/html/browsers/browsing-the-web/navigating-across-documents/014-expected.txt
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/external/wpt/html/browsers/browsing-the-web/navigating-across-documents/015-expected.txt
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/external/wpt/html/browsers/browsing-the-web/navigating-across-documents/javascript-url-abort/javascript-url-abort-return-value-string.tentative-expected.txt
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/external/wpt/html/semantics/scripting-1/the-script-element/execution-timing/029-expected.txt
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/external/wpt/html/semantics/scripting-1/the-script-element/execution-timing/030-expected.txt
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/external/wpt/html/semantics/scripting-1/the-script-element/execution-timing/040-expected.txt
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/external/wpt/html/semantics/scripting-1/the-script-element/execution-timing/080-expected.txt
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/external/wpt/html/semantics/scripting-1/the-script-element/execution-timing/108-expected.txt
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/external/wpt/html/semantics/scripting-1/the-script-element/execution-timing/109-expected.txt
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/dom/Element/id-in-frameset-expected.txt
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/dom/insertedIntoDocument-iframe.html
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/fast/dom/javascript-url-crash-function-expected.txt
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/fast/dom/javascript-url-crash-function.html
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/fast/dom/javascript-url-exception-isolation-expected.txt
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/fast/dom/javascript-url-exception-isolation.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/dom/replaced-image-map.html
[delete] https://crrev.com/06d7a5b8113780d08793bff5f157e6228f9f002c/third_party/blink/web_tests/fast/dom/resources/javascript-url-crash-function-iframe.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/events/access-key-self-destruct.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/events/click-anchor-blur-refocus-window-expected.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/events/click-anchor-refocus-window-expected.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/events/click-svganchor-blur-refocus-window-expected.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/events/click-svganchor-refocus-window-expected.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/events/domactivate-sets-underlying-click-event-as-handled-expected.txt
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/events/domactivate-sets-underlying-click-event-as-handled.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/forms/formaction-attribute.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/frames/adopt-from-created-document.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/frames/navigation-in-pagehide.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/frames/out-of-document-iframe-has-child-frame.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/loader/detach-iframe-while-loading-javascript-url-expected.txt
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/loader/detach-iframe-while-loading-javascript-url.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/loader/detach-while-printing-expected.txt
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/loader/detach-while-printing.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/loader/document-open-iframe-then-detach-expected.txt
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/loader/document-open-iframe-then-detach.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/parser/xml-error-adopted.xml
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/replaced/image-map-bug16782.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/fast/replaced/image-map.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/http/tests/navigation/lockedhistory-iframe-expected.txt
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/http/tests/navigation/lockedhistory-iframe.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/http/tests/security/javascriptURL/xss-ALLOWED-from-javascript-url-sub-frame.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/http/tests/security/popup-allowed-by-sandbox-is-sandboxed-control.html
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/http/tests/security/xssAuditor/resources/javascript-link-safe.php
[modify] https://crrev.com/ebbcef1f17bc37f5380bd7e87a17016defd1fdd0/third_party/blink/web_tests/paint/invalidation/scroll/repaint-during-scroll-with-zoom.html

Comment 12 by kbr@chromium.org, Yesterday (35 hours ago)

How's this bot doing? If it's still green let's close this as WontFix (not reproducible).

Comment 13 by senorblanco@chromium.org, Today (14 hours ago)

Status: WontFix (was: Assigned)
Seems fine now. Thanks for checking in!

Sign in to add a comment