New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 917040 link

Starred by 3 users

Issue metadata

Status: Assigned
Owner:
OOO until 2019-01-24
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: Bug

Blocked on:
issue 917264
issue 920003

Blocking:
issue 917079
issue 914447



Sign in to add a comment

False rejects caused by pixel_tests on macOS

Project Member Reported by erikc...@chromium.org, Dec 20

Issue description

CL: https://chromium-review.googlesource.com/c/chromium/src/+/1382670/3

Failing build: https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_chromium_rel_ng/212638

Build succeeds on retry:
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_chromium_rel_ng/212654

Failing task: https://chromium-swarm.appspot.com/task?id=41e391142db38810&refresh=10&show_raw=1
There appears to have been several reference image mismatches, and later there was a crash.

"""
9238:775:1220/003743.727815:INFO:CONSOLE(14)] "undefined", source: http://127.0.0.1:51970/content/test/data/gpu/pixel_webgl_sad_canvas.html (14)
Received signal 11 SEGV_MAPERR 000000000000
DevTools console [network]: Failed to load resource: the server responded with a status of 404 (File not found) http://127.0.0.1:51970/favicon.ico
0   Chromium Framework                  0x00000001233c7f5f base::debug::StackTrace::StackTrace(unsigned long) + 31
1   Chromium Framework                  0x00000001233c7db1 base::debug::(anonymous namespace)::StackDumpSignalHandler(int, __siginfo*, void*) + 2385
2   libsystem_platform.dylib            0x00007fff4fd70f5a _sigtramp + 26
3   ???                                 0x00007fbbf0809208 0x0 + 140445170569736
4   Chromium Framework                  0x000000012545853c gpu::GpuChannel::OnControlMessageReceived(IPC::Message const&) + 236
5   Chromium Framework                  0x000000012545986c gpu::GpuChannel::HandleMessageHelper(IPC::Message const&) + 140
6   Chromium Framework                  0x000000012545a970 base::internal::Invoker<base::internal::BindState<void (gpu::GpuChannel::*)(IPC::Message const&), base::WeakPtr<gpu::GpuChannel>, IPC::Message>, void ()>::RunOnce(base::internal::BindStateBase*) + 192
7   Chromium Framework                  0x00000001232adee1 base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) + 305
8   Chromium Framework                  0x000000012335d0fa base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWorkImpl(base::TimeTicks*) + 650
9   Chromium Framework                  0x00000001232e1bf3 base::MessagePumpCFRunLoopBase::RunWork() + 51
10  Chromium Framework                  0x00000001232c76da base::mac::CallWithEHFrame(void () block_pointer) + 10
11  Chromium Framework                  0x00000001232e13cf base::MessagePumpCFRunLoopBase::RunWorkSource(void*) + 63
12  CoreFoundation                      0x00007fff27c43a11 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
13  CoreFoundation                      0x00007fff27cfd42c __CFRunLoopDoSource0 + 108
14  CoreFoundation                      0x00007fff27c26470 __CFRunLoopDoSources0 + 208
15  CoreFoundation                      0x00007fff27c258ed __CFRunLoopRun + 1293
16  CoreFoundation                      0x00007fff27c25153 CFRunLoopRunSpecific + 483
17  Foundation                          0x00007fff29d21f26 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 277
18  Chromium Framework                  0x00000001232e2461 base::MessagePumpNSRunLoop::DoRun(base::MessagePump::Delegate*) + 113
19  Chromium Framework                  0x00000001232e0d3f base::MessagePumpCFRunLoopBase::Run(base::MessagePump::Delegate*) + 127
20  Chromium Framework                  0x000000012335d86b base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::Run(bool) + 155
21  Chromium Framework                  0x000000012331a1b9 base::RunLoop::Run() + 249
22  Chromium Framework                  0x00000001294f3374 content::GpuMain(content::MainFunctionParams const&) + 1412
23  Chromium Framework                  0x0000000122c90503 content::ContentMainRunnerImpl::Run(bool) + 435
24  Chromium Framework                  0x0000000126291a55 service_manager::Main(service_manager::MainParams const&) + 3173
25  Chromium Framework                  0x0000000122c8f614 content::ContentMain(content::ContentMainParams const&) + 68
26  Chromium Framework                  0x000000011ec2715f ChromeMain + 175
27  Chromium Helper                     0x000000010efd272e main + 494
28  libdyld.dylib                       0x00007fff4fa62015 start + 1
[end of stack trace]
[9238:775:1220/003743.930106:INFO:CONSOLE(0)] "WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost", source: http://127.0.0.1:51970/content/test/data/gpu/pixel_webgl_sad_canvas.html (0)
"""

This test was run on 12/20 3:30AM EST, which was after this revert landed [from a previous problem]:
https://chromium-review.googlesource.com/c/chromium/src/+/1283010
 
Blocking: 914447
Labels: -Pri-3 Pri-2
Owner: kbr@chromium.org
Status: Assigned (was: Untriaged)
The crash is deliberate - caused by the Pixel_WebGLSadCanvas test, which crashes the GPU process and ensures that the "sad canvas" is drawn.

The other pixel test failures are real, and mysterious - looking at the image diffs (from this log line):
"See http://chromium-browser-gpu-tests.commondatastorage.googleapis.com/view_test_results.html?f181416280cf39f3e7782dee044d78adf89b3dfc_mac_chromium_rel_ng_telemetry for this run's test results"

the failing tests:
Pixel_OffscreenCanvasTransferToImageBitmap
Pixel_WebGLGreenTriangle_AA_Alpha

show ~8 little red dots in the image diffs. I might chalk this up to broken hardware but need to investigate more.

There are some other recent failures:

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_chromium_rel_ng/212890
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_chromium_rel_ng/212889
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_chromium_rel_ng/212845

Need to look into these to see if there's a pattern.

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_chromium_rel_ng/212890
https://chromium-swarm.appspot.com/task?id=41e553969d041e10&refresh=10&show_raw=1
Pixel_OffscreenCanvasWebGLSoftwareCompositing failed
http://chromium-browser-gpu-tests.commondatastorage.googleapis.com/view_test_results.html?6c224c93f54e09a7f409de3ac96229b65ecc6705_mac_chromium_rel_ng_telemetry

Similar failure mode, cluster of garbage pixels off to the side of the rendered image.

Was bot 397-m4:
https://chromium-swarm.appspot.com/bot?id=build397-m4&sort_stats=total%3Adesc


https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_chromium_rel_ng/212889
https://chromium-swarm.appspot.com/task?id=41e550c383733910&refresh=10&show_raw=1
Pixel_OffscreenCanvasTransferToImageBitmap ; similar failure mode.

http://chromium-browser-gpu-tests.commondatastorage.googleapis.com/view_test_results.html?fb9e65b1547f9135e2dee472b5f8c02d5ae8b248_mac_chromium_rel_ng_telemetry

Happened on:
https://chromium-swarm.appspot.com/bot?id=build404-m4&sort_stats=total%3Adesc

That bot's run pixel_tests successfully since, but there's also another failure:
https://chromium-swarm.appspot.com/task?id=41e6b7acc9201310&refresh=10&show_raw=1
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_chromium_rel_ng/213268
Pixel_OffscreenCanvasWebGLDefaultWorker failed
http://chromium-browser-gpu-tests.commondatastorage.googleapis.com/view_test_results.html?370bd1fbfe08db1498ba360cbc4650df90d8dbfe_mac_chromium_rel_ng_telemetry

Same failure mode, garbage pixels off to the side. Have taken this bot out of the Swarming pool too.

Found additional failures:

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_chromium_rel_ng/213262
Pixel_OffscreenCanvasTransferToImageBitmap failed on build433-m4:

https://chromium-swarm.appspot.com/task?id=41e6b7afefb43d10&refresh=10&show_raw=1
https://chromium-swarm.appspot.com/bot?id=build433-m4&sort_stats=total%3Adesc
http://chromium-browser-gpu-tests.commondatastorage.googleapis.com/view_test_results.html?b6acd4574e64fc9acb473d178527847178e318cb_mac_chromium_rel_ng_telemetry

Took out of the Swarming pool.


https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_chromium_rel_ng/213234
https://chromium-swarm.appspot.com/task?id=41e69cea9aecb810&refresh=10&show_raw=1
Pixel_OffscreenCanvasAccelerated2DWorker
Pixel_OffscreenCanvasTransferBeforeStyleResize
Pixel_OffscreenCanvasWebglResizeOnWorker

https://chromium-swarm.appspot.com/task?id=41e69cea9aecb810&refresh=10&show_raw=1
http://chromium-browser-gpu-tests.commondatastorage.googleapis.com/view_test_results.html?ffa2a7a745891c34d2cab726430d20002ce59b51_mac_chromium_rel_ng_telemetry

Was build439-m4 again.


So far, have taken these bots out of the Swarming pool:

https://chromium-swarm.appspot.com/bot?id=build397-m4&sort_stats=total%3Adesc
https://chromium-swarm.appspot.com/bot?id=build404-m4&sort_stats=total%3Adesc
https://chromium-swarm.appspot.com/bot?id=build433-m4&sort_stats=total%3Adesc
https://chromium-swarm.appspot.com/bot?id=build439-m4&sort_stats=total%3Adesc

Will see whether things stabilize or if it looks like there are bugs in Chrome that are causing these sorts of failures.

Blocking: 917079
Blockedon: 917264
Blockedon: 920003
Cc: erikc...@chromium.org
 Issue 919737  has been merged into this issue.

Sign in to add a comment