New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 601266 link

Starred by 0 users

Issue metadata

Status: Duplicate
Merged: issue 550016
Owner:
OOO until 2019-01-24
Closed: Apr 2016
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

"maps_pixel_test (with patch)" is flaky

Project Member Reported by chromium...@appspot.gserviceaccount.com, Apr 7 2016

Issue description

"maps_pixel_test (with patch)" is flaky.

This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label.

We have detected 3 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyJwsSBUZsYWtlIhxtYXBzX3BpeGVsX3Rlc3QgKHdpdGggcGF0Y2gpDA.

Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs

This flaky test/step was previously tracked in  issue 599775 .
 
Project Member

Comment 1 by chromium...@appspot.gserviceaccount.com, Apr 8 2016

Detected 4 new flakes for test/step "maps_pixel_test (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyJwsSBUZsYWtlIhxtYXBzX3BpeGVsX3Rlc3QgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
I'm unable to make any useful conclusion out of the failures

The two recent Linux failures are similar though:
https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_rel_ng/builds/209354/steps/maps_pixel_test%20on%20NVIDIA%20GPU%20on%20Linux%20%28with%20patch%29%20on%20Linux/logs/stdio
https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_rel_ng/builds/209986/steps/maps_pixel_test%20on%20NVIDIA%20GPU%20on%20Linux%20%28with%20patch%29%20on%20Linux/logs/stdio

(INFO) 2016-04-10 14:16:05,552 shared_page_state._StartBrowser:217    use_virtualized_gl_contexts
[4981:5003:0410/141615:ERROR:cert_verify_proc_nss.cc(984)] CERT_PKIXVerifyCert for ssl.gstatic.com failed err=-8172
Traceback (most recent call last):
  File "/tmp/run8wBPUl/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 84, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "/tmp/run8wBPUl/content/test/gpu/gpu_tests/gpu_test_base.py", line 111, in RunStory
    RunStoryWithRetries(GpuSharedPageState, self, results)
  File "/tmp/run8wBPUl/content/test/gpu/gpu_tests/gpu_test_base.py", line 72, in RunStoryWithRetries
    super(cls, shared_page_state).RunStory(results)
  File "/tmp/run8wBPUl/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 338, in RunStory
    self._current_page, self._current_tab, results)
  File "/tmp/run8wBPUl/content/test/gpu/gpu_tests/maps.py", line 48, in ValidateAndMeasurePage
    page.display_name, screenshot, expected, dpr)
  File "/tmp/run8wBPUl/content/test/gpu/gpu_tests/cloud_storage_test_base.py", line 222, in _ValidateScreenshotSamples
    _CompareScreenshotSamples(screenshot, expectations, device_pixel_ratio)
  File "/tmp/run8wBPUl/content/test/gpu/gpu_tests/cloud_storage_test_base.py", line 55, in _CompareScreenshotSamples
    str(actual_color.b) + "]")
Failure: Expected pixel at [540, 45] to be [188, 223, 255] but got [197, 228, 255]

[  FAILED  ] Maps.maps_002 (24157 ms)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ]  Maps.maps_002

1 FAILED TEST


Since the earlier failures on other platforms are different than this, I suggest waiting and gather more data before disabling the test and/or taking other action.
Project Member

Comment 3 by chromium...@appspot.gserviceaccount.com, Apr 12 2016

Detected 3 new flakes for test/step "maps_pixel_test (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyJwsSBUZsYWtlIhxtYXBzX3BpeGVsX3Rlc3QgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
*** BROKEN  run_nacl_sync_cond_test
SKIPPING test  text_overlaps_rodata
SKIPPING test  text_overlaps_data
*** BROKEN  run_trusted_performance_test
*** BROKEN  run_ncval_annotate_test
*** BROKEN  run_breakpad_browser_process_crash_test
*** BROKEN  run_gdb_invalid_memory_test
*** BROKEN  run_nanosleep_test
*** BROKEN  run_performance_test
*** BROKEN  run_timefuncs_test
*** BROKEN  run_raw_timefuncs_test
*** BROKEN  run_main_thread_pthread_exit_test2
*** BROKEN  run_overflowinline_test
*** BROKEN  run_stack_frame_noopt_frame_test
*** BROKEN  run_stack_frame_noopt_noframe_test
*** BROKEN  run_stack_frame_opt_frame_test
*** BROKEN  run_stack_frame_opt_noframe_test
*** BROKEN  run_vla_test
*** BROKEN  run_gdb_invalid_memory_test_irt
*** BROKEN  run_inbrowser_test_runner_parallel
*** BROKEN  run_nanosleep_test_irt
*** BROKEN  run_performance_test_irt
*** BROKEN  run_timefuncs_test_irt
*** BROKEN  run_raw_timefuncs_test_irt
*** BROKEN  run_main_thread_pthread_exit_test2_irt
*** BROKEN  run_overflowinline_test_irt
*** BROKEN  run_stack_frame_noopt_frame_test_irt
*** BROKEN  run_stack_frame_noopt_noframe_test_irt
*** BROKEN  run_stack_frame_opt_frame_test_irt
*** BROKEN  run_stack_frame_opt_noframe_test_irt
*** BROKEN  run_vla_test_irt
There are 29 broken tests.
[ RUN      ] AudioRecorderTest.RecordingEndToEnd
[23228:1287:0411/190025:5567401457057:FATAL:audio_manager_mac.cc(221)] Check failed: AudioManager::Get()->GetTaskRunner()->BelongsToCurrentThread(). 
0   components_unittests                0x000000010372c2f3 _ZN4base5debug10StackTraceC1Ev + 19
1   components_unittests                0x0000000103746ec9 _ZN7logging10LogMessageD2Ev + 73
2   components_unittests                0x0000000105583c46 _ZN5mediaL21GetAudioDeviceIdByUIdEbRKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE + 118
3   components_unittests                0x0000000105583af2 _ZN5media15AudioManagerMac24GetInputStreamParametersERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE + 34
4   components_unittests                0x000000010348994b _ZN11audio_modem17AudioRecorderTest14CreateRecorderEm + 155
5   components_unittests                0x0000000103489883 _ZN11audio_modem40AudioRecorderTest_RecordingEndToEnd_Test8TestBodyEv + 19
6   components_unittests                0x000000010380a4aa _ZN7testing4Test3RunEv + 474
7   components_unittests                0x000000010380af7a _ZN7testing8TestInfo3RunEv + 410
8   components_unittests                0x000000010380b543 _ZN7testing8TestCase3RunEv + 451
9   components_unittests                0x0000000103811b59 _ZN7testing8internal12UnitTestImpl11RunAllTestsEv + 841
10  components_unittests                0x00000001038117dd _ZN7testing8UnitTest3RunEv + 269
11  components_unittests                0x00000001037d03a6 _ZN4base9TestSuite3RunEv + 150
12  components_unittests                0x00000001037c9c49 _ZN4base15LaunchUnitTestsEiPPcRKNS_8CallbackIFivELNS_8internal8CopyModeE1EEE + 137
13  components_unittests                0x00000001023a01ff main + 191
14  components_unittests                0x00000001023a0134 start + 52
[4846/5818] AudioRecorderTest.RecordingEndToEnd (CRASHED)


[ RUN      ] BodyStreamBufferTest.LoadBodyStreamBufferAsArrayBuffer
../../third_party/WebKit/Source/modules/fetch/BodyStreamBufferTest.cpp:68: Failure
Actual function call count doesn't match EXPECT_CALL(*client, didFetchDataLoadedArrayBufferMock(_))...
         Expected: to be called once
           Actual: never called - unsatisfied and active
unknown file: Failure

Unexpected mock function call - returning directly.
    Function call: Call(2)
Google Mock tried the following 2 expectations, but none matched:

../../third_party/WebKit/Source/modules/fetch/BodyStreamBufferTest.cpp:67: tried expectation #0: EXPECT_CALL(checkpoint, Call(1))...
  Expected arg #0: is equal to 1
           Actual: 2
         Expected: to be called once
           Actual: called once - saturated and active
../../third_party/WebKit/Source/modules/fetch/BodyStreamBufferTest.cpp:69: tried expectation #1: EXPECT_CALL(checkpoint, Call(2))...
         Expected: all pre-requisites are satisfied
           Actual: the following immediate pre-requisites are not satisfied:
../../third_party/WebKit/Source/modules/fetch/BodyStreamBufferTest.cpp:68: pre-requisite #0
                   (end of pre-requisites)
         Expected: to be called once
           Actual: never called - unsatisfied and active
Received signal 11 <unknown> 000000000000
Ignore post above. Got distracted by some other flaky tests.

Not sure about the cause of maps_pixel_tests being flaky. There isnt much of an error message:
step returned non-zero exit code: 1
@@@STEP_LOG_LINE@outdir_json@{@@@
@@@STEP_LINK@logdog-->outdir_json@https://luci-logdog.appspot.com/v/?s=bb%2Ftryserver.chromium.mac%2Fmac_chromium_rel_ng%2F209411%2F%2B%2Frecipes%2Fsteps%2Fmaps_pixel_test_on_NVIDIA_GPU_on_Mac_Retina__with_patch__on_Mac%2F0%2Flogs%2Foutdir_json%2F0@@@
@@@STEP_LOG_LINE@outdir_json@  "summary.json": "{\n  \"shards\": [\n    {\n      \"abandoned_ts\": \"2016-04-12T02:58:12.854250\", \n      \"created_ts\": \"2016-04-12T01:57:28.707740\", \n      \"etag\": \"\\\"Rysv0ZKKmjB3G6rZii1842pTFsk/Snq8t6fn7cCdRI1n0t_3z5D-v_4\\\"\", \n      \"failure\": false, \n      \"internal_failure\": false, \n      \"kind\": \"swarming#taskItem\", \n      \"modified_ts\": \"2016-04-12T02:58:12.854250\", \n      \"name\": \"maps_pixel_test on NVIDIA GPU on Mac Retina (with patch)/Mac/3148cb3610/mac_chromium_rel_ng/209411\", \n      \"output\": null, \n      \"state\": \"EXPIRED\", \n      \"tags\": [\n        \"buildername:mac_chromium_rel_ng\", \n        \"buildnumber:209411\", \n        \"cpu:x86-64\", \n        \"data:3148cb3610b99997150b2c9ed530d3b71659cdf1\", \n        \"gpu:10de:0fe9\", \n        \"hidpi:1\", \n        \"master:tryserver.chromium.mac\", \n        \"name:maps_pixel_test\", \n        \"os:Mac\", \n        \"patch_project:chromium\", \n        \"pool:Chrome\", \n        \"priority:30\", \n        \"project:chromium\", \n        \"purpose:CQ\", \n        \"purpose:pre-commit\", \n        \"rietveld:https://codereview.chromium.org/1877153002/#ps1\", \n        \"slavename:vm701-m4\", \n        \"stepname:maps_pixel_test on NVIDIA GPU on Mac Retina (with patch) on Mac\", \n        \"user:haraken@chromium.org\"\n      ], \n      \"task_id\": \"2e21019a05d05510\", \n      \"user\": \"haraken@chromium.org\"\n    }\n  ]\n}"@@@
@@@STEP_LOG_LINE@outdir_json@}@@@
@@@STEP_LOG_END@outdir_json@@@
@@@STEP_LOG_LINE@no_results_exc@'0/output.json'@@@
@@@STEP_LINK@logdog-->no_results_exc@https://luci-logdog.appspot.com/v/?s=bb%2Ftryserver.chromium.mac%2Fmac_chromium_rel_ng%2F209411%2F%2B%2Frecipes%2Fsteps%2Fmaps_pixel_test_on_NVIDIA_GPU_on_Mac_Retina__with_patch__on_Mac%2F0%2Flogs%2Fno_results_exc%2F0@@@
@@@STEP_LOG_END@no_results_exc@@@
@@@STEP_LOG_LINE@invalid_results_exc@'failures'@@@
@@@STEP_LINK@logdog-->invalid_results_exc@https://luci-logdog.appspot.com/v/?s=bb%2Ftryserver.chromium.mac%2Fmac_chromium_rel_ng%2F209411%2F%2B%2Frecipes%2Fsteps%2Fmaps_pixel_test_on_NVIDIA_GPU_on_Mac_Retina__with_patch__on_Mac%2F0%2Flogs%2Finvalid_results_exc%2F0@@@
@@@STEP_LOG_END@invalid_results_exc@@@
@@@STEP_FAILURE@@@
Owner: kbr@chromium.org
Status: Assigned (was: Untriaged)
Seems similar to 550016. kbr@ Does this seem reasonable? Give it back to me if you disagree.
Labels: -Sheriff-Chromium

Comment 9 by kbr@chromium.org, Apr 13 2016

Mergedinto: 550016
Status: Duplicate (was: Assigned)
The mass failures on mac_chromium_rel_ng were  Issue 550016 , as rjkroeke@ surmised. This outage has been resolved by disabling many tests on this configuration, as the machines were oversubscribed.

The two Linux failures that kjellander@ pointed out in #2 above:
https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_rel_ng/builds/209354/steps/maps_pixel_test%20on%20NVIDIA%20GPU%20on%20Linux%20%28with%20patch%29%20on%20Linux/logs/stdio
https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_rel_ng/builds/209986/steps/maps_pixel_test%20on%20NVIDIA%20GPU%20on%20Linux%20%28with%20patch%29%20on%20Linux/logs/stdio

are something different. The test is running, but actually failing. Not clear why that is or was happening -- most probable reason would be a rendering or JavaScript change. Let's see if that continues to occur.

Duplicating this into  Issue 550016  since that was the reason the majority of these failures were happening.

Sign in to add a comment