merge step of performance_test_suite fails on Android device Nexus 5 |
|||
Issue description
Log:
Traceback (most recent call last):
File "/b/c/b/Android_Nexus5_Perf/src/tools/perf/process_perf_results.py", line 518, in <module>
sys.exit(main())
File "/b/c/b/Android_Nexus5_Perf/src/tools/perf/process_perf_results.py", line 514, in main
args.smoke_test_mode)
File "/b/c/b/Android_Nexus5_Perf/src/tools/perf/process_perf_results.py", line 280, in _process_perf_results
_merge_json_output(output_json, test_results_list, extra_links)
File "/b/c/b/Android_Nexus5_Perf/src/tools/perf/process_perf_results.py", line 109, in _merge_json_output
merged_results = results_merger.merge_test_results(jsons_to_merge)
File "/b/c/b/Android_Nexus5_Perf/src/tools/perf/core/results_merger.py", line 72, in merge_test_results
return _merge_json_test_result_format(shard_results_list)
File "/b/c/b/Android_Nexus5_Perf/src/tools/perf/core/results_merger.py", line 145, in _merge_json_test_result_format
merge('tests', merge_tries)
File "/b/c/b/Android_Nexus5_Perf/src/tools/perf/core/results_merger.py", line 141, in <lambda>
result_json, merged_results, key, merge_func)
File "/b/c/b/Android_Nexus5_Perf/src/tools/perf/core/results_merger.py", line 264, in merge_value
dest[key] = merge_func(source[key], dest[key])
File "/b/c/b/Android_Nexus5_Perf/src/tools/perf/core/results_merger.py", line 214, in merge_tries
prefix, k, v, curr_node, dest_node))
core.results_merger.MergeException: MergeFailure for tests
:loading.mobile:Locanto:actual: u'PASS' not mergable, curr_node: {u'actual': u'PASS', u'artifacts': {u'logs': [u'https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/cff486fc-7ef4-11e8-a8e6-0242ac110005', u'https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/d0c8b2f6-7ef4-11e8-a8e6-0242ac110005']}, u'is_unexpected': False, u'times': [30.875025033950806, 16.667243003845215], u'time': 30.875025033950806, u'expected': u'PASS'}
dest_node: {u'expected': u'PASS', u'is_unexpected': True, u'actual': u'SKIP', u'time': None, u'times': [None, None]}
https://logs.chromium.org/v/?s=chrome%2Fbb%2Fchromium.perf%2FAndroid_Nexus5_Perf%2F1946%2F%2B%2Frecipes%2Fsteps%2Fperformance_test_suite_on_Android_device_Nexus_5%2F0%2Flogs%2FMerge_script_log%2F0
,
Jul 4
I debug further, and found that this because of midway through the test, the mobile device in Shard 7 crash hard, which makes the rest of all stories shown up as "SKIP", including Locanto story. I will fix Telemetry shows that we only add skip on all filtered stories, not all stories in the story_set
,
Jul 4
,
Jul 4
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/5abd99f5f7685f0e3b18f11e2fcda667a3be83e3 commit 5abd99f5f7685f0e3b18f11e2fcda667a3be83e3 Author: Nghia Nguyen <nednguyen@google.com> Date: Wed Jul 04 01:34:26 2018 Make sure that Telemetry only generate skipped entries for filtered stories when test is interrupted Bug: chromium:860091 Change-Id: I8bb50cbc3ca68f78ae947600b8f19c221d6a9c60 TBR=eyaich@chromium.org Reviewed-on: https://chromium-review.googlesource.com/1125504 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Ned Nguyen <nednguyen@google.com> [modify] https://crrev.com/5abd99f5f7685f0e3b18f11e2fcda667a3be83e3/telemetry/telemetry/internal/story_runner.py
,
Jul 4
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4ec0498222df0d0c18f37faeaba24e18ad6f3bd5 commit 4ec0498222df0d0c18f37faeaba24e18ad6f3bd5 Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Wed Jul 04 04:30:49 2018 Roll src/third_party/catapult 22b68f285d2c..5abd99f5f768 (1 commits) https://chromium.googlesource.com/catapult.git/+log/22b68f285d2c..5abd99f5f768 git log 22b68f285d2c..5abd99f5f768 --date=short --no-merges --format='%ad %ae %s' 2018-07-04 nednguyen@google.com Make sure that Telemetry only generate skipped entries for filtered stories when test is interrupted Created with: gclient setdep -r src/third_party/catapult@5abd99f5f768 The AutoRoll server is located here: https://catapult-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:860091 TBR=sullivan@chromium.org Change-Id: Ic208e995fe193cd69d99a42f5f62db669770ddeb Reviewed-on: https://chromium-review.googlesource.com/1124704 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#572475} [modify] https://crrev.com/4ec0498222df0d0c18f37faeaba24e18ad6f3bd5/DEPS
,
Jul 5
Still need to add test for this
,
Jul 17
Test in https://chromium-review.googlesource.com/c/catapult/+/1140359 Landing..
,
Jul 17
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/7b8db1f1135cc8cdf9daf6a0f6f5020eb5c0dac3 commit 7b8db1f1135cc8cdf9daf6a0f6f5020eb5c0dac3 Author: nednguyen <nednguyen@google.com> Date: Tue Jul 17 15:48:19 2018 Add test that verifies the way Telemetry generate skipped entries for filtered stories when benchmark is interrupted Bug: chromium:860091 Change-Id: I62378dc172d148abc1b71c7a651a031181a09f3f TBR=eyaich@chromium.org, perezju@chromium.org Reviewed-on: https://chromium-review.googlesource.com/1140359 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Ned Nguyen <nednguyen@google.com> [modify] https://crrev.com/7b8db1f1135cc8cdf9daf6a0f6f5020eb5c0dac3/telemetry/telemetry/internal/story_runner_unittest.py
,
Jul 18
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ee1d7d6a10afc8d71253cb849ed508f706cf92a9 commit ee1d7d6a10afc8d71253cb849ed508f706cf92a9 Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Wed Jul 18 07:21:37 2018 Roll src/third_party/catapult f5342c4cf3d3..e8dc05ccba6b (9 commits) https://chromium.googlesource.com/catapult.git/+log/f5342c4cf3d3..e8dc05ccba6b git log f5342c4cf3d3..e8dc05ccba6b --date=short --no-merges --format='%ad %ae %s' 2018-07-18 nednguyen@google.com Revert "Handle stale directories when pushing files" 2018-07-18 nednguyen@google.com Implement screenshot capturing at system level on Linux 2018-07-17 dtu@chromium.org [pinpoint] Case-insensitive compare for Webview. 2018-07-17 simonhatch@chromium.org Dashboard - Whitelist documentation urls in /add_histograms 2018-07-17 cphlipot0@gmail.com [Systrace] Raise error when specified device is not found 2018-07-17 nednguyen@google.com Add test that verifies the way Telemetry generate skipped entries for filtered stories when benchmark is interrupted 2018-07-17 xiaochengh@chromium.org Disable WebPageReplayGoServerTest.testSmokeStartingWebPageReplayGoServer on Windows 2018-07-16 dtu@chromium.org [pinpoint] Update list of bots available in Pinpoint. 2018-07-16 bsheedy@chromium.org Handle stale directories when pushing files Created with: gclient setdep -r src/third_party/catapult@e8dc05ccba6b The AutoRoll server is located here: https://catapult-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:862845 , chromium:862996 , chromium:863861 , chromium:860091 ,chromium:864294, chromium:863995 , chromium:862845 TBR=sullivan@chromium.org Change-Id: I5cab2a67db2dfee8c71f9ed7e445b82a9df41aec Reviewed-on: https://chromium-review.googlesource.com/1141465 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#575965} [modify] https://crrev.com/ee1d7d6a10afc8d71253cb849ed508f706cf92a9/DEPS
,
Jul 18
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/a9c23a6a3dae3b8e212485ce41cf13de0a1bf614 commit a9c23a6a3dae3b8e212485ce41cf13de0a1bf614 Author: nednguyen <nednguyen@google.com> Date: Wed Jul 18 18:03:13 2018 Remove stray debug print in page_test_results.py Bug: chromium:860091 Change-Id: Icae2551547108db11f0aaf6ae4f3c659589bce45 TBR=perezju@chromium.org, eyaich@chromium.org Reviewed-on: https://chromium-review.googlesource.com/1142199 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Ned Nguyen <nednguyen@google.com> [modify] https://crrev.com/a9c23a6a3dae3b8e212485ce41cf13de0a1bf614/telemetry/telemetry/internal/results/page_test_results.py
,
Jul 18
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/bc007d403e15f4fdce226c0495a8735770feda49 commit bc007d403e15f4fdce226c0495a8735770feda49 Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Wed Jul 18 23:44:11 2018 Roll src/third_party/catapult 941ac6857d71..26d6aff1b57b (2 commits) https://chromium.googlesource.com/catapult.git/+log/941ac6857d71..26d6aff1b57b git log 941ac6857d71..26d6aff1b57b --date=short --no-merges --format='%ad %ae %s' 2018-07-18 dtu@chromium.org [pinpoint] Add `android-pixel2-perf` config. 2018-07-18 nednguyen@google.com Remove stray debug print in page_test_results.py Created with: gclient setdep -r src/third_party/catapult@26d6aff1b57b The AutoRoll server is located here: https://catapult-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:860091 TBR=sullivan@chromium.org Change-Id: I3c3c08e18bde2a25e9565965cea0a9e4bac8e734 Reviewed-on: https://chromium-review.googlesource.com/1142012 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#576258} [modify] https://crrev.com/bc007d403e15f4fdce226c0495a8735770feda49/DEPS |
|||
►
Sign in to add a comment |
|||
Comment 1 by nednguyen@chromium.org
, Jul 4