Android perf bots failing: browser is crashing |
|||||||
Issue descriptionNexus 7v2 Perf is failing a lot of test with errors like device unreachable / no battor found / browser timeout. Bot: https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus7v2%20Perf Phone detected with no corresponding BattOr error: https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FAndroid_Nexus7v2_Perf%2F440%2F%2B%2Frecipes%2Fsteps%2Fsmoothness.sync_scroll.key_mobile_sites_smooth_on_Android%2F0%2Fstdout Device unreachable error: https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FAndroid_Nexus7v2_Perf%2F426%2F%2B%2Frecipes%2Fsteps%2Fsmoothness.sync_scroll.key_mobile_sites_smooth.reference_on_Android%2F0%2Fstdout
,
Aug 31 2017
Battor issue seems to be a red herring. The real problem is browser crashing on all tests on all Android devices.
,
Aug 31 2017
,
Aug 31 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8969747570595350304
,
Aug 31 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8969747355099992928
,
Aug 31 2017
,
Aug 31 2017
=== BISECT JOB RESULTS === Test failure found but unable to continue Bisect was stopped because a commit couldn't be classified as either good or bad. Bisect Details Configuration: android_nexus6_perf_bisect Benchmark : thread_times.key_idle_power_cases Metric : benchmark_duration/benchmark_duration To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_idle_power_cases More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8969747355099992928 For feedback, file a bug with component Speed>Bisection
,
Aug 31 2017
,
Aug 31 2017
Looking at the first failed builds on all of: https://luci-milo.appspot.com/buildbot/chromium.perf/Android%20Nexus6%20Perf/461 https://luci-milo.appspot.com/buildbot/chromium.perf/Android%20Nexus5X%20Perf/473 https://luci-milo.appspot.com/buildbot/chromium.perf/Android%20One%20Perf/453 They have an overlapping CL range of: http://test-results.appspot.com/revision_range?start=498630&end=498697 (68 commits) Will try to run some local tests and try to figure out why the bisect did not work.
,
Aug 31 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8969738894756999520
,
Aug 31 2017
=== Auto-CCing suspected CL author ranj@chromium.org === Hi ranj@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Test failure found with culprit Suspected Commit Author : Ran Ji Commit : 59355349151305f6eff50da743f3943576c2a5fe Date : Wed Aug 30 22:47:34 2017 Subject: Increment stability protos in ProcessMinidump Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:all_processes:process_count_avg/load_chrome/load_chrome_blank Revision Exit Code N chromium@498629 0 +- N/A 3 good chromium@498646 0 +- N/A 3 good chromium@498649 0 +- N/A 3 good chromium@498650 0 +- N/A 3 good chromium@498651 1 +- N/A 3 bad <-- chromium@498655 1 +- N/A 3 bad chromium@498663 1 +- N/A 3 bad chromium@498697 1 +- N/A 3 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.chrome.blank system_health.memory_mobile More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8969738894756999520 For feedback, file a bug with component Speed>Bisection
,
Aug 31 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/fe77736261ff9d61c9c7c4ff40c6276c242207d8 commit fe77736261ff9d61c9c7c4ff40c6276c242207d8 Author: Ned Nguyen <nednguyen@google.com> Date: Thu Aug 31 14:40:27 2017 Revert "Increment stability protos in ProcessMinidump" This reverts commit 59355349151305f6eff50da743f3943576c2a5fe. Reason for revert: breaking all performance tests on Android BUG= 760919 Original change's description: > Increment stability protos in ProcessMinidump > > Bug: 726513 > Change-Id: I2d40571b324103f766633f036e30c68508c5cf49 > Reviewed-on: https://chromium-review.googlesource.com/568101 > Commit-Queue: Ran Ji <ranj@chromium.org> > Reviewed-by: Reilly Grant <reillyg@chromium.org> > Reviewed-by: Yaron Friedman <yfriedman@chromium.org> > Reviewed-by: Alexei Svitkine (very slow) <asvitkine@chromium.org> > Reviewed-by: Lei Zhang <thestig@chromium.org> > Reviewed-by: Richard Coles <torne@chromium.org> > Reviewed-by: Mike West <mkwst@chromium.org> > Reviewed-by: Luke Halliwell <halliwell@chromium.org> > Reviewed-by: Peter Wen <wnwen@chromium.org> > Cr-Commit-Position: refs/heads/master@{#498651} TBR=yfriedman@chromium.org,thestig@chromium.org,wnwen@chromium.org,asvitkine@chromium.org,torne@chromium.org,reillyg@chromium.org,ranj@chromium.org,slan@chromium.org,halliwell@chromium.org,mkwst@chromium.org Change-Id: I4aefe331b5d7fb83f98a2a2f37b558e43f238e83 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: 726513 Reviewed-on: https://chromium-review.googlesource.com/646186 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#498860} [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/chrome/android/java/src/org/chromium/chrome/browser/metrics/UmaSessionStats.java [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/chrome/android/java/src/org/chromium/chrome/browser/tab/TabWebContentsObserver.java [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/chrome/browser/android/metrics/uma_session_stats.cc [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/chrome/browser/chrome_browser_main_android.cc [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/chrome/browser/metrics/chrome_stability_metrics_provider_unittest.cc [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/chromecast/browser/cast_browser_main_parts.cc [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/components/crash/content/browser/child_process_crash_observer_android.cc [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/components/crash/content/browser/child_process_crash_observer_android.h [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/components/crash/content/browser/crash_dump_manager_android.cc [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/components/crash/content/browser/crash_dump_manager_android.h [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/components/metrics/BUILD.gn [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/components/metrics/DEPS [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/components/metrics/stability_metrics_helper.cc [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/components/metrics/stability_metrics_helper.h [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/components/metrics/stability_metrics_helper_unittest.cc [modify] https://crrev.com/fe77736261ff9d61c9c7c4ff40c6276c242207d8/content/shell/browser/shell_browser_main_parts.cc
,
Aug 31 2017
Started revert here: https://chromium-review.googlesource.com/c/chromium/src/+/646206
,
Aug 31 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/980ea92e182130f2c1ddb98362dd4b10169ef775 commit 980ea92e182130f2c1ddb98362dd4b10169ef775 Author: Juan Antonio Navarro Pérez <perezju@chromium.org> Date: Thu Aug 31 14:41:40 2017 Revert "Increment stability protos in ProcessMinidump" This reverts commit 59355349151305f6eff50da743f3943576c2a5fe. Reason for revert: It's causing Chrome to crash on most Android tests. Original change's description: > Increment stability protos in ProcessMinidump > > Bug: 726513 > Change-Id: I2d40571b324103f766633f036e30c68508c5cf49 > Reviewed-on: https://chromium-review.googlesource.com/568101 > Commit-Queue: Ran Ji <ranj@chromium.org> > Reviewed-by: Reilly Grant <reillyg@chromium.org> > Reviewed-by: Yaron Friedman <yfriedman@chromium.org> > Reviewed-by: Alexei Svitkine (very slow) <asvitkine@chromium.org> > Reviewed-by: Lei Zhang <thestig@chromium.org> > Reviewed-by: Richard Coles <torne@chromium.org> > Reviewed-by: Mike West <mkwst@chromium.org> > Reviewed-by: Luke Halliwell <halliwell@chromium.org> > Reviewed-by: Peter Wen <wnwen@chromium.org> > Cr-Commit-Position: refs/heads/master@{#498651} TBR=yfriedman@chromium.org,thestig@chromium.org,wnwen@chromium.org,asvitkine@chromium.org,torne@chromium.org,reillyg@chromium.org,ranj@chromium.org,slan@chromium.org,halliwell@chromium.org,mkwst@chromium.org Change-Id: I888721a6c27c84d502c9dfb425da276e2bb9684c No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: 760919 ,726513 Reviewed-on: https://chromium-review.googlesource.com/646206 Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org> Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org> Cr-Commit-Position: refs/heads/master@{#498862}
,
Sep 1 2017
Issue 760950 has been merged into this issue.
,
Sep 1 2017
=== BISECT JOB RESULTS === Bisect was unable to run to completion Error: INFRA_FAILURE The bisect was able to narrow the range, you can try running with: good_revision: cb56ecbe82b5f6389fd9292e023f15d5f8cfdf05 bad_revision : e31b0b47faf7f774b7ad91d33b828218cc375c17 If failures persist contact the team (see below) and report the error. Bisect Details Configuration: android_nexus7_perf_bisect Benchmark : thread_times.key_silk_cases Metric : benchmark_duration/benchmark_duration Revision Exit Code N chromium@497690 0 +- N/A 20 good chromium@497830 0 +- N/A 20 good chromium@497865 0 +- N/A 20 good chromium@497900 1 +- N/A 20 bad chromium@497970 1 +- N/A 20 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_silk_cases More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8969747570595350304 For feedback, file a bug with component Speed>Bisection
,
Sep 1 2017
Issue no longer happening after the revert. |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by altimin@chromium.org
, Aug 31 2017