New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 756025 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Aug 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

16.5% regression in system_health.common_mobile at 492228:492231

Project Member Reported by ellenpli@google.com, Aug 16 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Aug 16 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=756025

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=57b37308e277244bfc4ce82a27866020fd28580856c1e40905ef0a526b2b5e6b


Bot(s) for this bug's original alert(s):

android-webview-nexus5X
 Issue 756024  has been merged into this issue.
Project Member

Comment 4 by 42576172...@developer.gserviceaccount.com, Aug 16 2017


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: android_webview_arm64_aosp_perf_bisect
  Benchmark    : system_health.common_mobile
  Metric       : timeToFirstMeaningfulPaint_avg/load_tools/load_tools_weather


To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.tools.weather system_health.common_mobile

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8971089988639355440


For feedback, file a bug with component Speed>Bisection
Widening bisect range.
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Aug 16 2017


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: android_webview_arm64_aosp_perf_bisect
  Benchmark    : system_health.common_mobile
  Metric       : timeToFirstMeaningfulPaint_avg/load_tools/load_tools_weather


To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.tools.weather system_health.common_mobile

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8971084927474171504


For feedback, file a bug with component Speed>Bisection
Cc: charliea@chromium.org nedngu...@google.com
Owner: charliea@chromium.org
Charlie, Ned: cc-ing you as owners of system_health.common_mobile. It's failing to produce values in these bisects. Some example stdout:

https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Ftryserver.chromium.perf%2Fandroid_webview_arm64_aosp_perf_bisect%2F1612%2F%2B%2Frecipes%2Fsteps%2FGathering_reference_values%2F0%2Fsteps%2FPerformance_Test_2_of_6%2F0%2Flogs%2FFailure_Output%2F0
https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Ftryserver.chromium.perf%2Fandroid_webview_arm64_aosp_perf_bisect%2F1612%2F%2B%2Frecipes%2Fsteps%2FGathering_reference_values%2F0%2Fsteps%2FPerformance_Test_1_of_6%2F0%2Flogs%2FFailure_Output%2F0
(ERROR) 2017-08-16 09:32:17,191 battor_wrapper._FlashBattOr:175  Git hash returned from BattOr was not as expected: [0816/093217.191104:ERROR:serial_io_handler.cc(152)] Failed to open serial port: FILE_ERROR_ACCESS_DENIED
Traceback (most recent call last):
  File "/b/c/b/android_webview_arm64_aosp_perf_bisect/src/third_party/catapult/common/battor/battor/battor_wrapper.py", line 162, in _FlashBattOr
    device_git_hash = self.GetFirmwareGitHash()
  File "/b/c/b/android_webview_arm64_aosp_perf_bisect/src/third_party/catapult/common/battor/battor/battor_wrapper.py", line 392, in GetFirmwareGitHash
    int(self._git_hash, 16)


The device is serial  020232abc2931127 on build225-b4 if you need to contact labs about the BattOr.
Status: Assigned (was: Untriaged)
Filed a bug to restart as  bug 759008 , and a bug to track this failure as 759004.
I restarted the bisect, but I'm also going to throw in that, based on the graphs, I think this is a very likely case of the ref build regressing by slightly less than the non-ref build, resulting in a regression where one probably doesn't exist. We'll see what the bisect turns up though.
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Aug 29 2017


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : system_health.common_mobile
  Metric       : idle:power_avg/load_tools/load_tools_weather

Revision             Result                   N
chromium@492431      2.19645 +- 0.108324      4      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.tools.weather system_health.common_mobile

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8969906398606227296


For feedback, file a bug with component Speed>Bisection
Status: WontFix (was: Assigned)

Sign in to add a comment