New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 742470 link

Starred by 4 users

Issue metadata

Status: Duplicate
Merged: issue 742472
Owner: ----
Closed: Jul 2017
Cc:
EstimatedDays: ----
NextAction: 2017-08-15
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

17.9%-64.3% regression in system_health.common_desktop at 485639:485931

Project Member Reported by pmeenan@chromium.org, Jul 13 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Jul 13 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=742470

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=f4ce029d5a4d2a881efd6cf147193be6da857e18da84ed2d1e0a1cebe8efa97a


Bot(s) for this bug's original alert(s):

chromium-rel-mac11-pro
chromium-rel-win7-gpu-ati
chromium-rel-win7-gpu-nvidia
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Jul 13 2017

Mergedinto: 742472
Status: Duplicate (was: Untriaged)

=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Sidney San MartĂ­n
  Commit : a20a025fb10592190b5b3bc90fd627c45c76676a
  Date   : Tue Jul 11 18:39:06 2017
  Subject: Enable ShouldUseFullSizeContentView().

Bisect Details
  Configuration: mac_pro_perf_bisect
  Benchmark    : system_health.common_desktop
  Metric       : load:energy_sum/load_media/load_media_google_images

Revision             Result                   N
chromium@485638      15.5319 +- 4.56904       14      good
chromium@485670      15.5635 +- 1.43891       6       good
chromium@485678      15.6281 +- 0.865305      6       good
chromium@485680      15.6605 +- 1.15613       6       good
chromium@485681      15.9829 +- 0.937429      6       good
chromium@485682      17.5785 +- 0.994813      6       bad       <--
chromium@485686      17.3421 +- 1.99871       9       bad
chromium@485702      17.3745 +- 0.823184      6       bad
chromium@485765      17.0519 +- 1.92437       9       bad
chromium@485891      16.674 +- 2.64005        14      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.media.google.images system_health.common_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8974158391573235872


For feedback, file a bug with component Speed>Bisection

Comment 4 by sdy@chromium.org, Jul 15 2017

I'm trying to recreate this locally, but when I run the benchmark I don't get a metric like this. Any thoughts?
Cc: charliea@chromium.org
These metrics are power usage coming from a BattOr device, so if you don't have a BattOr locally you won't see the metrics. +charliea, do you have documentation on the benchmark and running locally? (You can also run on the perf try bots: https://chromium.googlesource.com/chromium/src/+/master/docs/speed/perf_trybots.md

In the short term, it might be easiest to look at traces from the telemetry runs on the bisect bot:

Before your CL:
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-45-05-14647.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-46-01-13421.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-46-58-59640.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-47-54-46321.html


After your CL:
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-30-19-11604.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-31-15-65184.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-32-13-35002.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-35-02-2206.html

The metric that regressed is measuring total energy during load on a cached version of google images. At the top of the trace you can see rail stages (startup, idle successful load...) then under that you can see a trace of power ("interactions power") from the BattOr. The energy usage is higher during successful load after your CL. I attached a quick screenshot showing how to see the power consumption in a certain time period.
power-trace.png
491 KB View Download
Ah! Sorry. Not sure how I missed this.

Unfortunately, you need one of these BattOr power monitors on-site in order to run the test locally and reproduce the results. As Annie said, probably the easiest way is to use the perf try bot if you want to collect this metric on a particular branch that you're testing.

Given all of the other loading, responsiveness, and CPU-related alerts that occurred with the power regression, I think that the problem is likely that more work is happening at startup, which results in the load taking longer, which results in more total energy being used by the load. My thought is that you should roll the work back, focus on fixing  bug 742472 , and launch a perf try job before resubmitting this work to ensure that the power problem is fixed alongside the other problems.

Once again, sorry for the slow response.

Comment 7 by sdy@chromium.org, Jul 25 2017

Thanks for all of the information! I have a BattOr, but it's not compatible with my USB-C MBP. Aaron is walking me through retrofitting it :-). As I mentioned on the other bug, I'm working on turning this code path back off until I have more time to investigate.

Comment 8 by sdy@chromium.org, Aug 10 2017

I believe I landed a change yesterday (r492972) that should bring this back to where it was. What's the right way to monitor/confirm that? (Progress on my USB-C BattOr continues…)
Since this bug was duped by the bisect bot, it's a little extra work to check:

1) Go to the alerts from the duped bug:
https://chromeperf.appspot.com/group_report?bug_id=742472

2) Find the one that the bisect targeted: system_health.common_desktop, chromium-rel-mac11-pro, load_media_google_images. So you don't have to do all the clicking, here is the page I get when I click the graph icon in the table, and use the slider on the green chart at the bottom to stretch the time out to the present:
https://chromeperf.appspot.com/report?sid=42ba2e351e462829d88f60ddd8e7c7eaf158f93cfbf225be91dfe52ff7e75b39&start_rev=481654&end_rev=493276

3) It looks like there was a big improvement in the range r486806 - r486934, but not at r492972.
I reached out to sdy@ offline to decrease the latency of this conversation, but haven't heard back quite yet. I want to make sure I understand whether an improvement was expected in r492972 (which would be bad, because we didn't see one) or whether no regression was expected (which would be good, because we didn't see one).
NextAction: 2017-08-15
Sidney, the ToT build of Google images to go from ~14.5W at the start of the #c8 time series to ~18.5W at the end. However, in the same time range, the reference build (a fixed recent release of Chrome) went from using ~8.5W to ~10W, which suggests that something in the environment changed to make everything take more power (e.g. the test lab heated up, or something like that). In my experience, environmental power regressions tend to be a multiplier of the existing power (e.g. a heat-up in the lab might cause everything to take 1.1x more power than it used to). I have no intuition for _why_ this is given the things that I know about power, but I have noticed it.

Given that both the ToT and ref build seemed to go up by ~15%, my guess is that your changes have been successful in rolling back the regression. This seems consistent with your belief that the change shouldn't be doing anything on Mac 10.11 anyhow. If you notice that https://chromium-review.googlesource.com/c/613522 is a no-op on Mac Pro 10.11, please reach out to let us know that there's something funky going on. In the meanwhile, I'm going to leave this closed as a dupe with the belief that the difference that we're still seeing in the time series from the beginning to the end is just noise.
The NextAction date has arrived: 2017-08-15

Comment 14 by sdy@chromium.org, Aug 18 2017

charliea@: Thanks for the analysis! That all sounds reasonable to me.

Comment 15 by sdy@chromium.org, Aug 18 2017

Cc: sdy@chromium.org
 Issue 755224  has been merged into this issue.

Sign in to add a comment