Issue metadata
Sign in to add a comment
|
17.9%-64.3% regression in system_health.common_desktop at 485639:485931 |
||||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Jul 13 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8974158391573235872
,
Jul 13 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Sidney San MartĂn Commit : a20a025fb10592190b5b3bc90fd627c45c76676a Date : Tue Jul 11 18:39:06 2017 Subject: Enable ShouldUseFullSizeContentView(). Bisect Details Configuration: mac_pro_perf_bisect Benchmark : system_health.common_desktop Metric : load:energy_sum/load_media/load_media_google_images Revision Result N chromium@485638 15.5319 +- 4.56904 14 good chromium@485670 15.5635 +- 1.43891 6 good chromium@485678 15.6281 +- 0.865305 6 good chromium@485680 15.6605 +- 1.15613 6 good chromium@485681 15.9829 +- 0.937429 6 good chromium@485682 17.5785 +- 0.994813 6 bad <-- chromium@485686 17.3421 +- 1.99871 9 bad chromium@485702 17.3745 +- 0.823184 6 bad chromium@485765 17.0519 +- 1.92437 9 bad chromium@485891 16.674 +- 2.64005 14 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.media.google.images system_health.common_desktop More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8974158391573235872 For feedback, file a bug with component Speed>Bisection
,
Jul 15 2017
I'm trying to recreate this locally, but when I run the benchmark I don't get a metric like this. Any thoughts?
,
Jul 16 2017
These metrics are power usage coming from a BattOr device, so if you don't have a BattOr locally you won't see the metrics. +charliea, do you have documentation on the benchmark and running locally? (You can also run on the perf try bots: https://chromium.googlesource.com/chromium/src/+/master/docs/speed/perf_trybots.md In the short term, it might be easiest to look at traces from the telemetry runs on the bisect bot: Before your CL: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-45-05-14647.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-46-01-13421.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-46-58-59640.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-47-54-46321.html After your CL: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-30-19-11604.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-31-15-65184.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-32-13-35002.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-07-13_13-35-02-2206.html The metric that regressed is measuring total energy during load on a cached version of google images. At the top of the trace you can see rail stages (startup, idle successful load...) then under that you can see a trace of power ("interactions power") from the BattOr. The energy usage is higher during successful load after your CL. I attached a quick screenshot showing how to see the power consumption in a certain time period.
,
Jul 24 2017
Ah! Sorry. Not sure how I missed this. Unfortunately, you need one of these BattOr power monitors on-site in order to run the test locally and reproduce the results. As Annie said, probably the easiest way is to use the perf try bot if you want to collect this metric on a particular branch that you're testing. Given all of the other loading, responsiveness, and CPU-related alerts that occurred with the power regression, I think that the problem is likely that more work is happening at startup, which results in the load taking longer, which results in more total energy being used by the load. My thought is that you should roll the work back, focus on fixing bug 742472 , and launch a perf try job before resubmitting this work to ensure that the power problem is fixed alongside the other problems. Once again, sorry for the slow response.
,
Jul 25 2017
Thanks for all of the information! I have a BattOr, but it's not compatible with my USB-C MBP. Aaron is walking me through retrofitting it :-). As I mentioned on the other bug, I'm working on turning this code path back off until I have more time to investigate.
,
Aug 10 2017
I believe I landed a change yesterday (r492972) that should bring this back to where it was. What's the right way to monitor/confirm that? (Progress on my USB-C BattOr continues…)
,
Aug 10 2017
Since this bug was duped by the bisect bot, it's a little extra work to check: 1) Go to the alerts from the duped bug: https://chromeperf.appspot.com/group_report?bug_id=742472 2) Find the one that the bisect targeted: system_health.common_desktop, chromium-rel-mac11-pro, load_media_google_images. So you don't have to do all the clicking, here is the page I get when I click the graph icon in the table, and use the slider on the green chart at the bottom to stretch the time out to the present: https://chromeperf.appspot.com/report?sid=42ba2e351e462829d88f60ddd8e7c7eaf158f93cfbf225be91dfe52ff7e75b39&start_rev=481654&end_rev=493276 3) It looks like there was a big improvement in the range r486806 - r486934, but not at r492972.
,
Aug 14 2017
I reached out to sdy@ offline to decrease the latency of this conversation, but haven't heard back quite yet. I want to make sure I understand whether an improvement was expected in r492972 (which would be bad, because we didn't see one) or whether no regression was expected (which would be good, because we didn't see one).
,
Aug 14 2017
,
Aug 14 2017
Sidney, the ToT build of Google images to go from ~14.5W at the start of the #c8 time series to ~18.5W at the end. However, in the same time range, the reference build (a fixed recent release of Chrome) went from using ~8.5W to ~10W, which suggests that something in the environment changed to make everything take more power (e.g. the test lab heated up, or something like that). In my experience, environmental power regressions tend to be a multiplier of the existing power (e.g. a heat-up in the lab might cause everything to take 1.1x more power than it used to). I have no intuition for _why_ this is given the things that I know about power, but I have noticed it. Given that both the ToT and ref build seemed to go up by ~15%, my guess is that your changes have been successful in rolling back the regression. This seems consistent with your belief that the change shouldn't be doing anything on Mac 10.11 anyhow. If you notice that https://chromium-review.googlesource.com/c/613522 is a no-op on Mac Pro 10.11, please reach out to let us know that there's something funky going on. In the meanwhile, I'm going to leave this closed as a dupe with the belief that the difference that we're still seeing in the time series from the beginning to the end is just noise.
,
Aug 15 2017
The NextAction date has arrived: 2017-08-15
,
Aug 18 2017
charliea@: Thanks for the analysis! That all sounds reasonable to me.
,
Aug 18 2017
|
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by 42576172...@developer.gserviceaccount.com
, Jul 13 2017