Mac GPU power tests. |
||||||
Issue descriptionAdd Telemetry tests that measure Mac GPU usage. The goal is to be able to detect 10% improvements and regressions to Mac compositing.
,
Jun 3 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/dd3bbf7f0f1cfac1ff91c49b19a1e8078d43ce7b commit dd3bbf7f0f1cfac1ff91c49b19a1e8078d43ce7b Author: erikchen <erikchen@chromium.org> Date: Fri Jun 03 00:41:01 2016 telemetry: Add scrolling power test. The test creates a page full of text and slowly scrolls it using requestAnimationFrame. BUG= 616811 Review-Url: https://codereview.chromium.org/1998433003 Cr-Commit-Position: refs/heads/master@{#397564} [modify] https://crrev.com/dd3bbf7f0f1cfac1ff91c49b19a1e8078d43ce7b/tools/perf/benchmarks/power.py [add] https://crrev.com/dd3bbf7f0f1cfac1ff91c49b19a1e8078d43ce7b/tools/perf/page_sets/trivial_sites.py [add] https://crrev.com/dd3bbf7f0f1cfac1ff91c49b19a1e8078d43ce7b/tools/perf/page_sets/trivial_sites/trivial_scrolling_page.html
,
Jun 3 2016
Initial numbers look pretty stable. There is a single spike on chromium-rel-mac11. I'm curious how the numbers will look after we get some more data. It looks like chromium-rel-mac-hdd isn't correctly reporting mWh. nednguyen@, can you investigate? https://chromeperf.appspot.com/report?sid=e135aad6f55383890eface86512e700155b1533d18359a7bd96e0a81625fbeb1
,
Jun 3 2016
chromium-rel-mac-hdd is using "mac yosemite" (https://build.chromium.org/p/chromium.perf/builders/Mac%20HDD%20Perf%20%284%29/builds/2506/steps/power.trivial_scroll/logs/stdio) The mac power metric work for maverick & beyond: https://cs.chromium.org/chromium/src/third_party/catapult/telemetry/telemetry/internal/platform/power_monitor/powermetrics_power_monitor.py?rcl=0&l=60 So I am not sure what's stopping it. You can add some logging to https://cs.chromium.org/chromium/src/third_party/catapult/telemetry/telemetry/internal/platform/power_monitor/powermetrics_power_monitor.py?rcl=0&l=44 to print out the output & tbr'ed me.
,
Jun 6 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3885ecfd5ff0a486f97839089eb739a39f66960b commit 3885ecfd5ff0a486f97839089eb739a39f66960b Author: erikchen <erikchen@chromium.org> Date: Mon Jun 06 22:28:59 2016 telemetry, mac: Add more simple scrolling power tests. These tests measure the no-gpu and disable-mac-overlays cases. BUG= 616811 CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:mac_retina_perf_cq Review-Url: https://codereview.chromium.org/2034123002 Cr-Commit-Position: refs/heads/master@{#398143} [modify] https://crrev.com/3885ecfd5ff0a486f97839089eb739a39f66960b/tools/perf/benchmarks/power.py [add] https://crrev.com/3885ecfd5ff0a486f97839089eb739a39f66960b/tools/perf/page_sets/mac_gpu_sites.py [delete] https://crrev.com/26219410e3069f9cff75c5c487d2fc2d52f8ef4d/tools/perf/page_sets/trivial_sites.py
,
Jun 17 2016
Ping. Should this be Pri-1? If so please assign a milestone.
,
Jun 20 2016
,
Jun 27 2016
There is a different in measured power consumption from MSRs, but the difference appears very small. This is at least partly due to the fact that CPU power and GPU power get lumped together. I'm going to add a GPU heavy test to see if the measured differences become larger. Note: Lumping the three configurations (no gpu, no overlays, regular) together into a single benchmark has the problem that the three results get averaged together by default. I'm worried that if there is a 10% regression to the regular path, it will only look like a 3% regression. nednguyen, can you comment?
,
Jun 27 2016
We also have monitoring per page if that's what you want.
,
Jun 28 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4eb467453200da5ced2cf5fcc7d0a3e234b28dae commit 4eb467453200da5ced2cf5fcc7d0a3e234b28dae Author: erikchen <erikchen@chromium.org> Date: Tue Jun 28 16:36:09 2016 Add a power test for blinking cursor. Also fix the browser args, which should have been "--disable-gpu", not "--no-gpu". BUG= 616811 CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq Review-Url: https://codereview.chromium.org/2100223002 Cr-Commit-Position: refs/heads/master@{#402487} [modify] https://crrev.com/4eb467453200da5ced2cf5fcc7d0a3e234b28dae/tools/perf/benchmarks/power.py [modify] https://crrev.com/4eb467453200da5ced2cf5fcc7d0a3e234b28dae/tools/perf/page_sets/mac_gpu_sites.py [add] https://crrev.com/4eb467453200da5ced2cf5fcc7d0a3e234b28dae/tools/perf/page_sets/trivial_sites/trivial_blinking_cursor.html
,
Jul 6 2016
The first two simple tests are up and running. The results are most pronounced on chromium-rel-mac-retina. One screenshot is for a slowly scrolling page with lots of text. The other is for a blinking cursor. In both cases, there are 3 distinct lines for mwh: 1. The lowest is for Core Animation compositor. 2. The second is for GL compositor 3. The highest is for no-GPU.
,
Jul 6 2016
The benchmark media.tough_video_cases already has a pretty exhaustive suite of tests for different types of video & audio content (which measures mwh), defaulting to the CoreAnimation compositor. Since we really only care about performance of the CoreAnimation compositor, this is sufficient for our needs. That being said, when we look at some test pages like crowd2160.mp4, crowd1080.ogv, or crowd1080.mp4, we see that there's been no change expect for a surge between 3/17 and 5/11 (there are actually no data points in between those two points, since tough_video_cases were disabled on mac: https://bugs.chromium.org/p/chromium/issues/detail?id=595665)
,
Jul 9 2016
sullivan: Could you point me to a reference to per page monitoring, as mentioned by nednguyen in c#9?
,
Jul 11 2016
The monitoring config is set up as a simple path with '*' wildcards. The dashboard has both "summary" metrics and "per-page" metrics. The "summary" metric will alert on the average of all the pages: ChromiumPerf/*/power.trivial_pages/energy_consumption_mwh The "per-page" metric will alert on each individual page: ChromiumPerf/*/power.trivial_pages/energy_consumtion_mwh/* We generally prefer "per-page" alerting because it's more accurate, and the dashboard should group alerts on the same test together, so you should get a single bug if 10 pages regress at the same time.
,
Jul 12 2016
Oh, perfect, that does exactly what I want. Thanks for the info, sullivan!
,
Jul 12 2016
Issue 533174 has been merged into this issue.
,
Jul 13 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/bd717794cd228563dafe102ca66cf8c614363438 commit bd717794cd228563dafe102ca66cf8c614363438 Author: erikchen <erikchen@chromium.org> Date: Wed Jul 13 03:25:22 2016 Add 2d Canvas power test. The test is exceedingly simple. It uses requestAnimationFrame and 2D Canvas APIs to animate the background color of the page. BUG= 616811 Review-Url: https://codereview.chromium.org/2131413003 Cr-Commit-Position: refs/heads/master@{#404957} [modify] https://crrev.com/bd717794cd228563dafe102ca66cf8c614363438/tools/perf/page_sets/mac_gpu_sites.py [add] https://crrev.com/bd717794cd228563dafe102ca66cf8c614363438/tools/perf/page_sets/trivial_sites/trivial_canvas.html
,
Jul 13 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ad659671899c8629433fc7904e109ab34f15eafd commit ad659671899c8629433fc7904e109ab34f15eafd Author: erikchen <erikchen@chromium.org> Date: Wed Jul 13 16:26:40 2016 mac: Add WebGL power test The test is exceedingly simple. It uses requestAnimationFrame and WebGL APIs to animate the background color of the page. BUG= 616811 Review-Url: https://codereview.chromium.org/2143523002 Cr-Commit-Position: refs/heads/master@{#405182} [modify] https://crrev.com/ad659671899c8629433fc7904e109ab34f15eafd/tools/perf/page_sets/mac_gpu_sites.py [add] https://crrev.com/ad659671899c8629433fc7904e109ab34f15eafd/tools/perf/page_sets/trivial_sites/trivial_webgl.html
,
Jul 13 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/bd717794cd228563dafe102ca66cf8c614363438 commit bd717794cd228563dafe102ca66cf8c614363438 Author: erikchen <erikchen@chromium.org> Date: Wed Jul 13 03:25:22 2016 Add 2d Canvas power test. The test is exceedingly simple. It uses requestAnimationFrame and 2D Canvas APIs to animate the background color of the page. BUG= 616811 Review-Url: https://codereview.chromium.org/2131413003 Cr-Commit-Position: refs/heads/master@{#404957} [modify] https://crrev.com/bd717794cd228563dafe102ca66cf8c614363438/tools/perf/page_sets/mac_gpu_sites.py [add] https://crrev.com/bd717794cd228563dafe102ca66cf8c614363438/tools/perf/page_sets/trivial_sites/trivial_canvas.html
,
Jul 13 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ad659671899c8629433fc7904e109ab34f15eafd commit ad659671899c8629433fc7904e109ab34f15eafd Author: erikchen <erikchen@chromium.org> Date: Wed Jul 13 16:26:40 2016 mac: Add WebGL power test The test is exceedingly simple. It uses requestAnimationFrame and WebGL APIs to animate the background color of the page. BUG= 616811 Review-Url: https://codereview.chromium.org/2143523002 Cr-Commit-Position: refs/heads/master@{#405182} [modify] https://crrev.com/ad659671899c8629433fc7904e109ab34f15eafd/tools/perf/page_sets/mac_gpu_sites.py [add] https://crrev.com/ad659671899c8629433fc7904e109ab34f15eafd/tools/perf/page_sets/trivial_sites/trivial_webgl.html
,
Jul 16 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/76bfe25ea868d0038ca850bf2839cd605a49fb26 commit 76bfe25ea868d0038ca850bf2839cd605a49fb26 Author: erikchen <erikchen@chromium.org> Date: Sat Jul 16 01:16:25 2016 Add power test for css filter effect animation. The test is exceedingly simple. It uses 2D Canvas to draw a gradient, then it uses a css filter effect animation to modulate the radius of a gaussian blur effect. BUG= 616811 , 581526 Review-Url: https://codereview.chromium.org/2153913003 Cr-Commit-Position: refs/heads/master@{#405915} [modify] https://crrev.com/76bfe25ea868d0038ca850bf2839cd605a49fb26/tools/perf/page_sets/mac_gpu_sites.py [add] https://crrev.com/76bfe25ea868d0038ca850bf2839cd605a49fb26/tools/perf/page_sets/trivial_sites/trivial_blur_animation.html
,
Aug 9 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8bc0053763a01b684ec821798e152fd93091ec7b commit 8bc0053763a01b684ec821798e152fd93091ec7b Author: erikchen <erikchen@chromium.org> Date: Tue Aug 09 12:14:21 2016 Add fullscreen video power test. BUG= 616811 Review-Url: https://codereview.chromium.org/2223123002 Cr-Commit-Position: refs/heads/master@{#410639} [modify] https://crrev.com/8bc0053763a01b684ec821798e152fd93091ec7b/tools/perf/page_sets/mac_gpu_sites.py [add] https://crrev.com/8bc0053763a01b684ec821798e152fd93091ec7b/tools/perf/page_sets/trivial_sites/road_trip_640_480.mp4.sha1 [add] https://crrev.com/8bc0053763a01b684ec821798e152fd93091ec7b/tools/perf/page_sets/trivial_sites/trivial_fullscreen_video.html
,
Aug 19 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/750b27bd5d7542ad84eb52adfacfba02616f0cfc commit 750b27bd5d7542ad84eb52adfacfba02616f0cfc Author: erikchen <erikchen@chromium.org> Date: Fri Aug 19 01:33:38 2016 mac: Add a GPU power test for gifs. BUG= 616811 Review-Url: https://codereview.chromium.org/2243403006 Cr-Commit-Position: refs/heads/master@{#413008} [modify] https://crrev.com/750b27bd5d7542ad84eb52adfacfba02616f0cfc/tools/perf/page_sets/mac_gpu_sites.py [add] https://crrev.com/750b27bd5d7542ad84eb52adfacfba02616f0cfc/tools/perf/page_sets/trivial_sites/bunny.gif.sha1 [add] https://crrev.com/750b27bd5d7542ad84eb52adfacfba02616f0cfc/tools/perf/page_sets/trivial_sites/trivial_gif.html
,
Aug 29 2016
Numbers for gifs and fullscreen video are in: Fullscreen power shows very clear gains, about 2x reduction from no-overlays to overlays. Gifs shows that CA compositor actually does worse than no CA-compositor!
,
Aug 29 2016
I filed https://bugs.chromium.org/p/chromium/issues/detail?id=642176 to investigate GIF power usage. Otherwise, this bug is complete, as all desired test cases are measured.
,
Aug 29 2016
"...NoGPU..." is the CA compositor case?
,
Aug 29 2016
No, NoGPU is not using the GPU at all.
,
Aug 29 2016
How is the graph showing CA compositor doing worse than non-CA compositor?
,
Aug 30 2016
The problem is that the colors for the lines are too similar. the GIF graphs (c#24, image #2), CA compositor (SharedPageState) is about 10% worse than GL compositor (NoOverlays)
,
Aug 30 2016
Got it - thank you. |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by nedngu...@google.com
, Jun 2 2016