add power consumption metrics to existing tests |
|||||
Issue descriptionFor new Intel chips only the dev image contains a binary /usr/bin/dump_intel_rapl_consumption One can run it in the background during some benchmark using something like dump_intel_rapl_consumption --interval_ms=1000 --repeat Recommendation is for about 10s. Find a way to add this generically to all kind of existing tests to report the values to the chromeperf dashboard. Maybe start with graphics_WebGLAquarium etc, but make sure it also works for video_* tests and Android apps. That said there will be an interesting interplay with thermal throttling. But maybe lets not worry about this yet.
,
Oct 18 2017
There's rapl support in autotest here: autotest/files/client/cros/power_rapl.py and instantiations in power_Consumption, power_LoadTest & power_Idle I believe.
,
Oct 18 2017
So what do you do with the data? I don't see it monitored on either chromeperf or crosbolt.
,
Oct 19 2017
>so what do you do w/ the data? look at it on local runs. Happy to have it published though if there's no pushback to chromeperf/crosbolt keyval bloat.
,
Oct 19 2017
Record purpose:
Sample output from the existing rapl calculation.
{'pkg_pwr_min': 1.9148703301297543, 'pkg_pwr_max': 5. 1311435841242989, 'pp0_pwr_std': 0.58924593533649061, 'pp0_pwr_min': 0.61874864336752411, 'pp1_pwr_max': 1. 8508749749866691, 'pp1_pwr_cnt': 65, 'pp1_pwr_std': 0.51707656175626571, 'pp0_duration': 67.968544960021973, 'pp0_pwr_cnt': 65, 'pkg_pwr_cnt': 65, 'pkg_pwr': 3.7998257481004245, 'pp1_pwr': 1.1735954084247673, 'pp0_pwr': 1.3318093813262382, 'pp1_duration': 67.968544960021973, 'pp1_pwr_min': 0.043786294527125326, 'pp0_pwr_max': 1.9868078854601685, 'pkg_pwr_std': 1.0478888117141691, 'pkg_duration': 67.968544960021973}
I can't find 'dram' here.
The chromeperf already has its own min/max/std calculation if we report list of scalars. If we are gonna using the existing tool, it might be better to have the raw data.
,
Oct 21 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/fb8df0a00ecb28d4b80c59347a6ccde7fd11801f commit fb8df0a00ecb28d4b80c59347a6ccde7fd11801f Author: Po-Hsien Wang <pwang@chromium.org> Date: Sat Oct 21 06:55:10 2017 graphics_WebGLAquarium: rapl power consumption Add rapl power consumption reading to graphics_WebGLAquarium. BUG= chromium:775742 TEST=test_that graphics_WebGLAquarium.power Change-Id: I1c58fda1190d9f678f853812c2f6e481ee9d8c52 Reviewed-on: https://chromium-review.googlesource.com/729299 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/fb8df0a00ecb28d4b80c59347a6ccde7fd11801f/client/cros/power_status.py [modify] https://crrev.com/fb8df0a00ecb28d4b80c59347a6ccde7fd11801f/client/cros/power_rapl.py [modify] https://crrev.com/fb8df0a00ecb28d4b80c59347a6ccde7fd11801f/client/site_tests/graphics_WebGLAquarium/graphics_WebGLAquarium.py
,
Oct 24 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/f73a9ec6f2c96ce2d48f74130dfb1ca21839322b commit f73a9ec6f2c96ce2d48f74130dfb1ca21839322b Author: Po-Hsien Wang <pwang@chromium.org> Date: Tue Oct 24 05:39:01 2017 graphics_WebGLManyP..: add rapl power measurement Add power consumption measurement after the tests and reports the measurement to chromeperf for regression monitor. BUG= chromium:775742 TEST=test_that cave graphics_WebGLManyPlanetDeep CQ-DEPEND=CL:734328 Change-Id: I4882f1a3cc05ab73b5af8ef979ef2bf4b14ce443 Reviewed-on: https://chromium-review.googlesource.com/734306 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/f73a9ec6f2c96ce2d48f74130dfb1ca21839322b/client/site_tests/graphics_WebGLManyPlanetsDeep/graphics_WebGLManyPlanetsDeep.py
,
Oct 24 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/8afce4feab4921896c0ceb638c501b3951f5cf90 commit 8afce4feab4921896c0ceb638c501b3951f5cf90 Author: Po-Hsien Wang <pwang@chromium.org> Date: Tue Oct 24 05:39:01 2017 power_utils: disable rapl for SdyBridge/IvyBridge 1) Delete SandyBridge and IvyBridge from rapl supported architecture 2) PEP8 format See https://wmatrix.googleplex.com/unfiltered?hide_missing=True&releases=tot&tests=graphics_WebGLAquarium&days_back=20 BUG= chromium:775742 TEST=test_that Change-Id: I230f24481be0a89d2688ebefbf39b077378e9934 Reviewed-on: https://chromium-review.googlesource.com/734328 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/8afce4feab4921896c0ceb638c501b3951f5cf90/client/cros/power_utils.py
,
Oct 26 2017
,
Oct 27 2017
We see odd bistable behavior with gfx and pkg power draw, but so far only when running Aquarium with 1000 fishes (but not 50, and not ManyPlanets etc). Basically power consumption will double from time to time (go up by 1-2W) and stay there for a minute. As we only measure 10s in steady state this causes random fluctuations on chromeperf/. But we have not seen this in other benchmarks, so will continue adding them. Without this strange effect it may be possible to measure power with about 10 percent accuracy (or 200mW), maybe even better in the future.
,
Oct 31 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/a23341a789f4e483f90b612a7c5361d5ee0f7359 commit a23341a789f4e483f90b612a7c5361d5ee0f7359 Author: Po-Hsien Wang <pwang@chromium.org> Date: Tue Oct 31 22:15:28 2017 power_status: Remove statistics option We introduced statistics option for power_status.MeasurementLogger to return the readings in raw format instead of min/max/std/mean. (CL:729299) Let's delete it, add '_avg' suffix to the mean value and return the readings directly. Before: { 'tname_pp0_pwr': power_mean, 'tname_pp0_pwr_std': power_std, ... } After: { 'tname_pp0_pwr': power_readings, 'tname_pp0_pwr_avg': power_mean, 'tname_pp0_pwr_std': power_std, ... } BUG= chromium:775742 TEST=test_that cave graphics_WebGLAquarium/graphics_WebGLManyPlanetsDeep and check results-chart.json to see if the report is in same format. Change-Id: I05e166e1838b4196ef1e123cd94aeabcd4809bb3 Reviewed-on: https://chromium-review.googlesource.com/741035 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Todd Broch <tbroch@chromium.org> [modify] https://crrev.com/a23341a789f4e483f90b612a7c5361d5ee0f7359/client/site_tests/graphics_WebGLManyPlanetsDeep/graphics_WebGLManyPlanetsDeep.py [modify] https://crrev.com/a23341a789f4e483f90b612a7c5361d5ee0f7359/client/cros/power_status.py [modify] https://crrev.com/a23341a789f4e483f90b612a7c5361d5ee0f7359/client/cros/power_rapl.py [modify] https://crrev.com/a23341a789f4e483f90b612a7c5361d5ee0f7359/client/site_tests/graphics_WebGLAquarium/graphics_WebGLAquarium.py
,
Nov 3 2017
,
Nov 5 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/8d81870391f9903eef6c47384dada4483708011c commit 8d81870391f9903eef6c47384dada4483708011c Author: Po-Hsien Wang <pwang@chromium.org> Date: Sun Nov 05 07:54:19 2017 video_YouTubeHTML5: rapl power consumption 1) Add rapl power consumption to chromeperf regression monitoring. 2) PEP8 format youtube_helper BUG= chromium:775742 TEST=test_that cave video_YouTubeHTML5 Change-Id: I311bd6f784b4a97129d13b73adacd9f4da55c779 Reviewed-on: https://chromium-review.googlesource.com/740105 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Todd Broch <tbroch@chromium.org> [modify] https://crrev.com/8d81870391f9903eef6c47384dada4483708011c/client/cros/video/youtube_helper.py [modify] https://crrev.com/8d81870391f9903eef6c47384dada4483708011c/client/site_tests/video_YouTubeHTML5/video_YouTubeHTML5.py [modify] https://crrev.com/8d81870391f9903eef6c47384dada4483708011c/tko/perf_upload/perf_dashboard_config.json
,
Nov 6 2017
graphics_WebGLManyPlanetsDeep does not have the bistable effect and is stable with about 10% accuracy in gfx, dram readings. pp0, and pkg readings are heavily inaccurate in both graphics_WebGLAquarium and graphics_WebGLManyPlanetsDeep in lab, but our local experiment on a single DUT does not have the same behavior. My guess is that the variant of the board (different CPU/frequency) makes pp0 power consumption varies.
,
Nov 7 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/8eac583ecd1c3e33d00b0048f0f898640c83cb54 commit 8eac583ecd1c3e33d00b0048f0f898640c83cb54 Author: Todd Broch <tbroch@google.com> Date: Tue Nov 07 23:25:16 2017 power_status.py: remove statistics arg. Commit, a23341a78 power_status: Remove statistics option meant to remove this recently added arg but missed this one. BUG= chromium:775742 TEST=test_that power_LoadTest.fast_gaia passes Change-Id: Ibfe5cd4b5d1a4aad65149f6ffd0ff3acba5bfc15 Reviewed-on: https://chromium-review.googlesource.com/753134 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Todd Broch <tbroch@chromium.org> Reviewed-by: Po-Hsien Wang <pwang@chromium.org> [modify] https://crrev.com/8eac583ecd1c3e33d00b0048f0f898640c83cb54/client/cros/power_status.py
,
Nov 7 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/b8090190ffeeb29fe56386362d73b754728e802d commit b8090190ffeeb29fe56386362d73b754728e802d Author: Todd Broch <tbroch@google.com> Date: Tue Nov 07 23:25:16 2017 power: fixup system_pwr average keyval usage Commit, a23341a78 power_status: Remove statistics option changed the keyval for power logging average to include suffix _avg. This change incorporates that into tests that consume keyval. BUG= chromium:775742 TEST=power_LoadTest.fast_gaia & power_Consumption.fast passes Change-Id: I3ec95f03ed9f0213ed506ffdb0c42b369bbee342 Reviewed-on: https://chromium-review.googlesource.com/753135 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Todd Broch <tbroch@chromium.org> Reviewed-by: Po-Hsien Wang <pwang@chromium.org> [modify] https://crrev.com/b8090190ffeeb29fe56386362d73b754728e802d/client/site_tests/power_LoadTest/power_LoadTest.py [modify] https://crrev.com/b8090190ffeeb29fe56386362d73b754728e802d/client/site_tests/power_Consumption/power_Consumption.py
,
Nov 30 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/15ba61f4a55883887066362a023fc1b083b53392 commit 15ba61f4a55883887066362a023fc1b083b53392 Author: Po-Hsien Wang <pwang@chromium.org> Date: Thu Nov 30 02:05:04 2017 video_PlaybackPerf: add rapl power measurement Adding rapl power measurement along with existing power consumption tests. Eg. 8 entries will be added to video_PlaybackPerf.power_vp8 under rapl_power_consumption graph "sw_vp8_1080p_pkg_pwr" "hw_vp8_1080p_gfx_pwr" "hw_vp8_1080p_dram_pwr" "hw_vp8_1080p_pp0_pwr" "sw_vp8_1080p_pp0_pwr" "hw_vp8_1080p_pkg_pwr" "sw_vp8_1080p_gfx_pwr" "sw_vp8_1080p_dram_pwr" BUG= chromium:775742 TEST=test_that cave video_PlaybackPerf.power_vp9 Change-Id: Iaf370c8989b3a0485b9e74a6754fd2bd311af218 Reviewed-on: https://chromium-review.googlesource.com/792471 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/15ba61f4a55883887066362a023fc1b083b53392/client/site_tests/video_PlaybackPerf/video_PlaybackPerf.py
,
Dec 4 2017
I looked at some of the new poer results and they look plausible (between different sw/hw codecs). Now let it soak for a few weeks to collect more data!
,
Dec 11 2017
I filed the bug due to this change. PTAL. crbug.com/793703
,
Jan 30 2018
Now that we have some data, notice strong correlation between upper and lower sets of graphs: https://chromeperf.appspot.com/report?sid=653a4b0d5b1ccd09a7f05c94e245938313e0aa69038544a9e876526fb32f1d76&start_rev=32800051017600100&end_rev=33220001031900000 In other words, fairly confident that we are measuring something real.
,
Apr 24 2018
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/autotest-cheets/+/6a20cc303529014fec80ad3dc426d8c8d8689f33 commit 6a20cc303529014fec80ad3dc426d8c8d8689f33 Author: Todd Broch <tbroch@chromium.org> Date: Tue Apr 24 07:30:19 2018 |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by pwang@chromium.org
, Oct 18 2017To Record purpose: Sample running example: dump_intel_rapl_consumption --interval_ms=1000 --repeat pkg pp0 gfx dram (Note: 'pkg' includes 'pp0' and 'gfx'. Values in Watts) 5.859541 4.220923 0.261325 0.809362 5.876464 4.211110 0.269259 0.834264 5.455055 3.790129 0.269015 0.832249 4.717682 3.093185 0.252664 0.803949 4.496505 2.896543 0.241312 0.779902 4.269358 2.682439 0.226354 0.769201 4.440240 2.800790 0.240519 0.830863 4.331193 2.688061 0.248691 0.831086 4.510513 2.854620 0.252781 0.839145 4.585449 2.918590 0.255473 0.855217 4.693796 3.032435 0.253918 0.848469