power_LoadTest: Improve visibility into page load failures |
|
Issue description
Currently power_LoadTest provides some detail of success/failure of page loads in the form of these keyvals:
ext_audio_failed_loads{perf}=29_71_0_71_4_71_71_37
ext_audio_successful_loads{perf}=1_0_1_0_1_0_0_1
ext_docs_failed_loads{perf}=0_0_0_0_0_0_0_0
ext_docs_successful_loads{perf}=6_6_6_6_6_6_6_6
ext_email_failed_loads{perf}=0_0_0_0_0_0_0_0
ext_email_successful_loads{perf}=3_3_3_3_3_3_3_3
ext_status{perf}=good_good_good_good_good_good_good_good
ext_web_failed_loads{perf}=47_22_31_40_31_41_41_44
ext_web_successful_loads{perf}=28_33_31_30_31_29_29_29
where ext_<cycle>_failed_loads represent failed attempts to load the URL in time allotted for each of the test loops. For example,
ext_audio_failed_loads{perf}=29_71_0_71_4_71_71_37
29 is failed loads of an audio URL in the first loop, 71 in second loop and so on.
ext_<task>_successful_loads tracks when URL loads succeed before the timeout.
Possible ways to improve this data,
1. Add ext_<task>_desired_loads{perf}. While seasoned testers know the desired loads (36 for task=web, 6 for task=doc, etc) would be good to publish probably
2. Output to log or keyval specific URL failure and timestamp. the _failed_ keyval is an aggregate which when non-zero does indicate test unhealthiness but without any specifics. For example for web task loop1 there are 47 failed, 28 successful. With desired @ 36 we could speculate 47 / (36 - 28) =~ 5 attempts to load each of those failed URLs occurred over the 60secs allotted but knowing details of URL & timestamp would make that clear.
3. Add network health/data keyvals or logging. If facilities exist to track the resulting payload size on success or the specific HTTP status error on failure that would be useful. Additionally a keyval(s) aggregating the transmit & receive size and/or D0 (running) vs non-D0 (idle) status of the network connection could provide visibility into network power.
,
Sep 20 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/7c5ed505418980be00d6f5770ec6e5aa67b059ff commit 7c5ed505418980be00d6f5770ec6e5aa67b059ff Author: Ruben Rodriguez Buchillon <coconutruben@chromium.org> Date: Mon Sep 12 22:17:30 2016 power_LoadTest: Added mechanism to track time/url of successful and failed loads Extension script records on top of current success/failure mechanism url, timestamp, and load success/failure. Script puts this into the debug logging, but keeps it away from the keyvals. BUG=chromium:642383 TEST=power_LoadTest load success: 09/15 15:25:04.763 DEBUG| power_LoadTest:0682| [extension] @ loop_0 15:16:29:514 [load success] http://www.microsoft.com load failure: 09/15 15:25:04.765 DEBUG| power_LoadTest:0682| [extension] @ loop_0 15:16:41:36 [load failure] http://www.myspace.com load failure details: 09/15 15:25:04.719 DEBUG| power_LoadTest:0682| [extension] @ loop_0 15:19:51:356 [load failure details] http://www.twitter.com 15:19:48:437 | [response code] 200 | [url] http://www.shoplocal.com/static/common/img/favicon.ico| | [status line] HTTP/1.1 200 OK 15:19:48:891 | [response code] 200 | [url] https://abs.twimg.com/a/1473752312/css/t1/twitter_more_2.bundle.css| | [status line] HTTP/1.1 200 OK [...] 15:19:48:896 | [response code] 200 | [url] https://abs.twimg.com/a/1473752312/css/t1/twitter_more_1.bundle.css| | [status line] HTTP/1.1 200 OK Change-Id: I49ab90f20fb872f77351083ca5234c4f3a67de44 Reviewed-on: https://chromium-review.googlesource.com/384338 Tested-by: Ruben Rodriguez Buchillon <coconutruben@chromium.org> Reviewed-by: Todd Broch <tbroch@chromium.org> Commit-Queue: Ruben Rodriguez Buchillon <coconutruben@chromium.org> [modify] https://crrev.com/7c5ed505418980be00d6f5770ec6e5aa67b059ff/client/site_tests/power_LoadTest/extension/test.js [modify] https://crrev.com/7c5ed505418980be00d6f5770ec6e5aa67b059ff/client/site_tests/power_LoadTest/extension/time.js [modify] https://crrev.com/7c5ed505418980be00d6f5770ec6e5aa67b059ff/client/site_tests/power_LoadTest/power_LoadTest.py [modify] https://crrev.com/7c5ed505418980be00d6f5770ec6e5aa67b059ff/client/site_tests/power_LoadTest/extension/manifest.json
,
Jan 11
You started fixing this bug over two years ago. Are you still working on it? |
|
►
Sign in to add a comment |
|
Comment 1 by tbroch@chromium.org
, Sep 7 2016Owner: coconutruben@chromium.org
Status: Started (was: Available)