New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 642383 link

Starred by 1 user

Issue metadata

Status: Started
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

power_LoadTest: Improve visibility into page load failures

Project Member Reported by tbroch@chromium.org, Aug 30 2016

Issue description

Currently power_LoadTest provides some detail of success/failure of page loads in the form of these keyvals:

ext_audio_failed_loads{perf}=29_71_0_71_4_71_71_37
ext_audio_successful_loads{perf}=1_0_1_0_1_0_0_1
ext_docs_failed_loads{perf}=0_0_0_0_0_0_0_0
ext_docs_successful_loads{perf}=6_6_6_6_6_6_6_6
ext_email_failed_loads{perf}=0_0_0_0_0_0_0_0
ext_email_successful_loads{perf}=3_3_3_3_3_3_3_3
ext_status{perf}=good_good_good_good_good_good_good_good
ext_web_failed_loads{perf}=47_22_31_40_31_41_41_44
ext_web_successful_loads{perf}=28_33_31_30_31_29_29_29

where ext_<cycle>_failed_loads represent failed attempts to load the URL in time allotted for each of the test loops.  For example,

  ext_audio_failed_loads{perf}=29_71_0_71_4_71_71_37

29 is failed loads of an audio URL in the first loop, 71 in second loop and so on.

ext_<task>_successful_loads tracks when URL loads succeed before the timeout.

Possible ways to improve this data,

1. Add ext_<task>_desired_loads{perf}.  While seasoned testers know the desired loads (36 for task=web, 6 for task=doc, etc) would be good to publish probably
2. Output to log or keyval specific URL failure and timestamp.  the _failed_ keyval is an aggregate which when non-zero does indicate test unhealthiness but without any specifics.  For example for web task loop1 there are 47 failed, 28 successful.  With desired @ 36 we could speculate 47 / (36 - 28) =~ 5 attempts to load each of those failed URLs occurred over the 60secs allotted but knowing details of URL & timestamp would make that clear.
3. Add network health/data keyvals or logging.  If facilities exist to track the resulting payload size on success or the specific HTTP status error on failure that would be useful.  Additionally a keyval(s) aggregating the transmit & receive size and/or D0 (running) vs non-D0 (idle) status of the network connection could provide visibility into network power.




 
Cc: -coconutruben@chromium.org dbasehore@chromium.org
Owner: coconutruben@chromium.org
Status: Started (was: Available)
Ruben's made some progress on this so sending it his way
Project Member

Comment 2 by bugdroid1@chromium.org, Sep 20 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/7c5ed505418980be00d6f5770ec6e5aa67b059ff

commit 7c5ed505418980be00d6f5770ec6e5aa67b059ff
Author: Ruben Rodriguez Buchillon <coconutruben@chromium.org>
Date: Mon Sep 12 22:17:30 2016

power_LoadTest: Added mechanism to track time/url of successful and failed loads

Extension script records on top of current success/failure
mechanism url, timestamp, and load success/failure.
Script puts this into the debug logging, but keeps it
away from the keyvals.

BUG=chromium:642383
TEST=power_LoadTest

load success:

09/15 15:25:04.763 DEBUG|    power_LoadTest:0682| [extension] @ loop_0 15:16:29:514 [load success] http://www.microsoft.com

load failure:

09/15 15:25:04.765 DEBUG|    power_LoadTest:0682| [extension] @ loop_0 15:16:41:36 [load failure] http://www.myspace.com

load failure details:

09/15 15:25:04.719 DEBUG|    power_LoadTest:0682| [extension] @ loop_0 15:19:51:356 [load failure details] http://www.twitter.com

15:19:48:437 | [response code] 200 | [url] http://www.shoplocal.com/static/common/img/favicon.ico|  | [status line] HTTP/1.1 200 OK
15:19:48:891 | [response code] 200 | [url] https://abs.twimg.com/a/1473752312/css/t1/twitter_more_2.bundle.css| | [status line] HTTP/1.1 200 OK

[...]

15:19:48:896 | [response code] 200 | [url] https://abs.twimg.com/a/1473752312/css/t1/twitter_more_1.bundle.css|  | [status line] HTTP/1.1 200 OK

Change-Id: I49ab90f20fb872f77351083ca5234c4f3a67de44
Reviewed-on: https://chromium-review.googlesource.com/384338
Tested-by: Ruben Rodriguez Buchillon <coconutruben@chromium.org>
Reviewed-by: Todd Broch <tbroch@chromium.org>
Commit-Queue: Ruben Rodriguez Buchillon <coconutruben@chromium.org>

[modify] https://crrev.com/7c5ed505418980be00d6f5770ec6e5aa67b059ff/client/site_tests/power_LoadTest/extension/test.js
[modify] https://crrev.com/7c5ed505418980be00d6f5770ec6e5aa67b059ff/client/site_tests/power_LoadTest/extension/time.js
[modify] https://crrev.com/7c5ed505418980be00d6f5770ec6e5aa67b059ff/client/site_tests/power_LoadTest/power_LoadTest.py
[modify] https://crrev.com/7c5ed505418980be00d6f5770ec6e5aa67b059ff/client/site_tests/power_LoadTest/extension/manifest.json

You started fixing this bug over two years ago. Are you still working on it? 

Sign in to add a comment