Issue metadata
Sign in to add a comment
|
Run Control and Enabled tests on the same bot and upload the difference |
||||||||||||||||||||||
Issue descriptionWe want to reduce the noise by monitoring the difference between feature on/off on the same bot. More Discussions: https://groups.google.com/a/google.com/forum/#!topic/chrome-speed-services/MVp4dMdkBxk
,
Apr 6 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/51164e7b6bc047ac6ff0c0cd21b5f544a777fac4 commit 51164e7b6bc047ac6ff0c0cd21b5f544a777fac4 Author: Chong Zhang <chongz@chromium.org> Date: Fri Apr 06 19:06:29 2018 Run loading.desktop with Network Service on/off and record the difference Previously we run loading.desktop on the perf fyi bot with Network Service on, but it's hard to compare the numbers manually or setup sheriff alerts. This CL modified loading.desktop.network_service to run the test twice with feature on/off on the same bot, and report the diff to the perf dashboard (as well as the original data). More specifically, we will load the resulting 'results-chart.json' from two runs and combine them into one (with appropriated suffix). e.g. We will turn 'warm@@timeToFirstContentfulPaint_avg' into: * 'warm@@timeToFirstContentfulPaint_avg_control' * 'warm@@timeToFirstContentfulPaint_avg_enabled' * 'warm@@timeToFirstContentfulPaint_avg_diff' And things like 'timeToFirstContentfulPaint_avg_diff' will be the interesting metrics. Note: Please see bug for an example generated 'results-chart.json'. Also Note: --output-format=chartjson is going away soon, this CL added myself to 'tools/perf/contrib/network_service/OWNERS' for future maintainence. Bug: 827442 Change-Id: I26cd027d4b59f6b927736a43f068bd0d89d279c8 Reviewed-on: https://chromium-review.googlesource.com/987305 Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org> Reviewed-by: Simon Hatch <simonhatch@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Chong Zhang <chongz@chromium.org> Cr-Commit-Position: refs/heads/master@{#548879} [modify] https://crrev.com/51164e7b6bc047ac6ff0c0cd21b5f544a777fac4/tools/perf/contrib/network_service/OWNERS [modify] https://crrev.com/51164e7b6bc047ac6ff0c0cd21b5f544a777fac4/tools/perf/contrib/network_service/loading.py
,
Apr 10 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/29ea2a37f4e2c7b2af5596a56dfee23960aaf272 commit 29ea2a37f4e2c7b2af5596a56dfee23960aaf272 Author: Chong Zhang <chongz@chromium.org> Date: Tue Apr 10 06:17:08 2018 [Mojo FYI] Only merge common charts and points 'loading.desktop.network_service' fails randomly when uploading json results, e.g. https://ci.chromium.org/buildbot/chromium.perf.fyi/Mojo%20Linux%20Perf/5265 It seems that we might miss some charts in certain conditions. e.g. We don't have 'timeToInteractive_std' when all values are None. This CL handled the case by only doing diff on common charts / points between control and enabled chartjson result. Bug: 827442 Change-Id: Ibb1b9004d48bdfc2fa017c4acf57c989ae15c17f Reviewed-on: https://chromium-review.googlesource.com/1003520 Reviewed-by: Kouhei Ueno <kouhei@chromium.org> Commit-Queue: Chong Zhang <chongz@chromium.org> Cr-Commit-Position: refs/heads/master@{#549434} [modify] https://crrev.com/29ea2a37f4e2c7b2af5596a56dfee23960aaf272/tools/perf/contrib/network_service/loading.py
,
May 1 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b5d85ba7122683f2522c67c77dc89c7aa1264112 commit b5d85ba7122683f2522c67c77dc89c7aa1264112 Author: Chong Zhang <chongz@chromium.org> Date: Tue May 01 01:20:34 2018 [Mojo Perf FYI] Add none_value_reason to artificial points with None value The Mojo Linux Perf bot frequently fails due to invalid JSON. This patch fixes the issue by adding 'none_value_reason' to artificial points that has None value. Bug: 827442 Change-Id: I508091a052184c439f4de712724f8f691bd1bbd9 Reviewed-on: https://chromium-review.googlesource.com/1036463 Reviewed-by: Jun Cai <juncai@chromium.org> Commit-Queue: Chong Zhang <chongz@chromium.org> Cr-Commit-Position: refs/heads/master@{#554954} [modify] https://crrev.com/b5d85ba7122683f2522c67c77dc89c7aa1264112/tools/perf/contrib/network_service/loading.py
,
May 1 2018
The build is consistently green since https://ci.chromium.org/buildbot/chromium.perf.fyi/Mojo%20Linux%20Perf/5917 |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by chongz@chromium.org
, Mar 30 20184.5 MB
4.5 MB View Download