Performance regression of page_cycler_v2.typical_25 on ChromeOS using wprgo
Reported by
lingyun....@intel.com,
Aug 30 2017
|
||||
Issue description
UserAgent: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36
Platform: 9838.0.0 (Official Build) dev-channel reef test
Steps to reproduce the problem:
1. Run './run_benchmark --browser=cros-chrome --remote=$DUT_IP page_cycler_v2.typical_25 --reset-results' using chromium repo before and after commit 93a6187 (master@{#495436}, Convert page cycler archives to wprgo)
2. In catapult repo, need to have CL 3002793002 patched to ensure run_benchmark works on CrOS
3. Compare the performance results (lower is better) in pcv1-warm cache state: FCP/timeToOnload
What is the expected behavior?
Performance is aligned before and after using the wprgo files
What went wrong?
Performance regression occurred after using the wprgo files:
wpr wprgo regression
FCP 445 470 6%
timeToOnload 1987 2072 4%
More specifically, some pages show even larger regressions than the overall average score:
theonion.com
wpr wprgo regression
FCP 427 479 12%
timeToOnload 2125 3366 58%
ticketmaster.com
wpr wprgo regression
FCP 1162 1556 34%
timeToOnload 2408 2972 23%
Did this work before? Yes
Chrome version: 62.0.3176.0 Channel: dev
OS Version: 9838.0.0
Flash Version:
,
Sep 5 2017
,
Sep 5 2017
,
Sep 5 2017
WPRGO execrise more of the network stack & more realistic. This is a test change, so this regression is Working as Intended to me.
,
Sep 5 2017
Can we ignore this "regression" if the metrics are consistent post-migration? The reason for the increase in loading metrics (e.g. time to first contentful paint) is largely due to the fact that the legacy WPR used only a single cert for all connections. WprGO serves different certs for different connections and Chrome's cert verification takes longer (cert verification cache hit is much less frequent). You can read more in Issue 756481 . I have a WIP CL (https://codereview.chromium.org/3003143002/) which will reduce the cert verification time and hopefully the metrics will be closer to what they were before the migration.
,
Sep 7 2017
Thanks for the explanation nednguyen@ xunjieli@ I tried to run page_cycler_v2 using the latest Telemetry, with the above-mentioned CL (https://codereview.chromium.org/3003143002/) merged, but failed to load two pages (airbnb.com and flickr.com). More details are reported in https://bugs.chromium.org/p/chromium/issues/detail?id=762819
,
Oct 20 2017
Hi nednguyen@ xunjieli@, recently I found an potential issue in WPR-go server, the 'Date' in response header is the date and time the response was archived in the wprgo file, rather than the date it was generated and sent from the WPR-go server, which may impact the page-loading performance in warm cache case. I uploaded a CL to address this issue, PTAL. https://chromium-review.googlesource.com/c/catapult/+/729283
,
Nov 7 2017
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/b33d83ad5d4f4f59a87d7a68efa50455fc10a76d commit b33d83ad5d4f4f59a87d7a68efa50455fc10a76d Author: Lingyun Cai <lingyun.cai@intel.com> Date: Tue Nov 07 03:33:48 2017 [wpr-go] Ensure 'Date' is updated as current date in response header In WPR-go server, the 'Date' in response header is the date and time the response was archived in the wprgo file, rather than the date it was generated and sent from the WPR-go server. This would make some resources to be requested again, instead of loading from cache directly in warm cache case, which may impact user experience and browser performance. This CL is to make sure 'Date' header is updated as 'current time', and add date shift for 'Last-Modified' and 'Expires' headers in WPR-go server. Bug: chromium:760422 Change-Id: I479beff89d67e98c16189847aa9855856130b23c Reviewed-on: https://chromium-review.googlesource.com/729283 Reviewed-by: Tom Bergan <tombergan@chromium.org> Commit-Queue: Tom Bergan <tombergan@chromium.org> [modify] https://crrev.com/b33d83ad5d4f4f59a87d7a68efa50455fc10a76d/web_page_replay_go/src/webpagereplay/proxy_test.go [modify] https://crrev.com/b33d83ad5d4f4f59a87d7a68efa50455fc10a76d/web_page_replay_go/src/webpagereplay/proxy.go
,
Nov 7 2017
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/564e33d345ee080be36ad0e6b6107afbcd09df4d commit 564e33d345ee080be36ad0e6b6107afbcd09df4d Author: Nghia Nguyen <nednguyen@google.com> Date: Tue Nov 07 14:06:23 2017 Update wprgo binaries to latest version This is follows up of the change https://chromium-review.googlesource.com/c/catapult/+/729283 which update header sent out by WPRGO. This CL is made by executing ./telemetry/bin/update_wpr_go_binary script Bug: chromium:760422 Change-Id: If880da553b06c685ace34d9c7813292170634d30 Reviewed-on: https://chromium-review.googlesource.com/756793 Commit-Queue: Helen Li <xunjieli@chromium.org> Reviewed-by: Helen Li <xunjieli@chromium.org> [modify] https://crrev.com/564e33d345ee080be36ad0e6b6107afbcd09df4d/telemetry/telemetry/internal/binary_dependencies.json
,
Nov 7
Issue has not been modified or commented on in the last 365 days, please re-open or file a new bug if this is still an issue. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot |
||||
►
Sign in to add a comment |
||||
Comment 1 by zalcorn@chromium.org
, Aug 31 2017