New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 652243 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 653110
Owner:
Last visit > 30 days ago
Closed: Oct 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

5.7% regression in startup.cold.blank_page at 421564:421602

Project Member Reported by tdres...@chromium.org, Oct 3 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=652243

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg1aCjqwkM


Bot(s) for this bug's original alert(s):

chromium-rel-mac10
Cc: catapult-deps-roller@chromium.org
Owner: catapult-deps-roller@chromium.org

=== Auto-CCing suspected CL author catapult-deps-roller@chromium.org ===

Hi catapult-deps-roller@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Roll src/third_party/catapult/ f332dd6dd..deb46ea2e (2 commits).
Author  : catapult-deps-roller
Commit description:
  
https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/f332dd6dd55d..deb46ea2ee48

$ git log f332dd6dd..deb46ea2e --date=short --no-merges --format='%ad %ae %s'
2016-09-28 nednguyen [Telemetry] Add API for updating ts_proxy_server's traffic settings
2016-09-28 hjd Fix docs typo bator -> battor

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/2380803002
Cr-Commit-Position: refs/heads/master@{#421602}
Commit  : 6a4a338738b1cfe08ab09d7687136664adbbe686
Date    : Wed Sep 28 19:29:59 2016


===== TESTED REVISIONS =====
Revision                             Mean     Std Dev  N   Good?
chromium@421563                      2790.63  45.1314  8   good
chromium@421583                      2799.0   34.6194  5   good
chromium@421593                      2770.0   52.274   8   good
chromium@421598                      2785.25  39.7986  8   good
chromium@421600                      2784.63  51.3307  8   good
chromium@421601                      2836.89  80.3836  18  good
chromium@421601,catapult@deb46ea2ee  2844.83  30.4809  18  good
chromium@421602                      2923.83  94.9582  12  bad    <--

Bisect job ran on: mac_10_10_perf_bisect
Bug ID: 652243

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests startup.cold.blank_page
Test Metric: foreground_tab_load_complete/foreground_tab_load_complete
Relative Change: 4.83%
Score: 98.0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2415
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8999819650259257664


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5857411078815744

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Cc: robert...@chromium.org
Owner: nedngu...@google.com
Ned, this looks like the regression was caused by your tsproxy change.

Roberto, why didn't this descend into the catapult roll?
Cc: -robert...@chromium.org
Owner: robert...@chromium.org
No, my CL is just adding the API but not calling it anywhere.
Mergedinto: 653110
Status: Duplicate (was: Untriaged)

===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : [Telemetry] Add API for updating ts_proxy_server's traffic settings
Author  : nednguyen
Commit description:
  
BUG=catapult:#2584

Review-Url: https://codereview.chromium.org/2346093002
Commit  : deb46ea2ee48a9c8cd4acd57bd9e33d03f2f6d1b
Date    : Wed Sep 28 17:22:38 2016


===== TESTED REVISIONS =====
Revision                             Mean     Std Dev  N  Good?
chromium@421563                      2844.38  42.1966  8  good
chromium@421583                      2832.5   74.7759  8  good
chromium@421593                      2825.38  38.2844  8  good
chromium@421598                      2838.13  63.5305  8  good
chromium@421600                      2829.38  64.2182  8  good
chromium@421601                      2851.38  57.5151  8  good
chromium@421601,catapult@d5a6d996a3  2813.0   46.9628  5  good
chromium@421601,catapult@deb46ea2ee  2944.8   46.837   5  bad    <--
chromium@421602                      2988.38  134.184  8  bad

Bisect job ran on: mac_10_10_perf_bisect
Bug ID: 652243

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests startup.cold.blank_page
Test Metric: foreground_tab_request_start/foreground_tab_request_start
Relative Change: 6.27%
Score: 99.0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2428
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8999549561669338896


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5828623724969984

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

===== BISECT JOB RESULTS =====
Status: failed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@421553  3727.73  205.134  8  good
chromium@421588  3976.28  136.514  5  good
chromium@421622  4265.9   174.186  8  bad

Bisect job ran on: win_perf_bisect
Bug ID: 652243

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests startup.cold.blank_page
Test Metric: window_display_time/window_display_time
Relative Change: 11.88%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6981
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8999549557800390480


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4934279530807296

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 11 by 42576172...@developer.gserviceaccount.com, Mar 24 2017


=== BISECT JOB RESULTS ===
Bisect was unable to run to completion

Error: INFRA_FAILURE

The bisect was able to narrow the range, you can try running with:
  good_revision: 840961a10fc0c0e24299ced23a60908ffa09e9d1
  bad_revision : 1a46e69915c9b8e7d715243a470b8a7975e74109

If failures persist contact the team (see below) and report the error.


Bisect Details
  Configuration: win_perf_bisect
  Benchmark    : startup.cold.blank_page
  Metric       : window_display_time/window_display_time

Revision             Result                  N
chromium@421553      4392.13 +- 259.798      6      good
chromium@421588      4492.4 +- 108.59        6      good
chromium@421605      4201.1 +- 451.319       6      good
chromium@421622      4923.37 +- 250.272      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests startup.cold.blank_page

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8984378570203784208

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4934279530807296


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Sign in to add a comment