New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 648655 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Closed: Dec 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

1271.4%-1286.9% regression in smoothness.tough_filters_cases at 418806:418830

Project Member Reported by jasontiller@chromium.org, Sep 20 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=648655

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg-Zb_qwkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg-c6HqwoM


Bot(s) for this bug's original alert(s):

chromium-rel-win10
chromium-rel-win7-gpu-nvidia
Cc: senorblanco@chromium.org
+senorblanco@, thist test seem to be really noisy. Do you happen to have any insights into this?
Sadly, I've not managed to track down the noise here. It also seems to be platform-specific. Seems to be fine on Mac, for example, although did regress a bit around the same time: https://chromeperf.appspot.com/report?sid=f7e3a3351be90d8e5c58534c8694f09e885a3d7b8dc923d74f86c3f0db4efb9c&start_rev=413321&end_rev=419741
Note that the _ref version of this test regressed (or more accurately, stabilized at the upper bound) at the same time, so it's possible that this is something bot-related, and not an actual regression in Chrome.

https://chromeperf.appspot.com/report?sid=37c544f4f2d97d01f93627af2d9b73e1dc7c3ddce59267d094ba88b7821808ec&start_rev=416694&end_rev=419731
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Sep 20 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@418811  108.031  133.34   18  good
chromium@418823  235.94   12.8575  18  bad

Bisect job ran on: winx64nvidia_perf_bisect
Bug ID: 648655

Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.tough_filters_cases
Test Metric: frame_times/http___web.archive.org_web_20150502135732_http___ie.microsoft.com_testdrive_Performance_Pirates_Default.html
Relative Change: 94.73%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64nvidia_perf_bisect/builds/1872
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000982597414134128


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5345981481091072

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Sep 23 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@418805  137.134  138.497  18  good
chromium@418823  234.572  13.2651  18  bad

Bisect job ran on: winx64_10_perf_bisect
Bug ID: 648655

Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.tough_filters_cases
Test Metric: frame_times/http___web.archive.org_web_20150502135732_http___ie.microsoft.com_testdrive_Performance_Pirates_Default.html
Relative Change: 36.98%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_10_perf_bisect/builds/721
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000709644347102976


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6047306422419456

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Owner: ----
perf sheriff fixit: unassigning bugs owned by perf sheriffs to clarify that the rotation is responsible for triage.
Status: Untriaged (was: Assigned)
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Dec 17 2016

Cc: perezju@chromium.org
Owner: perezju@chromium.org

=== PERF REGRESSION ===


=== Auto-CCing suspected CL author perezju@chromium.org ===

Hi perezju@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Revert of "[telemetry] Wire TsProxy through telemetry's network stack." (patchset #1 id:1 of https://codereview.chromium.org/2335403003/ )
Author  : perezju
Commit description:
  
Reason for revert:
Did not actually break WPR. Fine to reland and enable TsProxy

Original issue's description:
> Reland of "[telemetry] Wire TsProxy through telemetry's network stack." (patchset #1 id:1 of https://codereview.chromium.org/2342503002/ )
>
> Reason for revert:
> Broke WPR replay on WebView bots
>
> Original issue's description:
> > Revert of Revert "[telemetry] Wire TsProxy through telemetry's network stack." (patchset #1 id:1 of https://codereview.chromium.org/2267613002/ )
> >
> > Reason for revert:
> > Webview now support --proxy-server flag (see attached bug)
> >
> > Original issue's description:
> > > Revert "[telemetry] Wire TsProxy through telemetry's network stack."
> > >
> > > This reverts commit 9fcccf0b65703994c9541273f00a993806a95356.
> > >
> > > Revert "[Telemetry] Explictly initialize & clean up network_controller"
> > >
> > > This reverts commit 30ce231266c6fb6147bb6df7c56a7eb75c23fccf.
> > >
> > > Seems like webview currently does not support proxy server?
> > > (See attached chromium bug)
> > >
> > > BUG=chromium:639632
> > > BUG=catapult:#2584
> > >
> > > TBR=pmeenan@chromium.org
> > >
> > > Committed: https://chromium.googlesource.com/external/github.com/catapult-project/catapult/+/947506bacfb5047857a52033256635faebf38af2
> >
> > TBR=nednguyen@chromium.org
> > # Not skipping CQ checks because original CL landed more than 1 days ago.
> > BUG=chromium:639632
> >
> > Committed: https://chromium.googlesource.com/external/github.com/catapult-project/catapult/+/b86cfef4b9da4749ff51e0fc4432869f8209dc42
>
> TBR=nednguyen@chromium.org,nednguyen@google.com
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=chromium:639632
>
> Committed: https://chromium.googlesource.com/external/github.com/catapult-project/catapult/+/32dd31095913725705e9cf5f5bc1671f32821a35

TBR=nednguyen@chromium.org,nednguyen@google.com
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:639632

Review-Url: https://codereview.chromium.org/2346773002
Commit  : 3cc40b37372927435ed69f91aec2117488b0825c
Date    : Thu Sep 15 09:25:34 2016


===== TESTED REVISIONS =====
Revision                             Mean     Std Dev  N     Good?
chromium@418811                      24.3842  2990.69  2440  good
chromium@418821                      20.0165  2136.83  2987  good
chromium@418822                      44.5766  4099.17  1313  good
chromium@418822,catapult@3cc40b3737  221.83   3927.2   264   bad    <--
chromium@418823                      239.889  3845.63  245   bad
chromium@418824                      239.066  3813.61  244   bad
chromium@418826                      230.062  3856.96  251   bad
chromium@418830                      230.116  3871.37  253   bad

Bisect job ran on: winx64nvidia_perf_bisect
Bug ID: 648655

Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...web.archive.org.web.20150502135732.http...ie.microsoft.com.testdrive.Performance.Pirates.Default.html smoothness.tough_filters_cases
Test Metric: frame_times/http___web.archive.org_web_20150502135732_http___ie.microsoft.com_testdrive_Performance_Pirates_Default.html
Relative Change: 843.71%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64nvidia_perf_bisect/builds/1980
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993071815056244160


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5274185109078016

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Cc: nednguyen@chromium.org
Status: WontFix (was: Untriaged)
Looks like nednguyen@ fixed this in https://chromium.googlesource.com/chromium/src/+/65023b46c146412aed79f66760207c7659757489, and it has been running at a solid ~60FPS since then. Closing.

Sign in to add a comment