New issue
Advanced search Search tips

Issue 671597 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Jul 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

4% regression in memory.top_10_mobile_stress at 436267:436297

Project Member Reported by tdres...@chromium.org, Dec 6 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=671597

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg5_2dswoM


Bot(s) for this bug's original alert(s):

android-nexus5

===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@436266  5476861  262759   27  good
chromium@436297  5461629  287706   27  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 671597

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=https.m.facebook.com.rihanna memory.top_10_mobile_stress
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/background/after_https_m_facebook_com_rihanna
Relative Change: 0.28%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4409
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994019730544557440


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5691676918546432

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Cc: csharrison@chromium.org
Owner: csharrison@chromium.org

=== PERF REGRESSION ===


=== Auto-CCing suspected CL author csharrison@chromium.org ===

Hi csharrison@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Don't remove CSSPreloaderResourceClient for unused speculative markup preloads
Author  : csharrison
Commit description:
  
The CSS @import scanner attaches a passive resource client to a css
preload request. This passive client should not affect the policy
decisions of the preload and should just observe notifications passively.

This patch fixes a bug where removing a passive client from an otherwise
unused preload ends up cancelling it, which removes the preload from
memory cache. This is very wrong behavior, and causes the optimization
to be less effective, and report bad metrics.

Simply not removing the client will not cause the resource to live
longer than necessary, because the client holds only weak references
to the resource.

BUG= 670295 , 662999 

Review-Url: https://codereview.chromium.org/2542183002
Cr-Commit-Position: refs/heads/master@{#436312}
Commit  : dff0384e2ab5ade040a08a38e72eece575ea0aee
Date    : Mon Dec 05 16:28:44 2016


===== TESTED REVISIONS =====
Revision         Mean      Std Dev   N   Good?
chromium@436272  135180    0.0       5   good
chromium@436296  135180    0.0       5   good
chromium@436308  135180    0.0       8   good
chromium@436311  1982818   14311739  12  good
chromium@436312  11221005  0.0       5   bad    <--
chromium@436313  11221005  0.0       5   bad
chromium@436314  9835277   10369840  8   bad
chromium@436320  11221005  0.0       5   bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 671597

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=https.mobile.twitter.com.justinbieber.skip.interstitial.true memory.top_10_mobile_stress
Test Metric: memory:chrome:all_processes:reported_by_chrome:skia:effective_size_avg/foreground/https_mobile_twitter_com_justinbieber_skip_interstitial_true
Relative Change: 8200.79%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3533
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994014126007007312


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5771965795663872

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Components: Internals>Skia
Owner: ----
It looks like the skia metric regressed to almost the exact same value for a brief period in 436007 - 436065:
http://test-results.appspot.com/revision_range?start=436007&end=436065

It has had many similar spikes in the past, all of which went pretty much right away. This one seems to be sticking.

This behavior indicates to me that the metric is flaky/fragile on this page, and I am a bit skeptical that my CL is to blame. Maybe a skia expert could help here?

Hopefully it helps, I have spun up a trace bisect here:
https://codereview.chromium.org/2554103002
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Apr 11 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : memory.top_10_mobile_stress
  Metric       : memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/background/after_https_m_facebook_com_rihanna

Revision             Result                 N
chromium@436266      5476861 +- 262759      27      good
chromium@436297      5461629 +- 287706      27      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=https.m.facebook.com.rihanna memory.top_10_mobile_stress

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982611676896708848

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5691676918546432


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Status: WontFix (was: Untriaged)
This alert was found before M-60 branched. Closing as WontFix as this is believed to either be invalid or non-reproducible. 

Sign in to add a comment