New issue
Advanced search Search tips

Issue 665971 link

Starred by 1 user
Status: WontFix
Owner: ----
Closed: Jul 2017
Cc:
perezju@chromium.org
simonhatch@chromium.org
benjhayden@chromium.org
sullivan@chromium.org
alex...@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 675639


Sign in to add a comment

5.5% regression in memory.top_10_mobile at 431023:431075

Project Member Reported by benjhayden@chromium.org, Nov 16 2016 
perezju: Can you take a look at the graph? I can't tell if this is a meaningful regression or if the metric was just improved to reduce noise and we discovered that the true value of the metric is higher than previously thought.

Comment 1 by benjhayden@chromium.org, Nov 16 2016 

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=665971

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgi4PYoAoM


Bot(s) for this bug's original alert(s):

android-nexus7v2
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Nov 17 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev   N   Good?
chromium@431022  63536697  15028847  27  good
chromium@431075  63989836  11603836  27  bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 665971

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.top_10_mobile
Test Metric: memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/foreground/http_search_yahoo_com_search__ylt_p_google
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3488
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995813115572254192


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6418606248689664

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 6 by perezju@chromium.org, Nov 17 2016 

Hmm, yes, it's not clear. I kicked off a couple more bisects to dig further in.
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Nov 17 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@430986  40315434  1724057  27  good
chromium@431022  40196802  1478075  27  bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 665971

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http.m.youtube.com.results.q.science memory.top_10_mobile
Test Metric: memory:chrome:all_processes:reported_by_os:system_memory:private_dirty_size_avg/background/after_http_m_youtube_com_results_q_science
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3491
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995754748771770080


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5262700895010816

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Nov 17 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@431022  64577991  3837798  27  good
chromium@431075  64648078  3688408  27  bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 665971

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http.search.yahoo.com.search..ylt.p.google memory.top_10_mobile
Test Metric: memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/foreground/http_search_yahoo_com_search__ylt_p_google
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3492
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995754715737482560


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6076698800422912

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 10 by perezju@chromium.org, Nov 17 2016 

Trying again, wider range.
Project Member

Comment 11 by 42576172...@developer.gserviceaccount.com, Nov 17 2016

Cc: alex...@chromium.org
Owner: alex...@chromium.org

=== Auto-CCing suspected CL author alexmos@chromium.org ===

Hi alexmos@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Allow navigations to non-web-accessible resources from chrome schemes.
Author  : alexmos
Commit description:
  
Recent fixes for web accessible resource checks in ShouldAllowOpenURL
(https://codereview.chromium.org/2454563003) removed the free pass
that non-HTTP(S) and non-extensions schemes used to have to load
non-web-accessible resources.  However, data gathered in that CL
showed that this is breaking navigations from NTP pages (e.g., if an
extensions page appears on the recent tiles) and chrome://history when
clicking on an extensions page.  To fix these, this CL relaxes the
rules in ShouldAllowOpenURL to allow chrome:// and chrome-search://
pages to navigate to non-WAR extension pages.

BUG=662602

Review-Url: https://codereview.chromium.org/2486843003
Cr-Commit-Position: refs/heads/master@{#431074}
Commit  : cf5145bb7919c53d6866829a0f4fa7f2af0b7ca4
Date    : Wed Nov 09 23:27:34 2016


===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@430688  40542459  1587888  18  good
chromium@430959  40412980  1656580  18  good
chromium@431027  40370654  2011118  27  good
chromium@431061  40331970  1881563  18  good
chromium@431066  40341375  1945931  27  good
chromium@431067  40213489  1658447  27  unknown
chromium@431068  40249746  2173852  27  unknown
chromium@431069  40235486  2100720  27  unknown
chromium@431070  40265675  2403799  27  unknown
chromium@431073  40327267  2263500  27  good
chromium@431074  39999739  2006237  27  bad      <--
chromium@431075  40094250  2397861  27  unknown
chromium@431076  40158724  2088971  27  unknown
chromium@431077  40205752  2001721  27  unknown
chromium@431078  40261124  1839231  27  unknown
chromium@431079  40118523  1693407  27  unknown
chromium@431080  40099863  1665076  27  unknown
chromium@431081  40016805  1326070  18  bad
chromium@431094  40300719  2081870  27  unknown
chromium@431095  40326357  1997683  27  unknown
chromium@431096  40046160  1187105  18  bad
chromium@431230  40081431  1414919  18  bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 665971

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http.m.youtube.com.results.q.science memory.top_10_mobile
Test Metric: memory:chrome:all_processes:reported_by_os:system_memory:private_dirty_size_avg/background/after_http_m_youtube_com_results_q_science
Relative Change: 1.14%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3493
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995749071458612320


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5794750974656512

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 12 by perezju@chromium.org, Nov 18 2016

Owner: ----
That bisect looks all over the place. I don't think it's telling us much.

I'll try again without story filter, let's see how that works this time.
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Nov 18 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@430688  44101199  5287783  27  good
chromium@431230  44113183  5476117  27  bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 665971

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.top_10_mobile
Test Metric: memory:chrome:all_processes:reported_by_os:system_memory:private_dirty_size_avg/background/after_http_m_youtube_com_results_q_science
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3496
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995666031750711280


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5717949644537856

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 15 by benhenry@google.com, Nov 22 2016

Status: Untriaged (was: Assigned)
No owner, changing to untriaged.

Comment 16 by perezju@chromium.org, Dec 16 2016 

Let's try again with a wider range

Comment 18 by perezju@chromium.org, Dec 16 2016 

The regression seems to come from the gpu_memory:graphics on the browser process, where it's also clearer that the "change in behavior" (noisy bot low values -> stable higher values) happened in the range 431023:431075.

If the bisect in #17 doesn't work we can try there:
https://chromeperf.appspot.com/report?sid=b8d13296616cfa034950722d71124cf50795ea7be64f6a3e71c3c08863d4b611&rev=430853
Project Member

Comment 20 by 42576172...@developer.gserviceaccount.com, Dec 16 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@430689  51462924  3525167  21  good
chromium@431304  51552646  3263018  21  bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 665971

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=https.mobile.twitter.com.justinbieber.skip.interstitial.true memory.top_10_mobile
Test Metric: memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/foreground/https_mobile_twitter_com_justinbieber_skip_interstitial_true
Relative Change: 0.17%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3557
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993120046424180000


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5896778333814784

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 21 by 42576172...@developer.gserviceaccount.com, Dec 16 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@431022  35225600  0.0      21  good
chromium@431075  35225600  0.0      21  bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 665971

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=https.mobile.twitter.com.justinbieber.skip.interstitial.true memory.top_10_mobile
Test Metric: memory:chrome:browser_process:reported_by_os:gpu_memory:graphics:proportional_resident_size_avg/foreground/https_mobile_twitter_com_justinbieber_skip_interstitial_true
Relative Change: 0.00%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3558
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993119313340976720


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5222967758290944

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 22 by perezju@chromium.org, Dec 16 2016

Cc: perezju@chromium.org

Comment 24 by perezju@chromium.org, Dec 16 2016 

From #21 I'm suspecting a device issue. That metric is indeed really clean, so maybe something changed on the device?

For now trying without --story-filter to see if that makes a difference.
Project Member

Comment 25 by 42576172...@developer.gserviceaccount.com, Dec 16 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev   N   Good?
chromium@431022  34780891  9113814   21  good
chromium@431075  34336183  12562525  21  bad

Bisect job ran on: android_nexus7_perf_bisect
Bug ID: 665971

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.top_10_mobile
Test Metric: memory:chrome:browser_process:reported_by_os:gpu_memory:graphics:proportional_resident_size_avg/foreground/https_mobile_twitter_com_justinbieber_skip_interstitial_true
Relative Change: 1.28%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3562
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993101776078159120


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5906515024674816

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 26 by perezju@chromium.org, Dec 19 2016

Cc: simonhatch@chromium.org sullivan@chromium.org
Again failing to reproduce regression both with and without story filter.


The last two bisects were trying to bisect the change here:
https://chromeperf.appspot.com/report?sid=b8d13296616cfa034950722d71124cf50795ea7be64f6a3e71c3c08863d4b611&start_rev=427712&end_rev=432435

Comment 27 by simonhatch@chromium.org, Dec 19 2016

Blockedon: 675639
Project Member

Comment 30 by 42576172...@developer.gserviceaccount.com, Apr 11 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : memory.top_10_mobile
  Metric       : memory:chrome:browser_process:reported_by_os:gpu_memory:graphics:proportional_resident_size_avg/foreground/https_mobile_twitter_com_justinbieber_skip_interstitial_true

Revision             Result               N
chromium@431022      35225600 +- 0.0      21      good
chromium@431075      35225600 +- 0.0      21      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=https.mobile.twitter.com.justinbieber.skip.interstitial.true memory.top_10_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982651685472614256

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5222967758290944


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 31 by 42576172...@developer.gserviceaccount.com, Apr 11 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : memory.top_10_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:private_dirty_size_avg/background/after_http_m_youtube_com_results_q_science

Revision             Result                   N
chromium@430986      40315434 +- 1724057      27      good
chromium@431022      40196802 +- 1478075      27      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http.m.youtube.com.results.q.science memory.top_10_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982632322743753664

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5262700895010816


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 33 by 42576172...@developer.gserviceaccount.com, Apr 11 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : memory.top_10_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:private_dirty_size_avg/background/after_http_m_youtube_com_results_q_science

Revision             Result                   N
chromium@430688      44101199 +- 5287783      27      good
chromium@431230      44113183 +- 5476117      27      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.top_10_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982611649209393040

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5717949644537856


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 34 by benhenry@chromium.org, Jul 27 2017

Status: WontFix (was: Untriaged)
This alert was found before M-60 branched. Closing as WontFix as this is believed to either be invalid or non-reproducible.
►

Sign in to add a comment