New issue
Advanced search Search tips

Issue 629330 link

Starred by 2 users

Issue metadata

Status: Duplicate
Merged: issue 628120
Owner:
Closed: Jul 2016
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

42.4% regression in top_10_mobile_memory at 405107:405123

Project Member Reported by m...@chromium.org, Jul 18 2016

Issue description

See the link to graphs below.
 

Comment 1 by m...@chromium.org, Jul 18 2016

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=629330

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgiuPjsQkM


Bot(s) for this bug's original alert(s):

chromium-rel-win7-dual
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Jul 19 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@405106  10733791  673884   12  good
chromium@405123  10991914  1052867  17  bad

Bisect job ran on: win_perf_bisect
Bug ID: 629330

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests top_10_mobile_memory
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:effective_size_max/http___www.baidu.com_s?word_google
Relative Change: 0.73%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6710
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006755744806590480


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5285505223098368

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 6 by m...@chromium.org, Jul 19 2016

Re-running biscets on Android and Windows where graphs clearly showed the "sharp step" around the regression.
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Jul 20 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@405107  2658683  38456.1  12  good
chromium@405129  2673899  52639.5  18  bad

Bisect job ran on: android_one_perf_bisect
Bug ID: 629330

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests top_10_mobile_memory
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:allocated_objects_size_avg/http___www.baidu.com_s?word_google
Relative Change: 1.27%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_one_perf_bisect/builds/1425
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006667514455867904


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5339804481355776

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Jul 20 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@405106  10387234  1104773  15  good
chromium@405123  10204754  676807   12  bad

Bisect job ran on: win_perf_bisect
Bug ID: 629330

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests top_10_mobile_memory
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:effective_size_max/http___www.baidu.com_s?word_google
Relative Change: 2.10%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6717
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006667483453829168


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5870058325147648

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 9 by m...@chromium.org, Jul 20 2016

Owner: majidvp@chromium.org
Strange that these bisects continue to indicate no problems, when the graphs clearly DO. Assigning to current perf sheriff for further investigation...
Trying another bisect.
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Jul 30 2016

Mergedinto: 628120
Status: Duplicate (was: Assigned)

===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : [perf] Change google.com to google.co.uk in top_10_mobile_memory and dual_browser page sets.
Author  : mythria
Commit description:
  
In the top10_mobile page set google.com gets redirected to google.ca. This
sometimes causes problems in telemetry with scrolling. As a short term fix,
replace google.com with google.co.uk in the page set. We cannot change it
to google.ca because memory_top_10_mobile uses the same page set but with
a different wpr that has a recording of google.co.uk. Updated dual_browser
page set to be consistent with top_10_mobile page set.

Also recorded new wprs for memory_top_10_mobile, dual_browser and
top_10_mobile_memory.

BUG= chromium:617914 
LOG=N
CQ_INCLUDE_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq

Review-Url: https://codereview.chromium.org/2138133003
Cr-Commit-Position: refs/heads/master@{#405099}
Commit  : c5acd3851a2f34de15dd89119f0be48c22a9aa72
Date    : Wed Jul 13 09:04:39 2016


===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N  Good?
chromium@404000  7450624   0.0      5  good
chromium@404600  7446528   0.0      5  good
chromium@404899  7446528   0.0      5  good
chromium@405050  7446528   0.0      5  good
chromium@405088  7446528   0.0      5  good
chromium@405097  7446528   0.0      6  good
chromium@405098  7446528   0.0      5  good
chromium@405099  11218780  1336776  5  bad    <--
chromium@405100  10554661  1142358  6  bad
chromium@405102  10723433  869980   4  bad
chromium@405106  10904335  826059   5  bad
chromium@405125  10917398  1236692  4  bad
chromium@405200  10810067  532190   6  bad

Bisect job ran on: win_perf_bisect
Bug ID: 629330

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests top_10_mobile_memory
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:effective_size_max/http___www.baidu.com_s?word_google
Relative Change: 46.52%
Score: 99.0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6763
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005787198686957424


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5331468679643136

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Sign in to add a comment