New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 681590 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Aug 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 675639



Sign in to add a comment

4.6% regression in memory.top_10_mobile_stress at 443389:443443

Project Member Reported by nzolghadr@chromium.org, Jan 16 2017

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=681590

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgoPCouwoM


Bot(s) for this bug's original alert(s):

android-nexus6
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Jan 16 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus6_perf_bisect
  Benchmark    : memory.top_10_mobile_stress
  Metric       : memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/background/after_http_en_m_wikipedia_org_wiki_Science

Revision             Result                 N
chromium@443389      6412163 +- 425971      21      good
chromium@443443      6413733 +- 489310      21      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http.en.m.wikipedia.org.wiki.Science memory.top_10_mobile_stress

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8990294572159565680

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5327210946232320


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Blockedon: 675639
Blocking on crbug.com/675639 for no memory.top_10_* repro
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Jan 25 2017

Cc: yangguo@chromium.org
Owner: yangguo@chromium.org

=== Auto-CCing suspected CL author yangguo@chromium.org ===

Hi yangguo@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : yangguo
  Commit : aa75904e3c2072f6b0d6ed92f75d1fb729d7cefa
  Date   : Thu Jan 12 14:18:45 2017
  Subject: [debugger] infrastructure for side-effect-free debug-evaluate.

Bisect Details
  Configuration: android_nexus6_perf_bisect
  Benchmark    : memory.top_10_mobile_stress
  Metric       : memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/background/after_http_en_m_wikipedia_org_wiki_Science
  Change       : 0.39% | 6410526.28571 -> 6435340.85714

Revision                           Result                  N
chromium@443216                    6410526 +- 376708       14      good
chromium@443330                    6356556 +- 439946       9       good
chromium@443387                    6323574 +- 533234       14      good
chromium@443394                    6400444 +- 331047       14      good
chromium@443398                    6339006 +- 737143       21      good
chromium@443400                    6305031 +- 355285       6       good
chromium@443400,v8@2bca05685f      6347977 +- 529641       14      good
chromium@443400,v8@9884fb91e1      6379218 +- 536292       14      good
chromium@443400,v8@e00eae9e89      6246603 +- 3522465      21      good
chromium@443400,v8@aa75904e3c      6482176 +- 395802       14      bad       <--
chromium@443401                    6418601 +- 477479       14      bad
chromium@443415                    6413446 +- 615376       21      bad
chromium@443443                    6435341 +- 369379       14      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.top_10_mobile_stress

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8989548036508830944

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5908329450176512


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
4.6% is well within flakiness range. Restarting bisect.
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Jan 25 2017


=== BISECT JOB RESULTS ===
Perf regression found but unable to narrow commit range

Build failures prevented the bisect from narrowing the range further.


Bisect Details
  Configuration: android_nexus6_perf_bisect
  Benchmark    : memory.top_10_mobile_stress
  Metric       : memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/background/after_http_en_m_wikipedia_org_wiki_Science
  Change       : 1.01% | 6398665.77778 -> 6463057.33333

Suspected Commit Range
  2 commits in range
  https://chromium.googlesource.com/chromium/src/+log/100c919f8fc51a06aee219a6898424f006cca92a..f5cb683f8e23e164b8ff7049d877fe266d4e8209


Revision             Result                  N
chromium@443388      6398666 +- 214805       9        good
chromium@443395      6405431 +- 279450       14       good
chromium@443399      6382469 +- 66359.0      14       good
chromium@443400      ---                     ---      build failure
chromium@443401      6446093 +- 211815       14       bad
chromium@443402      6433171 +- 13710.8      9        bad
chromium@443416      6447684 +- 93075.0      9        bad
chromium@443443      6463057 +- 167497       9        bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http.en.m.wikipedia.org.wiki.Science memory.top_10_mobile_stress

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8989496379715317696

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5278276337008640


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Status: Assigned (was: Untriaged)
This bug has an owner, but was in a state that our triage picked up. Marking as Assigned.
Status: WontFix (was: Assigned)
Repeated bisect resulted in something entirely different, which confirms my suspicion that the first bisect is wrong. The graph seems to have recovered anyways.
Cc: benhenry@chromium.org
Status: Assigned (was: WontFix)
Re #11: The difference between the bisects is:

# 3: Could not reproduce due to bug in bisect
# 6: Narrowed to your commit
# 9: Narrowed to the v8 roll with your commit in it, but the build of the v8 roll failed.

There are two graphs: the yandex one had some good runs, but didn't really recover, and the wikipedia one did not recover at all. All the bisects ran on the non-recovered wikipedia one. I ran the bisect on the yandex one.
Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2969
Failure reason: the build has failed due to infrastructure failure.

Status: WontFix (was: Assigned)
This benchmark has been removed.

Sign in to add a comment