New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 714861 link

Starred by 1 user
Status: WontFix
Owner:
erikc...@chromium.org
Closed: Jul 2017
Cc:
erikc...@chromium.org
█ rsch...@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

28.4%-304.5% regression in memory.long_running_idle_gmail_background_tbmv2 at 466326:466522

Project Member Reported by rsch...@chromium.org, Apr 24 2017 
See the link to graphs below.

Comment 1 by rsch...@chromium.org, Apr 24 2017 

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=714861

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgoreonwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4v364QgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4oGx7AoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4pGsrwsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgoreonwsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4o-mpQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4vubpwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4pDH_gsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4oGx7AkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4oa_4AsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4q6ZgQgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4tTL7gkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4s3WiwsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4uHnpwkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4qva4wkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4palowkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgoq-LtgsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4qj1tQsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4uDw-AoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4ra0qAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgos_E6QgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4vSBsgkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg4rmSoAsM


Bot(s) for this bug's original alert(s):

chromium-rel-mac-retina
chromium-rel-mac11
chromium-rel-mac11-air
chromium-rel-mac11-pro
chromium-rel-mac12
chromium-rel-mac12-mini-8gb
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Apr 25 2017 


=== BISECT JOB RESULTS ===
Bisect failed for unknown reasons

Please contact the team (see below) and report the error.


Bisect Details
  Configuration: mac_pro_perf_bisect
  Benchmark    : memory.long_running_idle_gmail_background_tbmv2
  Metric       : memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_avg/memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_avg


To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.long_running_idle_gmail_background_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981388815294673856

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5802411634982912


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Apr 25 2017 


=== BISECT JOB RESULTS ===
Bisect failed for unknown reasons

Please contact the team (see below) and report the error.


Bisect Details
  Configuration: mac_pro_perf_bisect
  Benchmark    : memory.long_running_idle_gmail_background_tbmv2
  Metric       : memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_avg/memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_avg


To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.long_running_idle_gmail_background_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981380129896469424

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5802411634982912


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Apr 25 2017 


=== BISECT JOB RESULTS ===
Bisect failed for unknown reasons

Please contact the team (see below) and report the error.


Bisect Details
  Configuration: mac_pro_perf_bisect
  Benchmark    : memory.long_running_idle_gmail_tbmv2
  Metric       : memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_avg/memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_avg


To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.long_running_idle_gmail_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981324650859845152

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6360090690453504


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Apr 25 2017

Cc: erikc...@chromium.org
Owner: erikc...@chromium.org

=== Auto-CCing suspected CL author erikchen@chromium.org ===

Hi erikchen@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : erikchen
  Commit : 12531f29074985a71e18f60a2e8ea79b4f24314c
  Date   : Fri Apr 21 19:48:51 2017
  Subject: Use faster, but less information methods to calculate memory dumps on macOS.

Bisect Details
  Configuration: mac_pro_perf_bisect
  Benchmark    : memory.long_running_idle_gmail_tbmv2
  Metric       : memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_avg/memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_avg
  Change       : 55.08% | 245144950.363 -> 380167429.485

Revision             Result                     N
chromium@466325      245144950 +- 4048846       6      good
chromium@466378      244614180 +- 7592655       6      good
chromium@466405      245061164 +- 9444646       6      good
chromium@466412      245784724 +- 11605168      6      good
chromium@466415      245399569 +- 6956457       6      good
chromium@466416      246566688 +- 4319704       6      good
chromium@466417      382919017 +- 3857181       6      bad       <--
chromium@466418      382879792 +- 8353144       6      bad
chromium@466431      380167429 +- 5750739       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.long_running_idle_gmail_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981318917751071840

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6360090690453504


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 10 by erikc...@chromium.org, Apr 25 2017 

My CL is a change in how we conduct measurements, so we should expect changes in the corresponding stats.
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Apr 26 2017 


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : erikchen
  Commit : 12531f29074985a71e18f60a2e8ea79b4f24314c
  Date   : Fri Apr 21 19:48:51 2017
  Subject: Use faster, but less information methods to calculate memory dumps on macOS.

Bisect Details
  Configuration: mac_air_perf_bisect
  Benchmark    : memory.long_running_idle_gmail_background_tbmv2
  Metric       : memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_avg/memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_avg
  Change       : 121.83% | 254877808.079 -> 565392005.406

Revision             Result                     N
chromium@466347      254877808 +- 5952701       6      good
chromium@466382      258910291 +- 22673334      6      good
chromium@466400      257381194 +- 18311998      6      good
chromium@466409      261288016 +- 11711516      6      good
chromium@466413      257562817 +- 18003656      6      good
chromium@466415      253062027 +- 6463796       6      good
chromium@466416      256897935 +- 13709461      6      good
chromium@466417      567713801 +- 22129315      6      bad       <--
chromium@466487      565392005 +- 9701714       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.long_running_idle_gmail_background_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981285973137373200

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5802411634982912


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 13 by benhenry@chromium.org, Jul 27 2017

Status: Assigned (was: Untriaged)
Explictly assigning. A CL you landed tripped one of the speed metrics we measure in the lab. If this is the first time this has happened to one of your CLs, or if it's been a while, please read: https://chromium.googlesource.com/chromium/src/+/master/docs/speed/addressing_performance_regressions.md

We're looking for one of the following:
1. Justification via explanation
2. Plan to revert or fix
3. Angry rage throwing of equipment at my head

Just be aware that I'm trained in trumpet playing and First Aid and am not afraid to use it.

Note: This was a bulk edit message and not very personal.

Comment 14 by erikc...@chromium.org, Jul 27 2017

Status: WontFix (was: Assigned)
The CL in question changes the collection technique used the metrics. :)
►

Sign in to add a comment