New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 725185 link

Starred by 1 user
Status: WontFix
Owner:
simonhatch@chromium.org
Closed: Sep 2017
Cc:
perezju@chromium.org
dewittj@chromium.org
paulir...@chromium.org
█ hjd@chromium.org
benhenry@google.com
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

4.7% regression in memory.long_running_idle_gmail_tbmv2 at 471957:472035

Project Member Reported by benhenry@google.com, May 22 2017 
See the link to graphs below.

Comment 1 by benhenry@google.com, May 22 2017 

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=725185

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg2uPb6QkM


Bot(s) for this bug's original alert(s):

chromium-rel-mac11
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, May 22 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: mac_10_11_perf_bisect
  Benchmark    : memory.long_running_idle_gmail_tbmv2
  Metric       : memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_max/memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_max

Revision             Result                      N
chromium@471956      464786374 +- 98510457       21      good
chromium@472035      454149130 +- 116888338      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.long_running_idle_gmail_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8978868307717459664

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5787696724705280


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, May 23 2017

Cc: paulir...@chromium.org
Owner: paulir...@chromium.org

=== Auto-CCing suspected CL author paulirish@chromium.org ===

Hi paulirish@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Paul Irish
  Commit : e23bdc0201d0327ae578615fda8674842348999a
  Date   : Tue May 16 06:53:03 2017
  Subject: DevTools: Roll Lighthouse to lighthouse@2.0.0-alpha.5

Bisect Details
  Configuration: mac_10_11_perf_bisect
  Benchmark    : memory.long_running_idle_gmail_tbmv2
  Metric       : memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_max/memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_max
  Change       : 10.67% | 446245096.667 -> 469934659.714

Revision             Result                      N
chromium@471956      446245097 +- 47439582       6       good
chromium@471996      448749845 +- 78586477       14      good
chromium@472015      440502388 +- 52711755       14      good
chromium@472026      439803770 +- 58493508       21      good
chromium@472031      438903979 +- 58938716       21      good
chromium@472032      445420757 +- 55501338       9       bad       <--
chromium@472033      491810464 +- 5131177        6       bad
chromium@472035      469934660 +- 104824024      14      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.long_running_idle_gmail_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8978850256561288176

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5787696724705280


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 6 by alexclarke@chromium.org, Jun 12 2017 

I find that bisect unlikely, lets try again.
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Jun 12 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: mac_10_11_perf_bisect
  Benchmark    : memory.long_running_idle_gmail_tbmv2
  Metric       : memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_max/memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_max

Revision             Result                      N
chromium@471956      445288305 +- 86004276       21      good
chromium@472035      449052740 +- 101571560      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.long_running_idle_gmail_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8977000605127250080

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4807069984620544


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 10 by 42576172...@developer.gserviceaccount.com, Jun 12 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: mac_10_11_perf_bisect
  Benchmark    : memory.long_running_idle_gmail_tbmv2
  Metric       : memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_max/memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_max

Revision             Result                      N
chromium@471956      444513174 +- 84484943       21      good
chromium@472035      449330695 +- 103706514      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.long_running_idle_gmail_tbmv2

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976981693832231120

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4807069984620544


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 11 by sheriffbot@chromium.org, Jul 14 2017

Labels: Hotlist-Google

Comment 12 by benhenry@chromium.org, Jul 27 2017

Status: Assigned (was: Untriaged)
This seems like an unlikely culprit, but worth calling out. Paul, what do you think?
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Aug 5 2017

Cc: dewittj@chromium.org
Owner: dewittj@chromium.org

=== Auto-CCing suspected CL author dewittj@chromium.org ===

Hi dewittj@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : dewittj
  Commit : 6c47dc6c68d89edb553cb66eedc823e011f46b68
  Date   : Mon May 15 18:28:44 2017
  Subject: [Offline Pages] Adds the GetPagesByNamespace method to OfflinePageBridge.

Bisect Details
  Configuration: mac_10_11_perf_bisect
  Benchmark    : memory.long_running_idle_gmail_tbmv2
  Metric       : memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_max/memory:chrome:renderer_processes:reported_by_os:system_memory:private_dirty_size_max
  Change       : 1.52% | 443526100.381 -> 450283630.286

Revision             Result                      N
chromium@470000      443526100 +- 100436692      21      good
chromium@471050      438199738 +- 43844806       14      good
chromium@471575      437600816 +- 43433280       9       good
chromium@471707      439283400 +- 44760563       9       good
chromium@471773      431168039 +- 9578462        6       good
chromium@471806      434938258 +- 9567765        6       good
chromium@471822      438304646 +- 45778074       9       good
chromium@471830      440132835 +- 42299786       9       good
chromium@471834      446027527 +- 56309378       9       good
chromium@471836      439332584 +- 44933354       9       good
chromium@471837      437802479 +- 50843281       9       good
chromium@471838      479835953 +- 6550684        9       bad       <--
chromium@472100      450283630 +- 89934648       14      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.long_running_idle_gmail_tbmv2

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8972189897167683200


For feedback, file a bug with component Speed>Bisection

Comment 15 by sullivan@chromium.org, Sep 18 2017

Cc: perezju@chromium.org hjd@chromium.org
Owner: u...@chromium.org
Assignign to test owner ulan and cc-ing perezju and hjd for metrics. The bisects are really all over the place. Should we keep investigating? What would be next steps?

Comment 16 by perezju@chromium.org, Sep 19 2017 

These looks like a real regression, but I'm these "long running" benchmarks are always going to be problematic to bisect. Even being conservative, when the metric is "stable" it still varies in a range of ~9MiB.

Ulan, maybe looking at some traces could help figure this one out?

Comment 17 by u...@chromium.org, Sep 29 2017

Status: WontFix (was: Assigned)
sullivan@, would it be possible to remove private_dirty_size_max from alerts? This metric looks too noisy.

I added this benchmark to track V8 memory usage and GC metrics.

Comment 18 by sullivan@chromium.org, Oct 6 2017

Owner: simonhatch@chromium.org
Simon, can you remove private_dirty_size_max from alerting config per #17? Would need to check v8 sheriffing rotations in addition to chromium perf sheriff.

Comment 19 by simonhatch@chromium.org, Oct 10 2017 

Done.
►

Sign in to add a comment