New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 735071 link

Starred by 5 users

Issue metadata

Status: WontFix
Owner:
Closed: Sep 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

1.5%-299.3% regression in system_health.memory_mobile at 480313:480369

Project Member Reported by lanwei@chromium.org, Jun 20 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Jun 20 2017

Cc: yhirano@chromium.org
Owner: yhirano@chromium.org

=== Auto-CCing suspected CL author yhirano@chromium.org ===

Hi yhirano@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Yutaka Hirano
  Commit : 84047549ac37bab70d7306566c912e59131f3e1f
  Date   : Mon Jun 19 04:16:15 2017
  Subject: [Loading] Save one post-task when the data pipe is ready on arrival

Bisect Details
  Configuration: android_one_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_chrome:gpu:effective_size_avg/load_social/load_social_twitter
  Change       : 139.80% | 765666.666667 -> 1836088.0

Revision             Result                  N
chromium@480312      765667 +- 1716493       9       good
chromium@480330      951352 +- 2467411       14      good
chromium@480339      765667 +- 1716493       9       good
chromium@480341      459832 +- 0.0           6       good
chromium@480342      1737784 +- 1326193      14      bad       <--
chromium@480344      1737784 +- 1326193      14      bad
chromium@480348      1836088 +- 0.0          9       bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.social.twitter system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976252739121755584

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6368470431694848


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 4 by 42576172...@developer.gserviceaccount.com, Jun 20 2017

Cc: jarin@google.com jarin@chromium.org
 Issue 734927  has been merged into this issue.
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Jun 21 2017

Issue 735070 has been merged into this issue.
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Jun 21 2017

Issue 735069 has been merged into this issue.
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Jun 21 2017

 Issue 735677  has been merged into this issue.
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Jun 21 2017

 Issue 735681  has been merged into this issue.
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Jun 22 2017

 Issue 735657  has been merged into this issue.
Project Member

Comment 10 by 42576172...@developer.gserviceaccount.com, Jun 22 2017

Cc: kraynov@chromium.org
 Issue 735859  has been merged into this issue.
Project Member

Comment 11 by 42576172...@developer.gserviceaccount.com, Jun 22 2017

 Issue 735859  has been merged into this issue.
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Jun 22 2017

Cc: mvstanton@google.com mvstan...@chromium.org
 Issue 735888  has been merged into this issue.
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Jun 22 2017

 Issue 735894  has been merged into this issue.
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Jun 22 2017

 Issue 735911  has been merged into this issue.
Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, Jun 22 2017

 Issue 735911  has been merged into this issue.
Project Member

Comment 16 by 42576172...@developer.gserviceaccount.com, Jun 22 2017

Issue 735670 has been merged into this issue.
Project Member

Comment 17 by 42576172...@developer.gserviceaccount.com, Jun 22 2017

 Issue 735935  has been merged into this issue.
Project Member

Comment 18 by 42576172...@developer.gserviceaccount.com, Jun 22 2017

 Issue 735935  has been merged into this issue.
Project Member

Comment 19 by 42576172...@developer.gserviceaccount.com, Jun 22 2017

 Issue 735934  has been merged into this issue.
Project Member

Comment 20 by 42576172...@developer.gserviceaccount.com, Jun 22 2017

Issue 735672 has been merged into this issue.
Cc: tzik@chromium.org
The change changes the loading timing. It has both positive and negative impacts, but looking at https://chromeperf.appspot.com/group_report?rev=480342 I think there are more positive results than negative ones (Note: the list contains perf changes not caused by this change). I would like to keep the CL and close this issue as WontFix, what do you think?
Status: WontFix (was: Untriaged)
Closing the issue. See #21 for the reasoning.
Owner: benhenry@chromium.org
Status: Assigned (was: WontFix)
Ben, can you take a look at #21? It does seem there are a lot of positive memory changes with this CL in addition to the regressions. How should we determine whether to keep it?
Owner: yhirano@chromium.org
This is like that tupperware in the cupboard meme.

But - https://chromeperf.appspot.com/group_report?rev=480342&3651=y

Which groups correspond directly to this CL and which do not? 

+primiano because the largest standout seems to not be justified by the progressions also discovered by the lab around this CL. Without digging into these 100s of alerts, it looks like this CL caused a large regression (i.e. >10MiB), and the progressions don't balance that, so I disagree that we should wont fix this. This is large and requires investigation.
Project Member

Comment 25 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740444  has been merged into this issue.
Project Member

Comment 26 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740447  has been merged into this issue.
Project Member

Comment 27 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740442  has been merged into this issue.
Project Member

Comment 28 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740441  has been merged into this issue.
Project Member

Comment 29 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740443  has been merged into this issue.
Project Member

Comment 30 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740440  has been merged into this issue.
Project Member

Comment 31 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740445  has been merged into this issue.
The change is originally intended to improve the loading performance by removing one task hopping. It seems it affects the memory consumption in both positive and negative ways.


Project Member

Comment 33 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740538  has been merged into this issue.
Project Member

Comment 34 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740539  has been merged into this issue.
Project Member

Comment 35 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740532  has been merged into this issue.
Project Member

Comment 36 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740531  has been merged into this issue.
Project Member

Comment 37 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740536  has been merged into this issue.
Project Member

Comment 38 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740537  has been merged into this issue.
Project Member

Comment 39 by 42576172...@developer.gserviceaccount.com, Jul 10 2017

 Issue 740533  has been merged into this issue.
Project Member

Comment 40 by 42576172...@developer.gserviceaccount.com, Jul 11 2017

Cc: nainar@chromium.org
 Issue 740439  has been merged into this issue.
Cc: -nainar@chromium.org
Project Member

Comment 42 by 42576172...@developer.gserviceaccount.com, Jul 11 2017

 Issue 740535  has been merged into this issue.
This change improves https://chromeperf.appspot.com/report?sid=73d61ca98f66698e324e3f5053ead60b41ef2945bf04420ba150fce1d272518f that r475528 regressed (Issue 729008).
r475528 also changed <img> and possibly <body> onload timing (delayed them a little, especially for background tabs).

According to #32, probably this is another example of system_health.memory_mobile improvements/regressions caused by loading timing changes?
(Issue 729008 and  Issue 711044  (Comment #23) have similar discussions; This issue is not directly targeting onload events though)

Cc: hirosh...@chromium.org
Project Member

Comment 46 by 42576172...@developer.gserviceaccount.com, Jul 11 2017


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Yutaka Hirano
  Commit : 84047549ac37bab70d7306566c912e59131f3e1f
  Date   : Mon Jun 19 04:16:15 2017
  Subject: [Loading] Save one post-task when the data pipe is ready on arrival

Bisect Details
  Configuration: android_nexus6_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_chrome:cc:effective_size_avg/background_search/background_search_google
  Change       : 34.02% | 48418101.3333 -> 31946720.0

Revision             Result                   N
chromium@480312      48418101 +- 2477032      6      good
chromium@480329      48355296 +- 2409221      6      good
chromium@480338      48292491 +- 2329307      6      good
chromium@480340      48700725 +- 343999       6      good
chromium@480341      48229685 +- 2235993      6      good
chromium@480342      32380077 +- 1444796      5      bad       <--
chromium@480346      32668981 +- 1858868      6      bad
chromium@480379      31946720 +- 1547995      6      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=background.search.google system_health.memory_mobile

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8974339021372300752


For feedback, file a bug with component Speed>Bisection
Cc: -jarin@google.com -yhirano@chromium.org allan.je...@qt.io primiano@chromium.org
benhenry@, primiano@, do you have any suggestions? See #21, #24, #32, #44.
Cc: -allan.je...@qt.io
Cc: benhenry@google.com
benhenry@, primiano@, do you have any suggestions? See #21, #24, #32, #44.

I have no idea what to say https://chromeperf.appspot.com/group_report?rev=480342 has like ~500 changes, either positive or negative.

The TTFMP changes seem pretty much all green but the memory changes.
The memory changes seem more 50-50 red/green. I am not sure how one can tell by just looking at them that the memory impact of this is good, bad, or a no-op. Somebody should look into the details and figure out what's going on. 
I looked regressions closely at  issue 603396  and our conclusion there was memory consumption in system_health.memory_mobile was too flaky for loading timing changes. See https://docs.google.com/document/d/1wissv31WUjeHaSaf78uAH0FiFQfSU6sqv14Lcuo1toA/edit#heading=h.y7ian1czdfko.

It looks it's aligned with what hiroshige@ sees at issue 729008.
Status: WontFix (was: Assigned)
Marking as WontFix per #51.

Sign in to add a comment