New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 729773 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Last visit > 30 days ago
Closed: Jul 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

2.4%-200.1% regression in loading.desktop at 476260:476539

Project Member Reported by m...@chromium.org, Jun 5 2017

Issue description

See the link to graphs below.
 

Comment 1 by m...@chromium.org, Jun 5 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=729773

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxt2GxQgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDght3VgAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgpq_orAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg-oO8mgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDghr22pQgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgps_KsQsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxqKT-QoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxretiwkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgpvSfuQkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxpvF_QkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxu-OogkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgpsje5AoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxsunzAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgps-0oAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDghoj6kAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDghpXaqQgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgpsq1vAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgptu2uAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxpvFgwgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxpv05AgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxsHZzwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDghtXd-goM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxuewiAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDghofH9wkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgpsjepAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxq6SrggM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxuCwpAgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxpOYwAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxqKT-QgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDghu6P5QsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxq2gzwkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxq2qgAgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxv6ewQgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxueVjAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgpsjepAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDghtXd-ggM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgps_eqAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxqGcqAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDghv3-xgsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgppqH5gkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxuCwxAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgxuDajAkM


Bot(s) for this bug's original alert(s):

chromium-rel-mac11
chromium-rel-mac11-pro
chromium-rel-mac12-mini-8gb
chromium-rel-win10
chromium-rel-win7-gpu-ati
chromium-rel-win7-gpu-intel
chromium-rel-win7-gpu-nvidia
chromium-rel-win7-x64-dual
chromium-rel-win8-dual
linux-release
win-high-dpi
Cc: hamelphi@chromium.org
Owner: hamelphi@chromium.org

=== Auto-CCing suspected CL author hamelphi@chromium.org ===

Hi hamelphi@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Philippe Hamel
  Commit : 1bc3d8c1ce10761406e57b07f7a2dc1ad301221e
  Date   : Thu Jun 01 16:57:23 2017
  Subject: Add query only fieldtrial for TranslateRankerModel.

Bisect Details
  Configuration: mac_10_11_perf_bisect
  Benchmark    : loading.desktop
  Metric       : timeToFirstContentfulPaint_avg/pcv1-cold/FC2Blog
  Change       : 163.60% | 86.0142857156 -> 226.73164286

Revision             Result                  N
chromium@476262      86.0143 +- 23.0675      14      good
chromium@476304      88.4125 +- 5.45038      6       good
chromium@476325      86.593 +- 14.2798       14      good
chromium@476327      86.8777 +- 19.5087      14      good
chromium@476328      219.808 +- 389.63       21      bad       <--
chromium@476330      244.794 +- 272.396      14      bad
chromium@476335      273.283 +- 52.1143      6       bad
chromium@476345      272.085 +- 59.5984      6       bad
chromium@476428      226.732 +- 288.195      14      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=FC2Blog loading.desktop

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8977590273094428768

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6128007611154432


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Cc: hansberry@chromium.org
 Issue 729752  has been merged into this issue.
This is related to other regression bugs we have been seeing in link with the TranslateRanker. These seem to point to something caused by the TranslateUI. TranslateRanker will sometimes suppress the TranslateUI. So, this regression may be caused by additional TranslateUIs being triggered.

See https://docs.google.com/document/d/1eeTSJg18rJ4v_oxxSJU8MK20UywnEhocWk_WrWv8kQE/edit#heading=h.wt5942hvwhfp for more details.

It looks like disabling the Enforcement experiment of TranslateRankerModel exposes a regression. The culprit CL replaces the Enforcement experiment with the Query Experiment (only the first experiment in a field trial config is run by perf bots), which puts the feature in Ghost mode (i.e. it never actively suppress the translate UI). I am running perf bots on a few different patches to confirm this.

If I am correct, the regression we are seeing here is actually the real behavior in Beta and Stable, since TranslateRanker is not active yet. Thus, activating TranslateRanker would actually help these metrics get better.

Comment 6 by m...@chromium.org, Jun 6 2017

Cc: -m...@chromium.org
Cc: rogerm@chromium.org yyushkina@chromium.org
Cc: chcunningham@chromium.org
 Issue 730124  has been merged into this issue.
Status: Assigned (was: Untriaged)
Explictly assigning. A CL you landed tripped one of the speed metrics we measure in the lab. If this is the first time this has happened to one of your CLs, or if it's been a while, please read: https://chromium.googlesource.com/chromium/src/+/master/docs/speed/addressing_performance_regressions.md

We're looking for one of the following:
1. Justification via explanation
2. Plan to revert or fix
3. Angry rage throwing of equipment at my head

Just be aware that I'm trained in trumpet playing and First Aid and am not afraid to use it.

Note: This was a bulk edit message and not very personal.
The CL in question adds a field-trial configuration that is representative of the current state of prod, to "Query" for but not enforce translate ranker recommendations.

The previous config only had the aspirational configuration that augments the query functionality with "Enforcement" of the translate ranker recommendations.

It appears that Enforcement is a performance win.

The regression caught by the bot is the performance loss that prod users are currently experiencing by not having Enforcement enabled.

The difference between the two is that Enforcement will suppress the offer to translate this page when it thinks the user is unlikely to accept the offer.

I think this should be resolved as won't fix.

As a side note, we've not seen evidence of a perf impact of either config in UMA.

hamelphi is OOO until the beginning of August, but has kindly captured much of the investigations behind this in the following doc:

https://docs.google.com/a/google.com/document/d/1eeTSJg18rJ4v_oxxSJU8MK20UywnEhocWk_WrWv8kQE/edit?usp=sharing
(sorry, Googlers only)
Status: WontFix (was: Assigned)
I'm WontFix-ing per #10
Cc: lunalu@chromium.org
Owner: lunalu@chromium.org

=== Auto-CCing suspected CL author lunalu@chromium.org ===

Hi lunalu@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Luna Lu
  Commit : 47be21fa9c0e81c2fdb2ab57002e08152cde13a0
  Date   : Fri Jun 02 16:35:15 2017
  Subject: Replace UseCounter::Feature by WebFeature in workers

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_chrome:java_heap:effective_size_avg/browse_news/browse_news_qq
  Change       : 0.22% | 64700247.0 -> 64840424.6667

Revision             Result                   N
chromium@475000      64700247 +- 62792.4      6       good
chromium@476000      64740453 +- 51748.6      6       good
chromium@476500      64771217 +- 81400.0      9       good
chromium@476625      64774284 +- 198720       14      good
chromium@476688      64818046 +- 535836       14      good
chromium@476689      64938786 +- 244491       9       bad       <--
chromium@476690      65259391 +- 506077       6       bad
chromium@476692      65192027 +- 172473       6       bad
chromium@476696      65069842 +- 109956       9       bad
chromium@476704      65069543 +- 105055       9       bad
chromium@476719      64935799 +- 619291       21      bad
chromium@476750      64841126 +- 317701       14      bad
chromium@477000      64840425 +- 37820.6      6       bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.news.qq system_health.memory_mobile

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8972188322299649360


For feedback, file a bug with component Speed>Bisection
lunalu: it's fine to ignore this, the metric that regressed is pretty noisy and it doesn't look related to your cl. sorry for the noise.

Sign in to add a comment