New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 797065 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Aug 8
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

30.3%-63.3% regression in system_health.common_mobile at 524665:524708

Project Member Reported by briander...@chromium.org, Dec 21 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Dec 21 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=797065

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=13eb35172645076f527e658043e0f92fb4cd0cdff3bdfa0a178a97635e25e568


Bot(s) for this bug's original alert(s):

android-nexus5
android-nexus5X
android-nexus6
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Dec 22 2017

Cc: mlamouri@chromium.org
Owner: mlamouri@chromium.org
Status: Assigned (was: Untriaged)

=== Auto-CCing suspected CL author mlamouri@chromium.org ===

Hi mlamouri@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Mounir Lamouri
  Commit : 8a4ac91dd3927e5edb725d452775055a695fa62d
  Date   : Mon Dec 18 12:02:12 2017
  Subject: Disable autoplay restrictions on Chrome via command line flag.

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : system_health.common_mobile
  Metric       : cpu_time_percentage_avg/browse_media/browse_media_youtube
  Change       : 60.87% | 0.396146679282 -> 0.637279617807

Revision                                 Result                      N
chromium@524667                          0.396147 +- 0.00419356      6      good
chromium@524686                          0.395902 +- 0.00525491      6      good
chromium@524696                          0.397743 +- 0.0113865       6      good
chromium@524698                          0.397487 +- 0.00688662      6      good
chromium@524698,catapult@8a4ac91dd3      0.63852 +- 0.00984111       6      bad       <--
chromium@524699                          0.639057 +- 0.00855657      6      bad
chromium@524701                          0.63908 +- 0.0101859        6      bad
chromium@524705                          0.63728 +- 0.0107712        6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.media.youtube system_health.common_mobile

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8959561430472116464


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 4 by 42576172...@developer.gserviceaccount.com, Dec 23 2017

Issue 797097 has been merged into this issue.
Cc: perezju@chromium.org
Cc: nedngu...@google.com
+nednguyen

context: Mounir added --autoplay-policy=no-user-gesture-required to the chrome browser args, since some tests need autoplay to be enabled. This is now being restricted to desktop only [1], so other platforms where this is not needed can run the same as before.

I'm wondering now, however, whether instead of tying the auto-play policy to the platform, we should be able to decide at the test/benchmark level?

[1]: https://chromium-review.googlesource.com/c/catapult/+/847475
perezju@, I think this would be a good idea as it would allow most benchmarks to run closer to reality and the ones requiring actual playback to start to do so.

Would it make sense to still land the CL you linked in order to fix the Android regression and open a follow-up bug for the more granular solution?
Yep, I think it's fine to land the fix for android regressions and file a new bug for the follow up.
It seems that some catapult android tests are expecting some kind of autoplay. perezju@, would it make sense to add a user gesture in the PlayActionTest so autoplay works there?
Hmm, the failing tests are:

telemetry.internal.actions.play_unittest.PlayActionTest.testPlayWaitForEnded
telemetry.internal.actions.play_unittest.PlayActionTest.testPlayWithAllSelector
telemetry.internal.actions.play_unittest.PlayActionTest.testPlayWithNoSelector
telemetry.internal.actions.play_unittest.PlayActionTest.testPlayWithVideoSelector

It sounds like these tests were already broken in Android but they were just passing because of the autoplay?
Cc: sullivan@chromium.org rjwright@chromium.org catapult-deps-roller@chromium.org
 Issue 797093  has been merged into this issue.
Issue 797099 has been merged into this issue.

Comment 13 by dtu@chromium.org, Jan 25 2018

Cc: beccahughes@chromium.org
 Issue 797123  has been merged into this issue.
 Issue 797415  has been merged into this issue.
 Issue 797030  has been merged into this issue.
Components: Speed>Metrics>SystemHealthRegressions
friendly ping from today's perf sheriff
any update?
Status: WontFix (was: Assigned)
I think there is nothing else to do here. Auto-play is now enabled on all platforms [1], which is probably what we want in our tests.

[1]: https://cs.chromium.org/chromium/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome/chrome_startup_args.py?rcl=5e7ddfe4f2b219d3e95a675bf8e09aaa0773d935&l=55

Sign in to add a comment