New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Performance Regression: 5-20% WebGLAquarium 1000 Fishes time across most boards

Project Member Reported by mshe...@chromium.org, Mar 11 2016 
Overview of Regression:
Across most boards there has been a 5-10% (occasionally 20%) regression in WebGLAquarium (graphics_WebGLAquarium/avg_fps_1000_fishes.avg_fps_1000_fishes) in 50-dev starting around 7933.0.0

Size of Regression:
The regression ranges in size from roughly 5-10%, for a select few devices (nyan) it surpasses 20%

Affected Device(s)/Platform(s):
This regression appears across all devices

Affected Build(s):
Chrome OS: builds >= 7933.0.0 for some devices and >= 7954.0.0 for others

Crosbolt Link: 
https://cros-goldeneye.corp.google.com/console/listCrosbolt?baselineMilestoneChannel=48%20%3A%20STABLE&comparisonMilestoneChannel=50%20%3A%20DEV&comparisonLimitResults=5&analysisType=BUILD_TO_BUILD&selectedDevices&graphSKU=gnawty_intel_celeron_n2940_4Gb&graphTest=graphics_WebGLAquarium%2Favg_fps_1000_fishes.avg_fps_1000_fishes

Good Graphs to View (click cell to view):
- WebGLAquarium 1000 Fishes / big
- WebGLAquarium 1000 Fishes / gnawty
- WebGLAquarium 1000 Fishes / heli
And any or all other red (or yellow) "Scroll Smoothness Percentage" cells

For a regression across all Chrome OS platforms, is there a similar regression on Chromeperf for OS X and/or Win systems:
N/a - This is a CrOS specific test

Notes:
The regression seems to show up on different boards at different times. Examples: Regression range(s): 7933-7934 (big, blaze, cyan, more), 7954 - 7955 (expresso, gnawty, heli, more). There's a chance that it is two separate regressions, but I figured we could start w/ one bug.

Comment 1 by mshe...@chromium.org, Mar 17 2016 

Hi Ilja, just wanted to check in here. Please let me know if I can help in any way. In case it is helpful, you can no click thru to the logs from the points on the Crosbolt graph.

Comment 2 by ihf@chromium.org, Apr 5 2016

Cc: ihf@chromium.org marc...@chromium.org
Components: -OS>Kernel>Graphics Blink>JavaScript>Runtime
Owner: danno@chromium.org
Daniel, can we get some help with this? (More details in crosbug.com/p/51083)

This is caused by 375485 and affects low end machines.
https://chromium.googlesource.com/v8/v8/+log/98d2ed87..aaf1b5c3

On celes (4 cores)
- temperatures are between 30'C idle and 40'C during the test (no thermal throttling)

- immediately before the CL (375484):
-- FPS counter is a bit jittery but stays during the whole test around 44fps.
-- autotest results are always above 43, usually around 44fps.
-- top shows
    %CPU %MEM     TIME+ COMMAND
   104.7  2.6   0:20.83 /opt/google/chrome/chrome --type=gpu-process
    91.1  3.5   0:18.49 /opt/google/chrome/chrome --type=renderer

- immediately after v8 roll landed (375485):
-- FPS counter is much smoother, but stays around 42-43fps most of the time
-- top shows renderer using much more CPU
    %CPU %MEM     TIME+ COMMAND
   106.7  3.6   0:21.64 /opt/google/chrome/chrome --type=renderer
   102.0  2.6   0:52.31 /opt/google/chrome/chrome --type=gpu-process
-- In addition (the test runs for 30s) after about 25s the counter drops to about 33fps for a little before recovering
-- during the drop top shows the renderer drowning the gpu-process
    %CPU %MEM     TIME+ COMMAND 
   125.5  3.7   0:34.37 /opt/google/chrome/chrome --type=renderer
    98.4  2.6   1:04.86 /opt/google/chrome/chrome --type=gpu-process

Comment 3 by danno@chromium.org, Apr 6 2016

Cc: jkummerow@chromium.org verwa...@chromium.org hablich@chromium.org
My guess is that that it's related to Jakob's change:

https://chromium.googlesource.com/v8/v8/+/5aa2cb3bccbdf339827b3768b1a8f80338b773c7

but adding a Toon and Michael H. for further triaging, too.

Comment 4 by hablich@chromium.org, Apr 7 2016

Cc: danno@chromium.org
Owner: jkummerow@chromium.org

Comment 5 by ihf@chromium.org, Apr 12 2016 

Hi Jakob, any chance to take a look at this?

Comment 6 by jkummerow@chromium.org, Apr 14 2016

Labels: -Pri-1 Pri-2
#5: Yes, although to be frank it doesn't have terribly high priority.

How can I run the benchmark locally in order to profile it?

#2 looks like you're mostly concerned about the GPU process. V8 doesn't run in the GPU process, so you may want to look for culprits elsewhere.

Comment 7 by marc...@chromium.org, Apr 14 2016

Cc: zelidrag@chromium.org puneetster@chromium.org
Labels: -Pri-2 Pri-1
It is actually quite high priority for us -- since these changes, V8 eats so much CPU that some of our WebGL graphics benchmarks have become CPU-limited.

Comment 8 by zelidrag@chromium.org, Apr 14 2016

Cc: josa...@chromium.org
Labels: -Type-Bug ReleaseBlock-Stable Type-Bug-Regression

Comment 9 by mshe...@chromium.org, Apr 14 2016

Cc: keta...@chromium.org

Comment 10 by ihf@chromium.org, Apr 14 2016 

I wasn't concerned about the gpu process (as it stayed constant), but about the 20-40% increase in the renderer process due to the v8 roll.

I think the easiest way to reproduce should be with a low end device of your choice pointed to http://webglsamples.org/aquarium/aquarium.html and selecting 1000-4000 fishes. Observe CPU usage before/after your change using "top". If that doesn't show the problem I'll give you detailed instructions how to get the numbers on a Chromebook using autotest.

Comment 11 by keta...@chromium.org, Apr 14 2016

Cc: krk@chromium.org

Comment 12 by hablich@chromium.org, Apr 20 2016 

@ChromeOS folks: when are you planning to do the next M50 Stable push?

Comment 13 by keta...@chromium.org, Apr 20 2016 

marcheu@ did you have any luck with the repro per Ihf@?

Comment 14 by marcheu@google.com, Apr 20 2016 

@13: The repro has been clear since the beginning and yes the regression is still visible today. At this point the bug is in the V8 team's camp. We need them to look at it and fix it, I'm not knowledgeable in V8 (I could revert V8 to before that roll but that's it).
Project Member

Comment 15 by bugdroid1@chromium.org, Apr 21 2016 

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/9bebebd909cf393bf7ea5512478a15aff70042d7

commit 9bebebd909cf393bf7ea5512478a15aff70042d7
Author: jkummerow <jkummerow@chromium.org>
Date: Thu Apr 21 13:18:00 2016

[ic] Restore PROPERTY key tracking in keyed ICs

Non-vectorized KeyedLoadICs used to remember whether they had seen Names
as keys; Crankshaft uses this information to avoid emitting elements
accesses which would always deopt. This CL restores that functionality
for vector ICs.

BUG= chromium:594183 
LOG=y
R=mvstanton@chromium.org

Review URL: https://codereview.chromium.org/1912593002

Cr-Commit-Position: refs/heads/master@{#35706}

[modify] https://crrev.com/9bebebd909cf393bf7ea5512478a15aff70042d7/src/ic/ic.cc
[modify] https://crrev.com/9bebebd909cf393bf7ea5512478a15aff70042d7/src/ic/ic.h
[modify] https://crrev.com/9bebebd909cf393bf7ea5512478a15aff70042d7/src/type-feedback-vector.cc
[modify] https://crrev.com/9bebebd909cf393bf7ea5512478a15aff70042d7/src/type-feedback-vector.h
[modify] https://crrev.com/9bebebd909cf393bf7ea5512478a15aff70042d7/src/type-info.cc
[add] https://crrev.com/9bebebd909cf393bf7ea5512478a15aff70042d7/test/mjsunit/regress/regress-crbug-594183.js

Comment 16 by jkummerow@chromium.org, Apr 26 2016

Labels: Merge-Request-50 Merge-Request-51
Status: Fixed (was: Assigned)
Goldeneye confirms that the performance lost in 375485 has been regained in recent M52-Dev builds. Requesting merge to M50 and M51.

Comment 17 by tin...@google.com, Apr 26 2016

Labels: -Merge-Request-50 Merge-Review-50 Hotlist-Merge-Review
[Automated comment] Request affecting a post-stable build (M50), manual review required.

Comment 18 by tin...@google.com, Apr 26 2016

Labels: -Merge-Request-51 Merge-Approved-51 Hotlist-Merge-Approved
Your change meets the bar and is auto-approved for M51 (branch: 2704)

Comment 19 by hablich@chromium.org, Apr 27 2016

Labels: -Merge-Review-50 Merge-Approved-50

Comment 22 by hablich@chromium.org, Apr 27 2016

Labels: -Merge-Approved-50 -Merge-Approved-51 merge-merged-50 Merge-merged-51

Comment 23 by patricia@chromium.org, Jul 1 2016

Labels: VerifyIn-53

Comment 24 by patricia@chromium.org, Aug 29 2016

Labels: VerifyIn-54

Comment 25 by ka...@chromium.org, Aug 31 2016

Labels: Bulk-Verified
Status: Verified (was: Fixed)
►

Sign in to add a comment