I had jank this morning and could managed to get trace.

Deleted: trace_jank.json.gz 12.1 MB

trace_jank.json.gz 12.1 MB Download

tons of mouse jank when scrolling on a google map, eventually causing it to crash:

https://feedback.corp.google.com/#/Report/21557360612

Description:
lots of mouse jank when scrolling on a map and then a crash.

UI language: en-US

Product Specific Data (whitelisted):
CHROME VERSION: 54.0.2824.5 dev
CHROMEOS_AUSERVER: <URL: 1>
CHROMEOS_RELEASE_BOARD: samus-signed-mpkeys
CHROMEOS_RELEASE_DESCRIPTION: 8688.3.0 (Official Build) dev-channel samus
CHROMEOS_RELEASE_TRACK: dev-channel
CHROMEOS_RELEASE_VERSION: 8688.3.0
ENTERPRISE_ENROLLED: Managed
expi:  3300110 3300132 3312390 3312577 3312791
hardware_class: SAMUS E25-H7R-W5L

Browser:
Chrome 54.0.2824.5

Not sure if this is related, In https://feedback.corp.google.com/#/Report/21557360612 
I see a lot of this error:
[2186:2186:0913/104044:ERROR:tab_manager_delegate_chromeos.cc(137)] NOTREACHED() hit.


Also
[1:20:0908/081434:WARNING:generic_decoder.cc(63)] Too many frames backed up in the decoder, dropping this one.
[1:772:0908/081434:WARNING:rtc_video_decoder.cc(631)] Too many pending buffers!

Something wrong with the renderer?

feels like I can repro pretty consistently when on a hangout video chat....

Thanks for the information. I wonder if the issue is only seen on a enterprise enrolled device?

Could you record a trace next time it happens? Thanks!

how do I do record a trace?

When it occurs, 
1. Open a new tab and type "about:tracing".
2. Press "Record". Choose the "Manually select settings" preset, click the All button above the left column and press "Record".
3. Switch back to the tab with the content you're debugging and do whatever it is that's slower than it should be (or behaving incorrectly). A few seconds of the bad activity is usually sufficient.
4. Switch back to the tracing tab and press "Stop", then "Save"

Detailed information can be found here:
https://www.chromium.org/developers/how-tos/trace-event-profiling-tool/recording-tracing-runs 

Thanks!

OK. getting lots of mouse jank on opening a sheets doc...did this tracing capture it?

Deleted: trace_mouse_jank.json.gz 13.2 MB

trace_mouse_jank.json.gz 13.2 MB Download

more jank

Deleted: trace_more_lag.json.gz 8.9 MB

trace_more_lag.json.gz 8.9 MB Download

and again

Deleted: trace_more_lag_2.json.gz 11.6 MB

trace_more_lag_2.json.gz 11.6 MB Download

for some reason, the tracing tool crashes if I let it go beyond 100%. also, sometimes it doesn't show graphs.... no idea why.

Deleted: trace_mouse_jank_3.json.gz 14.1 MB

trace_mouse_jank_3.json.gz 14.1 MB Download

I've found that restarting the computer makes the jank go away. After a few days of not restarting, the jank gets progressively worse until it's almost unusable.  hangouts VCs / opening big new tabs tend to make it really bad.

Originally reported on CBC:
https://productforums.google.com/forum/#!topic/chromebook-central/q1-2GK22bLU
OP states this has been an issue since the M53 update and is happening on four Dell Chromebook 11 (3120) models.

#CBC-RS/TC-watchlist

FWIW, here is my RAM right now...things are pretty laggy.

Deleted: Screenshot 2016-09-24 at 11.09.35 AM.png 1.1 MB

	Screenshot 2016-09-24 at 11.09.35 AM.png 1.1 MB View Download

I hit the issue again, but didn't get to enable the H option in top as requested.  Here is the default top view, though.  You can understand why it is laggy given these numbers. Load of 34! I tried to grab a trace, but it became unresponsive and I couldn't stop the trace and download the file before it crashed.  I think the trace exacerbated the problem but it was bad before I started tracing, too.  Any other ideas of how to debug this?

Deleted: IMG_20160928_215022-2.jpg 1.3 MB

	IMG_20160928_215022-2.jpg 1.3 MB View Download

afakhry@ could this be related to the V8 performance issue you were investigating with lots of allocated memory?

Having kswapd0 at the top might mean that a lot of GC maybe going on and touching blocks in various pages. 

Also looking at screenshot in #18, the machine seems to be under high memory pressure. The actions chrome is taking in response to that (e.g. tab discarding ...etc.) could also be triggering kswapd0.

Let me look at the traces.

Looking at trace in #13, the browser UI thread is straved. The hangouts renderer is doing a lot of V8 activities, but they're all starved as well. [Compare the Wall and CPU durations in the screenshots "UI_Thread" and "hangouts_renderer"].

Those V8 activities could be the trigger of whatever consuming the CPU. This brigs us to the Browser's IO thread. Take a look at the screenshot "IO_Thread". The IO Thread is flooded with the following IPC messages:

ExtensionHostMsg_RequestForIOThread
ResourceHostMsg_RequestResource

And their handling is not starved!

Adding random owners from the path leading to the first message being sent.

Deleted: UI_Thread.png 40.3 KB

	UI_Thread.png 40.3 KB View Download

Deleted: hangouts_renderer.png 125 KB

	hangouts_renderer.png 125 KB View Download

Deleted: IO_Thread.png 114 KB

	IO_Thread.png 114 KB View Download

+devline for ExtensionFunctionDispatcher and the IPC message in #21.

ExtensionHostMsg_RequestForIOThread is only used by the webRequest API for extensions.  erikchen@ recently found and fixed a regression in that API, so depending on what versions are here, that might be the culprit.

The reporter seems to be on:
CHROME VERSION: 54.0.2824.5 dev
CHROMEOS_AUSERVER: <URL: 1>
CHROMEOS_RELEASE_BOARD: samus-signed-mpkeys
CHROMEOS_RELEASE_DESCRIPTION: 8688.3.0 (Official Build) dev-channel samus
CHROMEOS_RELEASE_TRACK: dev-channel
CHROMEOS_RELEASE_VERSION: 8688.3.0

Hit this again and captured top with H option this time.  Here are two samples a bit apart.  The trace I took afterwards hung the machine so I couldn't get that.

Deleted: Screenshot 2016-09-29 at 11.42.55 PM.png 511 KB

	Screenshot 2016-09-29 at 11.42.55 PM.png 511 KB View Download

Deleted: Screenshot 2016-09-29 at 11.43.15 PM.png 577 KB

	Screenshot 2016-09-29 at 11.43.15 PM.png 577 KB View Download

FWIW. I find I hit the issue far more frequently after a number of hangouts VCs...

For me, I often see it after I've scrolled down a ways in my facebook feed + opened some links in new tabs and also when I've doing media related things (VC, long video playback, etc)

The machine is low on RAM. Could you also attach a screenshot of the task manager?

- shift-esc first to bring up the task manager
- alt-right then select JavaScript memory too

I noticed when the system is under memory pressure due to many apps and tabs using a lot of memory, kswapd0 climbs to the top with the top chrome function being:

blink::MajorGCWrapperVisitor::VisitPersistentHandle().

Note that nothing felt janky as kswapd0 in the experiments I'm running doesn't stay at top for long.

[See attached].

If you manage to repro again, take a screenshot of the task manager while showing Javascript memory as suggested by bccheng@.

Deleted: Screenshot 2016-09-30 at 1.02.05 PM.png 746 KB

	Screenshot 2016-09-30 at 1.02.05 PM.png 746 KB View Download

Deleted: perf_top.txt 27.4 KB

perf_top.txt 27.4 KB View Download

Thanks for the screenshot - although the system doesn't feel janky, the top task (kswapd0) and top method (isolate_lru_pages) are exactly where the CPU cycles are spent when the kernel is busy locating pages to swap. Since it is a blocking operation (no/slow forward progress until enough pages are all swapped) the system will feel janky when the system spends long time doing that. Once we know more details through task manager we will know who is using memory and evaluate whether it is normal.

I think the key to root cause this issue is to find out which chrome tab/site = uses all the memory. I just did a 1-hour hangout but the memory used by it is very normal (~300MB). Please attach a task manager screenshot when the problem is encountered.

Hi I have the Acer CXI2-4GKM chromebox (4GB RAM and I'm on the stable channel)- and I'm using it with a Logitech K-400 Plus wireless keyboard (w/ trackpad). The problem is that the two finger scroll on the trackpad is very choppy. By that I mean sometimes the web page feels like it's stuck in molasses and it's hard to get the page to start scrolling - and then all of a sudden the scrolling goes "hyper" going too far and too fast. So basically there is no fine control. I did notice that the two finger scroll works better on pages with minimal content - and worse on pages overloaded with content, videos, etc. The up/down arrows on the keyboard work pretty good and much better than the two finger scroll. (Although even the up/down arrows can be a bit choppy on pages crowded with a lot of content). Is this problem a function of the Chrome browser, Chrome OS, or the hardware such as the Logitech keyboard? I'm thinking the problem isn't with the keyboard - because I have a friend with a Chromebox and a Logitech wireless trackball and he says he also experiences a lack of fine control when scrolling web pages.

I tried an extension in the Chrome store called "Smooth Scroll" (and a few others) - but I really didn't notice any difference. 

This has migrated to https://b.corp.google.com/issues/32464369

Issue 663515 has been merged into this issue.

Have been constantly seeing this on my HP Chromebook 11 G1, ever since two or three system updates back. Not sure which version.

Over the last several days the mouse cursor on YouTube has gotten much worse. It's jumping around all over the place when a YouTube video is playing. And the two-finger scroll (especially on YouTube) hasn't gotten any better yet. So after all of this time (since the bug was reported) - not only have things not gotten better - they have gotten worse (because now the mouse cursor is very difficult to control when a YouTube video is playing). Earth to Google engineers - HELLOOOO anybody out there????

As I mentioned previously I have the Acer CXI2-4GKM chromebox (4GB RAM and I'm on the stable channel)- and I'm using it with a Logitech K-400 Plus wireless keyboard (w/ trackpad). I'm on the latest Chrome 55 update.

#39: this is being worked on, see comment #35.  There seems to be a kernel bug that prevents reclaim of a lot of available memory.

It's unclear that whether the problem sheikubooty runs into is the same as the samus problem at #35. The kernel reclamation problem leaves kswapd0 busy and couldn't make any progress, which often leads to system hang and crash, not only laggy cursor. 

I've summarized the samus problem in the mail to kernel mailing list linux-mm:
http://marc.info/?l=linux-mm&m=148415497306032

In short 3.14 kernel couldn't find page to reclaim in kernel anon inactive LRU list, so kswap0d keeps busy until system dies. However some page in anon active LRU list could be reclaimed. Moving more pages from active list helps. But I don't have a decent solution yet.

If update kernel to 4.4 the problem goes away.

 

what's the downside of the hack you proposed for 3.14?

The problem is I'm not sure how to evaluate the effect of the removal of line 2420
http://lxr.free-electrons.com/source/mm/vmscan.c#L2420

The if statement is there for some reason. I can only foresee swap becomes more aggressive after the removal.

What are next steps here?

Can we compare the codepaths around the questionable IF in 3.14 and 4.4?

Could we bisect in some way to figure out what change between 3.14 and 4.4 fixed the problem?

bisecting between 4.4 and 3.14 probably isn't feasible, I think we need to do some directed tab switch testing with and without that change to quantify the impact.

I was investigating jank on another platform and noticed the tab discarder didn't seem to be working properly, and I'm suspicious that is the case here as well.  Some fixes were merged in R56 

see  https://bugs.chromium.org/p/chromium/issues/detail?id=680350

although I was still seeing issues with Chrome tabs not getting thrown away when under memory pressure as noted in that bug

puneetster@ and zelidrag@ any suggestions for additional eyes we can get on this to help narrow it down?

Any further progress here?  If I'm understanding this correctly, any device on kernel 3.14 is going to have issues reclaiming all pages.  What can we do to further investigate?

There're many similar issues related to swap, but not the same cause.

Caroline (and maybe cave) are especially sluggish when swapping, the problem probably should be (or has been) traced elsewhere. They use 3.18 kernel. It doesn't seem lead to kernel panic or system crash.

On samus (v3.14), the problem is special because page could not be reclaimed, it usually leads to kernel panic. And it's unclear that why other 3.14 device doesn't seem to have the problem. I only see it on samus.

I *thought* this report is to trace the specific samus issue. Maybe I should change the issue title ?

(Bulk Edit) Adding the new conops Chrome OS hotlist to all open issues with the "#CBC-RS/TC-watchlist" tag, our former tracking tag.