fwiw this page has 26226 elements and 29191 text nodes = 55417 nodes.

The stack is:

WTF::Partitions::fastFree(void *)
blink::WebNode::reset()
content::BlinkAXTreeSource::GetChildren(blink::WebAXObject,std::vector<blink::WebAXObject,std::allocator<blink::WebAXObject> > *)
ui::AXTreeSerializer<blink::WebAXObject,content::AXContentNodeData,content::AXContentTreeData>::WalkAllDescendants(blink::WebAXObject)
ui::AXTreeSerializer<blink::WebAXObject,content::AXContentNodeData,content::AXContentTreeData>::WalkAllDescendants(blink::WebAXObject)
ui::AXTreeSerializer<blink::WebAXObject,content::AXContentNodeData,content::AXContentTreeData>::SerializeChanges(blink::WebAXObject,ui::AXTreeUpdateBase<content::AXContentNodeData,content::AXContentTreeData> *)
content::RenderAccessibilityImpl::SendPendingAccessibilityEvents()
IPC::MessageT<AccessibilityMsg_Events_ACK_Meta,std::tuple<>,void>::Dispatch<content::RenderAccessibilityImpl,content::RenderAccessibilityImpl,void,void ( content::RenderAccessibilityImpl::*)(void)>(IPC::Message const *,content::RenderAccessibilityImpl *,content::RenderAccessibilityImpl *,void *,void ( content::RenderAccessibilityImpl::*)(void))
content::RenderAccessibilityImpl::OnMessageReceived(IPC::Message const &)

that AX code is doing thousands of string copies, vector resizes, etc. The code is also creating and destroying Oilpan Persistent handles (inside WebNode) very rapidly for every everything in the page, so we're benchmarking how fast we can create and destroy 55k Persistent handles.

https://cs.chromium.org/chromium/src/third_party/WebKit/public/platform/WebPrivatePtr.h?sq=package:chromium&rcl=1469589434&l=165

The bug with cs.chromium.org in particular is  http://crbug.com/459381  - I have improvements coming, but there really are an insane number of elements.

We shouldn't be enabling full accessibility support just because there's a touch screen, though.

@jochen, could you add a breakpoint in BrowserAccessibilityStateImpl::AddAccessibilityMode and tell me the stack trace?

Also, do you happen to have LastPass installed or any other software that might use UI automation to control other apps?

Full accessibility mode should not be enabled when not needed, but regardless, we should strive to make that mode as performant as well. There are developers with screen readers facing this issue. We already have a bug filed for improving the performance of accessibility mode 624951. You might want to merge into this one, or limit the scope of this bug to not turn accessibility mode on when not necessary.

I'll post the stacktrace tomorrow

Still compiling... will post the stack tomorrow

This is the stack:

0:000> dv /V
00000000`0014e590 @rsp+0x0030                  this = 0x00000000`48a84ab0
00000000`0014e598 @rsp+0x0038                  mode = AccessibilityModeComplete (0n3)
0:000> kF 30
 #   Memory  Child-SP          RetAddr           Call Site
00           00000000`0014e560 00000000`12e15d38 content!content::BrowserAccessibilityStateImpl::AddAccessibilityMode+0x2c [c:\src\blink\src\content\browser\accessibility\browser_accessibility_state_impl.cc @ 137]
01        30 00000000`0014e590 00000000`12e15e98 content!content::BrowserAccessibilityStateImpl::EnableAccessibility+0x18 [c:\src\blink\src\content\browser\accessibility\browser_accessibility_state_impl.cc @ 74]
02        30 00000000`0014e5c0 00000000`13c05283 content!content::BrowserAccessibilityStateImpl::OnScreenReaderDetected+0x38 [c:\src\blink\src\content\browser\accessibility\browser_accessibility_state_impl.cc @ 70]
03        30 00000000`0014e5f0 00000000`12e13568 content!content::LegacyRenderWidgetHostHWND::OnGetObject+0x43 [c:\src\blink\src\content\browser\renderer_host\legacy_render_widget_host_win.cc @ 168]
04        70 00000000`0014e660 00000000`12e11c52 content!content::LegacyRenderWidgetHostHWND::_ProcessWindowMessage+0x68 [c:\src\blink\src\content\browser\renderer_host\legacy_render_widget_host_win.h @ 76]
05        60 00000000`0014e6c0 00000000`12e12bab content!content::LegacyRenderWidgetHostHWND::ProcessWindowMessage+0x62 [c:\src\blink\src\content\browser\renderer_host\legacy_render_widget_host_win.h @ 75]
06        60 00000000`0014e720 00007fff`d5f31169 content!ATL::CWindowImplBaseT<ATL::CWindow,ATL::CWinTraits<1073741824,0> >::WindowProc+0xcb [c:\src\depot_tools\win_toolchain\vs_files\95ddda401ec5678f15eeed01d2bee08fcbc5ee97\vc\atlmfc\include\atlwin.h @ 3526]
07        c0 00000000`0014e7e0 00007fff`d5f30ee2 USER32!UserCallWinProcCheckWow+0x1f9
08        f0 00000000`0014e8d0 00007fff`d5f40bee USER32!DispatchClientMessage+0xa2
09        60 00000000`0014e930 00007fff`d8bc8ba4 USER32!_fnDWORD+0x3e
0a        60 00000000`0014e990 00007fff`d5f52134 ntdll!KiUserCallbackDispatcherContinue
0b        88 00000000`0014ea18 00007fff`d5f3c94f USER32!NtUserPeekMessage+0x14
0c         8 00000000`0014ea20 00000000`00812815 USER32!PeekMessageW+0x7f
0d        40 00000000`0014ea60 00000000`00810b03 base!base::MessagePumpForUI::ProcessNextWindowsMessage+0x55 [c:\src\blink\src\base\message_loop\message_pump_win.cc @ 446]
0e        80 00000000`0014eae0 00000000`0081310d base!base::MessagePumpForUI::DoRunLoop+0x13 [c:\src\blink\src\base\message_loop\message_pump_win.cc @ 258]
0f        50 00000000`0014eb30 00000000`008057d6 base!base::MessagePumpWin::Run+0x9d [c:\src\blink\src\base\message_loop\message_pump_win.cc @ 142]
10        70 00000000`0014eba0 00000000`008e72ed base!base::MessageLoop::RunHandler+0xf6 [c:\src\blink\src\base\message_loop\message_loop.cc @ 460]
*** WARNING: Unable to verify checksum for C:\src\blink\src\out\Debug\chrome.dll
11       180 00000000`0014ed20 00000000`087bc44b base!base::RunLoop::Run+0x3d [c:\src\blink\src\base\run_loop.cc @ 36]
12        60 00000000`0014ed80 00000000`130f8971 chrome_68c0000!ChromeBrowserMainParts::MainMessageLoopRun+0x16b [c:\src\blink\src\chrome\browser\chrome_browser_main.cc @ 1971]
13       180 00000000`0014ef00 00000000`130ffa58 content!content::BrowserMainLoop::RunMainMessageLoopParts+0x141 [c:\src\blink\src\content\browser\browser_main_loop.cc @ 961]
14        c0 00000000`0014efc0 00000000`130e8e4e content!content::BrowserMainRunnerImpl::Run+0x1a8 [c:\src\blink\src\content\browser\browser_main_runner.cc @ 156]
15       270 00000000`0014f230 00000000`15713224 content!content::BrowserMain+0xee [c:\src\blink\src\content\browser\browser_main.cc @ 46]
16        90 00000000`0014f2c0 00000000`15713068 content!content::RunNamedProcessTypeMain+0xd4 [c:\src\blink\src\content\app\content_main_runner.cc @ 418]
17       160 00000000`0014f420 00000000`157104e1 content!content::ContentMainRunnerImpl::Run+0x278 [c:\src\blink\src\content\app\content_main_runner.cc @ 785]
18       2e0 00000000`0014f700 00000000`06e77a71 content!content::ContentMain+0x81 [c:\src\blink\src\content\app\content_main.cc @ 20]
19        60 00000000`0014f760 00000001`400385a5 chrome_68c0000!ChromeMain+0x111 [c:\src\blink\src\chrome\app\chrome_main.cc @ 85]
1a       1b0 00000000`0014f910 00000001`40032734 chrome!MainDllLoader::Launch+0x455 [c:\src\blink\src\chrome\app\main_dll_loader_win.cc @ 185]
1b       1e0 00000000`0014faf0 00000001`401e659d chrome!wWinMain+0x2e4 [c:\src\blink\src\chrome\app\chrome_exe_main_win.cc @ 250]
1c       370 00000000`0014fe60 00000001`401e643e chrome!invoke_main+0x2d [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 118]
1d        40 00000000`0014fea0 00000001`401e62fe chrome!__scrt_common_main_seh+0x12e [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 255]
1e        60 00000000`0014ff00 00000001`401e65b9 chrome!__scrt_common_main+0xe [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 300]
1f        30 00000000`0014ff30 00007fff`d8608102 chrome!wWinMainCRTStartup+0x9 [f:\dd\vctools\crt\vcstartup\src\startup\exe_wwinmain.cpp @ 17]
20        30 00000000`0014ff60 00007fff`d8b7c5b4 KERNEL32!BaseThreadInitThunk+0x22
21        30 00000000`0014ff90 00000000`00000000 ntdll!RtlUserThreadStart+0x34

I checked in "Easy of access" that narator isn't enabled. Any idea how to figure out what screenreader I have?

How do you trigger this bug on regular Chrome? When I browse on my desktop or laptop the performance is fine (maybe not great, but fine) and I'd like to be able to reproduce the full AX slowdown.

Simply visit chrome://accessibility and enable "global accessibility mode".
Read more info here https://www.chromium.org/developers/design-documents/accessibility
Also, you may use the "--force-renderer-accessibility" command-line flag.

I grabbed an ETW trace of this happening on Windows. It's got a good sampling profiler so it gives a good overview of where CPU time is going. Handy if there's any uncertainty about that. It shows:

80% of the time is spent in chrome_child.dll!content::RenderAccessibilityImpl::SendPendingAccessibilityEvents, with most of that time spent in calls to chrome_child.dll!ui::AXTreeSerializer<blink::WebAXObject,content::AXContentNodeData,content::AXContentTreeData>::SerializeChanges, and most of that time spent in calls to chrome_child.dll!ui::AXTreeSerializer<blink::WebAXObject,content::AXContentNodeData,content::AXContentTreeData>::WalkAllDescendants.
A good chunk (~38% of the total) of that time is inside chrome_child.dll!blink::EventPath::initialize.

10% of time is spent in chrome_child.dll!blink::BaseArena::lazySweep

I don't know if this is all obvious already or not.

Why is it computing EventPaths? I don't see that code dispatching events when I look at it. The GC cost is because it's creating and discarding 50k Persistent handles.

The page load starts with 2.3 s of CPU time, mostly in JavaScript and in blink (layout?), spread out over 7.5 s of elapsed time.

After that the page is entirely CPU bound for as long as my trace lasted (only about 26 s).

The page spends ~3.4 s on this stack (many child nodes removed):
chrome_child.dll!scheduler::TaskQueueManager::DoWork
chrome_child.dll!scheduler::TaskQueueManager::ProcessTaskFromWorkQueue
chrome_child.dll!base::debug::TaskAnnotator::RunTask
chrome_child.dll!IPC::ChannelProxy::Context::OnDispatchMessage
chrome_child.dll!base::internal::InvokeHelper<1,void>::MakeItSo<void (cdecl media::VideoRendererImpl::*const & ptr64)(void) ptr64,base::WeakPtr<media::VideoRendererImpl> const & ptr64>
chrome_child.dll!content::RenderAccessibilityImpl::SendPendingAccessibilityEvents
chrome_child.dll!ui::AXTreeSerializer<blink::WebAXObject,content::AXContentNodeData,content::AXContentTreeData>::SerializeChanges

Then about 8.4 s are spent on this stack (again, many child nodes removed):
chrome_child.dll!scheduler::TaskQueueManager::DoWork
chrome_child.dll!scheduler::TaskQueueManager::ProcessTaskFromWorkQueue
chrome_child.dll!base::debug::TaskAnnotator::RunTask
chrome_child.dll!IPC::ChannelProxy::Context::OnDispatchMessage
chrome_child.dll!content::ChildThreadImpl::OnMessageReceived
chrome_child.dll!IPC::MessageRouter::RouteMessage
chrome_child.dll!content::RenderFrameImpl::OnMessageReceived
chrome_child.dll!content::RenderAccessibilityImpl::OnMessageReceived
chrome_child.dll!IPC::MessageT<AccessibilityMsg_Events_ACK_Meta,std::tuple<>,void>::Dispatch<content::RenderAccessibilityImpl,content::RenderAccessibilityImpl,void,void (cdecl content::RenderAccessibilityImpl::*)(void) ptr64>
chrome_child.dll!content::RenderAccessibilityImpl::SendPendingAccessibilityEvents
chrome_child.dll!ui::AXTreeSerializer<blink::WebAXObject,content::AXContentNodeData,content::AXContentTreeData>::SerializeChanges

Then 2.8 s of Javascript

Then 8.4 s are spent on this call stack - or more, because the trace ended while this was still happening:
chrome_child.dll!scheduler::TaskQueueManager::DoWork
chrome_child.dll!scheduler::TaskQueueManager::ProcessTaskFromWorkQueue
chrome_child.dll!base::debug::TaskAnnotator::RunTask
chrome_child.dll!IPC::ChannelProxy::Context::OnDispatchMessage
chrome_child.dll!content::ChildThreadImpl::OnMessageReceived
chrome_child.dll!IPC::MessageRouter::RouteMessage
chrome_child.dll!content::RenderFrameImpl::OnMessageReceived
chrome_child.dll!content::RenderAccessibilityImpl::OnMessageReceived
chrome_child.dll!IPC::MessageT<AccessibilityMsg_Events_ACK_Meta,std::tuple<>,void>::Dispatch<content::RenderAccessibilityImpl,content::RenderAccessibilityImpl,void,void (cdecl content::RenderAccessibilityImpl::*)(void) ptr64>
chrome_child.dll!content::RenderAccessibilityImpl::SendPendingAccessibilityEvents
chrome_child.dll!ui::AXTreeSerializer<blink::WebAXObject,content::AXContentNodeData,content::AXContentTreeData>::SerializeChanges

So, any single call stack will fail to understand what is going on because it is going through phases. I don't know if it is cycling back and forth between Javascript and AccessibilityMsg_Events_ACK_Meta, or ???

The way that accessibility works is by copying all information from Blink to the browser process when the page gets created, and from then on copying all the changes that are happening to the page. Information includes the hierarchy and type of all HTML nodes, certain HTML attributes and layout information. That's the only way to expose this information to assistive aids. It has to be in the browser process.
However, there might be opportunities for improving the amount of information passed from Blink to the browser.

If you move the code into blink you'd get at least a 15℅ improvement. It'd also open the opportunities for other improvements.

The accessibility code should expose information through native platform APIs. For example on Windows, Assistive software loads in-process DLLs into the browser process to gather accessibility information. Blink is locked down and cannot be directly accessed like this. Unless there is a method we are missing?

I did some profiling on that exact url today and saw some low-hanging fruit we missed before. We can freeze the computation of the root and focused object in the accessibility tree during serialization. This speeds up the initial load of this particular url by 50% and single-node updates by 400%.

https://codereview.chromium.org/2205083002

Once that change is applied, there's nothing nearly so obvious just from profiling.

I'm also working on  http://crbug.com/618120 , which will store relative bounding boxes instead of absolute bounding boxes for all accessible nodes, which helps immensely on cs.chromium.org because we won't need to recompute all of the bounding boxes when you scroll.

Elliot is right that there are more potential gains by moving what happens inside of Blink and what happens outside.

Some opportunities I see:

* Not creating a new WebAXObject each time when exploring the tree, perhaps we can have one WebAXObject per AXObject?
* Driving the serialization of one node from inside Blink via one serialize() call rather than calling a bunch of separate public APIs from outside
* Instead of a bunch of functions that test for a bunch of rare accessibility attributes, we should iterate over all of the attributes of each element once and call handlers if any accessibility attributes are found.

Let's not forget, though, that as much as we should make this faster, most users shouldn't have accessibility enabled at all. I filed  http://crbug.com/633733  to dig into jochen's issue. We post an EVENT_SYSTEM_ALERT event on a nonexistent window and check to see if anyone responds. In the past that's been a reliable way to detect assistive technology and filter out software that uses an occasional MSAA call but doesn't need Chrome to enable accessibility support.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/36ef7b3f562ce421927e6abc941e716db2eb9dc6

commit 36ef7b3f562ce421927e6abc941e716db2eb9dc6
Author: dmazzoni <dmazzoni@chromium.org>
Date: Wed Sep 07 22:05:01 2016

Optimize BlinkAXTreeSource by adding freeze/thaw for things like root, focus.

When profiling some sites that are slow to build an accessibility tree,
a lot of time was spent retrieving the document, root accessibility object,
or focused accessibility object. Introduce the concept of freezing
BlinkAXTreeSource, which precomputes these once, and then allows them to
be reused through the rest of a function scope.

This is safe because the accessibility tree does not change during processing
of an accessibility event.

For one cs.chromium.org page, this sped up initial load by 50% (1000 ms
to 650 ms) and single-node updates by 400% (90 ms to 23 ms).

BUG= 631923 , 638474 

Review-Url: https://codereview.chromium.org/2205083002
Cr-Commit-Position: refs/heads/master@{#417073}

[modify] https://crrev.com/36ef7b3f562ce421927e6abc941e716db2eb9dc6/content/renderer/accessibility/blink_ax_tree_source.cc
[modify] https://crrev.com/36ef7b3f562ce421927e6abc941e716db2eb9dc6/content/renderer/accessibility/blink_ax_tree_source.h
[modify] https://crrev.com/36ef7b3f562ce421927e6abc941e716db2eb9dc6/content/renderer/accessibility/render_accessibility_impl.cc

 Issue 459381  has been merged into this issue.

CLosing this issue during triage as freezing the tree source, at least in part, helps with the original cs.chromium.org repro. I have a screen reader running and the page loads fine.