New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 18 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android , Windows
Pri: 2
Type: Bug
STS
Team-Accessibility

Blocked on:
issue 547648
issue 672205



Sign in to add a comment

Optimize accessibility support for tools like LastPass

Project Member Reported by dmazz...@chromium.org, Oct 29 2014 Back to list

Issue description

Accessibility support carries a performance cost if we need to constantly send updates about what's going on in a webpage to clients like screen readers.

However, other uses of accessibility APIs are tools like LastPass - on both Windows and Android (maybe Mac too?), LastPass uses accessibility APIs to find username and password fields and fill them in.

We should be able to identify usage patterns like LastPass's and optimize for those. In particular, tools like LastPass will occasionally scan the whole page looking for text fields, then interact with them - but when they're not doing that, they're essentially in the background and we ought to be able to disable accessibility support until triggered again.

This requires a lot more investigation to figure out the details. The pattern might end up being a lot different on Android and Windows.

This might be easy if LastPass only trigger based on a user action. If it's constantly running in the background looking for text fields this may be a lot harder, though I'm sure there are still optimizations we could do.

 
Enabling accessibility support for Last pass in Chrome for Android cripples performance even on a high end phone. The amount of logcat spam is insane.
 I have started using Firefox on Android which has no performance issues with Last pass.
Related issue with Accessibility APIs and performance on android:  Issue 547648 
Related to this bug: https://code.google.com/p/android/issues/detail?id=81950

If there was a way for password managers to be a provider for Smart Lock for Passwords, this would obviate using accessibility.

Cc: sabineb@chromium.org
Thanks for dropping by. Agree that dedicated API for smartlock would make the most sense. 

Sabine, as an FYI, we have large performance issues with the current APIs that lastpass/password managers use for integration on Android. Basically they hook into the system much deeper than necessary. Exclusive smartlock APIs for 3rd party would probably help here.
Cc: mkwst@chromium.org agektmr@chromium.org
It appears that the Credential Management API will likely be well-suited for LastPass to adopt.

AFAIK, it's behind a flag, but something you can start experimenting with.
https://w3c.github.io/webappsec-credential-management/

I believe agektmr has more docs he can share
It looks like that API is more appropriate for Chromium to implement, for use by web apps? I don't see how this applies to integrating the Chrome app with an external password manager app.
I'm not sure LastPass' Android adoption Paul mentioned is about native app or Chrome. If it is about Chrome, wonder how it works.
If there's any way to inject JS on Chrome for Android it may be possible for LastPass or 1Password to integrate with Credential Management API.  AFAIK the spec is written with third party integration in mind, but current implementation only allows Chrome itself to get/store credential information.

Developer friendly doc is on its way. Stay tuned.

Comment 8 by rdb@chromium.org, Dec 15 2015

Cc: rdb@chromium.org
Blockedon: android:81950
Blockedon: chromium:547648
Cc: dmazz...@chromium.org samccone@google.com sgu...@chromium.org
anatoly, thanks again for being involved here.

An engineer from RescueTime pointed out a few things they use to minimize the performance hit the page gets, over here: https://code.google.com/p/chromium/issues/detail?id=547648#c10

Certainly, they have different needs than a password manager, but using the notificationTimeout to throttle the events (to ~150ms) would definitely solve the cases where I notice these problem.

Attached is are a look at Chrome trace of some scroll jank caused by pulling the accessibility tree for LastPass. You can see the big pauses (~15ms) happening every 30ms or so.
I've also attached a screenrecord video so you can see it in action.

I was able to map that jank the AccessibilityMsg_SnapshotTree IPC call: https://code.google.com/p/chromium/codesearch#chromium/src/content/common/accessibility_messages.h&q=f:_messages.h%20f:access&sq=package:chromium&type=cs&l=189

IMO, we still have more work to do to between Chrome and Android to deliver Accessibility data faster. Can you tell us what events/APIs are most problematic?  I'm guessing that http://developer.android.com/reference/android/view/accessibility/AccessibilityRecord.html#getSource() is likely one of them.

However, it appears as though you can mitigate things on your side. I believe you can safely use notificationTimeout to reduce how frequently the thread is paused to collect this data.
http://developer.android.com/reference/android/accessibilityservice/AccessibilityServiceInfo.html#attr_android:notificationTimeout

Throttling would address the core perf issue here. Being more clever about event types would be a good second pass. And Chrome/Android sorting out the core issue here is the inevitability. :)

Does that make sense?
Screen Shot 2016-01-22 at 6.27.53 PM.png
375 KB View Download
Nexus_5X_digiday_headerjank_lastpass.mp4
3.7 MB Download
Screen Shot 2016-01-22 at 6.31.07 PM.png
152 KB View Download
Thanks for the info about notificationTimeout. Is there no default value set? I'm embarrassed to say that we missed this attribute in the documentation, and will definitely add it for the next release. We are already very selective about which events we process, so this would probably explain why scrolling still lags when we completely turned off our processing code.

Also, we had already made another change in the Jan 5th release: on new installs or reinstalls, our new default is not to scan apps for password fields, which means we could turn off the window content changed event, and only turn it on if the user requested it. This improved performance noticeably, and the notificationTimeout should hopefully take us most of the rest of the way.
Project Member

Comment 12 by sheriffbot@chromium.org, Jan 24 2017

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been available for more than 365 days, and should be re-evaluated. Please re-triage this issue.
The Hotlist-Recharge-Cold label is applied for tracking purposes, and should not be removed after re-triaging the issue.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Blockedon: -android:81950
Labels: -Hotlist-Recharge-Cold NewComponent-Accessibility-Internals NewComponent-Accessibility
Components: Internals>Accessibility
Components: -UI>Accessibility
Labels: -newcomponent-accessibility-internals -newcomponent-accessibility
 Issue 467123  has been merged into this issue.
Blockedon: 672205
Going to collect some stats to make progress on this, based in part on what we did with  bug 672205  on Windows.


Owner: paulmiller@chromium.org
passing to Paul.
Status: Assigned (was: Untriaged)
Project Member

Comment 20 by bugdroid1@chromium.org, Jun 7 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f33b08933cff6791612633f2e4a191917bb9d567

commit f33b08933cff6791612633f2e4a191917bb9d567
Author: dmazzoni <dmazzoni@chromium.org>
Date: Wed Jun 07 19:16:51 2017

Collect metrics on running Accessibility services on Android

We'd like to optimize accessibility by only enabling the features actually
needed by running accessibility services. In order to determine what would
be the best signals of what running accessibility services actually need,
add some code to collect metrics on running accessibility services that we
can analyze later.

Manually verified that TalkBack, Select-to-Speak, and LastPass all result in
very different sets of metrics being collected.

BUG=428494

Review-Url: https://codereview.chromium.org/2903583006
Cr-Commit-Position: refs/heads/master@{#477721}

[modify] https://crrev.com/f33b08933cff6791612633f2e4a191917bb9d567/content/browser/accessibility/browser_accessibility_manager_android.cc
[modify] https://crrev.com/f33b08933cff6791612633f2e4a191917bb9d567/content/browser/accessibility/browser_accessibility_manager_android.h
[modify] https://crrev.com/f33b08933cff6791612633f2e4a191917bb9d567/content/public/android/java/src/org/chromium/content/browser/accessibility/BrowserAccessibilityManager.java
[modify] https://crrev.com/f33b08933cff6791612633f2e4a191917bb9d567/content/public/android/java/src/org/chromium/content/browser/accessibility/KitKatBrowserAccessibilityManager.java
[modify] https://crrev.com/f33b08933cff6791612633f2e4a191917bb9d567/tools/metrics/histograms/enums.xml
[modify] https://crrev.com/f33b08933cff6791612633f2e4a191917bb9d567/tools/metrics/histograms/histograms.xml

Maybe a better option is to provide API for the applications to work as an extension.
Samsung already did something on that  http://developer.samsung.com/internet/android/extension-guide

But it should also support injecting code, permissions...
Cc: battre@chromium.org
Issue 733550 has been merged into this issue.
On dmazzoni's instructions, I measured the speed of SendPendingAccessibilityEvents, normally and then with some AccessiblityModes disabled. My test code is:

https://chromium-review.googlesource.com/c/568780/

I built chrome_apk (is_debug=false, is_component_build=false, target_cpu="arm") and installed it on a Nexus 6. With TalkBack enabled, I loaded https://en.wikipedia.org, which resulted in a few calls to SendPendingAccessibilityEvents. Then I scrolled around a bit, which resulted in many more calls to SendPendingAccessibilityEvents. I did this 9 times: 3 times with all AccessiblityModes enabled, 3 times with kInlineTextBoxes and kScreenReader disabled, and 3 times with kInlineTextBoxes, kScreenReader, and kHTML disabled. The results are here:

https://docs.google.com/a/google.com/spreadsheets/d/1fBFJa4VeMkEgFiJT0tFVZlDlvQ9m2s-jzRoCJAP2GKE/edit?usp=sharing

Each number is an individual log line from SendPendingAccessibilityEvents, showing a duration in microseconds. Contrary to our expectations, there's no difference when disabling AccessibilityModes. Maybe I've done something wrong? Or Maybe Wikipedia isn't demanding enough of a test page?
I did similar tests on http://www.cnn.com, which is a heavier page, but still saw no difference. While scrolling, each call takes ~10-20 milleseconds (which strikes me as quite a lot), whether or not AccessibilityModes are disabled.
I did a bunch of tracing experiments and it looks like the most expensive call inside SendPendingAccessibilityEvents is often AXTreeSerializer::WalkAllDescendants (by way of AXTreeSerializer::SerializeChanges). Inside WalkAllDescendants, I don't see any one operation that's particularly expensive; it's just slow because the tree is large. So that makes it hard to optimize. I wonder if we could avoid calling WalkAllDescendants on every SerializeChanges, but I'm not familiar with the problem it was put there to fix.

Attached is one of the traces. The new trace events I added are:
- AXTreeSerializer::LeastCommonAncestor
- AXTreeSerializer::SerializeChanges
- AXTreeSerializer::WalkAllDescendants (a single event around the initial call from SerializeChanges, to avoid the recursion)
- AXTreeSerializer::SerializeChangedNodes
trace_scroll-cnn-with-talkback.json.xz
776 KB Download
Labels: triage-dominic-internals
Labels: -triage-dominic-internals
Cc: -sgu...@chromium.org
Cc: changwan@chromium.org
Labels: STS
Labels: a11y-secondary

Sign in to add a comment