New issue
Advanced search Search tips

Issue 661623 link

Starred by 5 users

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 3
Type: ----


Sign in to add a comment

Many tests failing in x64 android builds

Project Member Reported by thakis@chromium.org, Nov 2 2016

Issue description

We just got a fancy new x64 chrome/android bot, the first of its kind. Many of the tests it runs fail: https://build.chromium.org/p/chromium.fyi/builders/ClangToTAndroid%20x64/builds/126

This might be due to bot config issues, but the issues I looked at feel like code issues.

If you know anything about any of the tests, either file a sub bug for that test and mark this one blocked on the sub bug, or comment here.
 
Blockedon: -656046 -534815
Blocking: 656046 534815
Should pass (and do pass on other platforms):
android_webview_test_apk
chrome_public_test_apk
content_shell_test_apk
base_unittests
components_unittests
content_unittests
webkit_unit_tests

Unknown / not running on non-FYI bots AFAIK:
display_unittests
gfx_unittests
jingle_unittests
media_blink_unittests
remoting_unittests
url_unittests
wtf_unittests

Should not pass / should not be running:
blink_platform_unittests ( bug 590888 ,  bug 584508 )
browser_tests (bug 611756)

Comment 3 by boliu@chromium.org, Nov 2 2016

This bot is arm debug, and probably has the most complete list of passing tests:
https://build.chromium.org/p/chromium.linux/builders/Android%20Tests%20(dbg)

I worked on instrumentation tests recently, and can comment on those. The instrumentations tests (android_webview_test_apk, chrome_public_test_apk, chrome_sync_shell_test_apk, content_shell_test_apk) mostly looks ok. I think some of the failures are due to device-specific things, like maybe screen isn't on during testing, or just wrong assumptions made by tests. And of course there might be flakes.
Blockedon: 611756 584508 590888
Thanks! I'll remove blink_platform_unitttests and browser_tests for now then.

Several of the tests in the "Unknown" lists sound like they should run and pass on android -- are there bugs for getting them to pass and run everywhere?  (same for browser_tests and blink_platform_unitttests I suppose, but those at least have bugs)
I don't see any such bugs on a cursory search. Some of those suites appear to be surprisingly close to passing, while others (ahem remoting_unittests) may be less so.

At a glance, it looks like several of the suites that did pass aren't really run on Android on most bots. Those should likely be run more widely as well.
(Also, I'm not sure where the original list of tests came from, but it seems likely that it wasn't from an existing Android bot.)
Blockedon: 608541
Project Member

Comment 9 by bugdroid1@chromium.org, Nov 2 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d5db1ef9a94f0d7e58118c9ff4822df7b6f56317

commit d5db1ef9a94f0d7e58118c9ff4822df7b6f56317
Author: thakis <thakis@chromium.org>
Date: Wed Nov 02 18:00:28 2016

Don't run blink_platform_unittests, browser_tests on x64 Android bot.

They don't pass on Android yet.

BUG=661623,611756, 584508 , 590888 

Review-Url: https://codereview.chromium.org/2473463003
Cr-Commit-Position: refs/heads/master@{#429337}

[modify] https://crrev.com/d5db1ef9a94f0d7e58118c9ff4822df7b6f56317/testing/buildbot/chromium.fyi.json

Cc: bpastene@chromium.org
Re #3: It may very well be that our device provisioning isn't behaving 100% correctly on these devices. I managed to nab one for myself, so I can debug any weird screen/setup-wizard weirdness we might be seeing when I'm back at my desk next week.

Re #6: Yeah, I think I copy-pasted them from the wrong bot in one of the spec files. My bad.
I did a first pass scraping the logs. I ran out of time once I reached components unittests. Here's some partial analysis from my side:

- The failures in android_webview_test_apk are because of this test which checks whether "x-com.oem.test-keysystem" (whatever it is) is supported by the platform. Very unlikely this can be caused by a compiler bug as this calls into java code. Feels to me more like a feature that doesn't work/load on 64 bit.

- chrome_public_test_apk This seems an autofill failure: "Last suggestion should be off the screen after switching fields". Again this is all in java code-

  !!! mojo_test_apk:  this quite worries me as this is crashing in Mojo parsing validation. See stacktrace from the tombstone base_unittests: 
  - There are a bunch of logging-related failures that seem due to file access. Smells like a selinux / wrong path issue.

!!! FilePersistentMemoryAllocatorTest.AcceptableTest seems a red herring: [ERROR:persistent_memory_allocator.cc(678)] Corruption detected in shared-memory segment.

!!! CallbackListTest.ArityTest this one is a red herring. It is crashing but unfortunately I didn't find any trace in the tombstone, probably because of all the other failures/crashes.

!!! blink_platform_unittests this one also is suspicious
I   69.288s run_tests_on_device(16E7431D)  ASSERTION FAILED: PassRefPtr<blink::SimpleFontData> blink::FontCache::fontDataFromFontPlatformData(const blink::FontPlatformData *, blink::ShouldRetain, bool)
I   69.288s run_tests_on_device(16E7431D)  �� f(1) : ��K+f

- browser_tests: no idea why this failed, seems a device issue

- components_unittests: all these are failing to mmap the pak files. Smells more an instrumentation issue than a compiler issue.

Other crashes in the tombstone that look red herrings:
!!! Blink ComplexClipsTracking test: this is crashing in SkCoincidentSpans::correctOneEnd. Skia has a mixture of asm and very smart code.
!!! CacheStorageOperationTest.OperationSlow this ended up dereferencing a NP while invoking a callback

Blockedon: 662559
Blockedon: 663886
content_browsertests and content_unittests started passing after I flipped the build to do dcheck-always-on release builds instead of debug builds.

gfx_unittests just used to fail on android; i made that pass and it now runs on all android bots.
Is this still relevant?
Status: Archived (was: Untriaged)
Status: Untriaged (was: Archived)
Pretty sure this is still true. We made our bot (now at https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/ToTAndroid%20x64/1962) not run tests, but from what I understand we have 0 bot coverage on 64-bit intel. So I can't imagine that tests pass there.

Sign in to add a comment