Project: chromium Issues People Development process History Sign in
New issue
Advanced search Search tips
Starred by 1 user
Status: Fixed
Owner:
Closed: Jul 20
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocking:
issue 746591



Sign in to add a comment
Android boot test fails for strago boards
Project Member Reported by jrbarnette@chromium.org, Jul 19 Back to list
The wizpig and sentry paladins recently started failing with
complaints on two tests:

[Test-Logs]: cheets_StartAndroid.stress: FAIL: Android did not boot!
[Test-Logs]: cheets_MediaPlayerVideoHWDecodeUsed: retry_count: 2, FAIL: Android did not boot!

The first recorded CQ failure is this run:
    https://luci-milo.appspot.com/buildbot/chromeos/master-paladin/15439

That run may contain the changes that introduced the problem.  It's
also possible than an Android- or Chrome PFQ run introduced the problem.

 
Cc: tfiga@chromium.org
Checking the canaries, wizpig, sentry, and edgar are not yet exhibiting
the failure.  If the problem were introduced through either PFQ, I'd
have expected it to show up no later than 9759.0.0, but it didn't

The CLs in the failed CQ run landed in the 9760.0.0 run, so it's likely
that the bug made it in to the tree via the CQ.

Here's the relevant blamelist:
    chromiumos-overlay | tfiga | 488744 | fails:0(15) | eclass/arc-build: Enable multilib support
    chromiumos-overlay | tfiga | 488743 | fails:0(8) | chromeos-base/arc-build-nyc: Add support for multilib
    chromiumos-overlay | tfiga | 488747 | fails:0(8) | x11-libs/arc-libdrm: Enable support for multilib
    chromiumos-overlay | tfiga | 488748 | fails:0(8) | media-libs/arc-mesa: Add support for multilib
    chromiumos-overlay | tfiga | 488749 | fails:0(8) | media-libs/arc-cros-gralloc: Enable support for multilib
    chromiumos-overlay | tfiga | 485001 | fails:0(7) | profiles: Enable multilib infrastructure for amd64 chromeos profile
    chromiumos-overlay | tfiga | 488746 | fails:0(7) | x11-libs/arc-libpciaccess: Add support for multilib
    platform2 | kitching | 566755 | fails:1(2) | hammerd: fix interface number bug
    ec-private-cr52 | domrizzo | *405253 | fails:0(2) | CR52: Generate k for ECDSA sign via HMAC-DRBG
    platform2 | kitching | 571342 | fails:1 | hammerd: fix segmentation faults
    arc-camera | henryhsu | 575125 | common: Add include file for DISALLOW_COPY_AND_ASSIGN
    chromiumos-overlay | tfiga | 568338 | profiles: arch/arm: Sync up make.defaults with upstream Gentoo
    ec-private-cr52 | domrizzo | *413153 | CR52: Add miscellaneous tests; pop count and fatal FIPS error.
    ec-private-cr52 | mschilder | *413988 | CR52: aes MCT driver
    kernel | naveen.m-AT-intel.com | 532756 | BACKPORT: ASoC: Intel: Skylake: Fix module load when module size > DMA buffer...

+tfiga@ as the most likely owner of the root cause.

Components: Platform>ARC
I'll take a look in ~30 mins.
The build linked above is a successful run, and it seems to be green for ~15 hours now, am I missing something?

FWIW we are seeing ToT ARC failures in EGL startup which we think might also be related to these multilib changes
Cc: jhorwich@chromium.org
yes, it does appear it's the same issue we're running into because Surfaceflinger is continually crashing.  Looks like it passed the CQ because the failures that would have been caught on wizpig were exceptions but are experimental builders anyway
Owner: jhorwich@chromium.org
over to Josh who is attempting a minimal revert of the above blame list to fix
We're also seeing failures in the PFQ, with 3 failing builds each on cyan, caroline, reef, veyron_minnie, starting yesterday night. I'm not entirely sure if it's the same issue: https://bugs.chromium.org/p/chromium/issues/detail?id=746573


> We're also seeing failures in the PFQ, with 3 failing builds
> each on cyan, caroline, reef, veyron_minnie, starting yesterday
> night. I'm not entirely sure if it's the same issue:
> https://bugs.chromium.org/p/chromium/issues/detail?id=746573

The cyan and caroline failures don't match the symptom.  The reef
and veyron_minnie changes _do_match the symptom.  However:
  * The Chrome PFQ failures pre-date the CQ failures, so the timing
    is wrong.
  * The PFQ failures aren't showing up on the veyron_minnie or reef
    canary builders, or in the CQ.

So, the Chrome PFQ failures are almost certainly caused by a Chrome
bug.

> The build linked above is a successful run, and it seems to be green for ~15 hours now, am I missing something?

The CQ run passed because the strago builders are all still experimental,
so the paladin failures were ignored.

Blocking: 746591
Status: Started
Sent https://chromium-review.googlesource.com/q/topic:%22revert-arc-multilib%22+(status:open%20OR%20status:merged) to CQ to revert multilib-related patches until we properly sort out a fix.
Project Member Comment 14 by bugdroid1@chromium.org, Jul 20
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/04ffd34753ed1da48d22f7afe3a3b37822e979df

commit 04ffd34753ed1da48d22f7afe3a3b37822e979df
Author: Josh Horwich <jhorwich@chromium.org>
Date: Thu Jul 20 00:58:24 2017

Revert "chromeos-base/arc-build-nyc: Add support for multilib"

This reverts commit b4d8ff0fc902787624b8d9e864e95e177ad19b33.

Reason for revert:
Unable to start surfaceflinger - /vendor/lib64/egl now has the
libraries when trying to run 32-bit ABI Android.

BUG= chromium:746548 
TEST=Start x86 Android on a 64-bit Intel device
CQ-DEPEND=CL:578671

Change-Id: I07ed27e76613c6ad9141232402af6ed443636570
Reviewed-on: https://chromium-review.googlesource.com/578564
Tested-by: Josh Horwich <jhorwich@chromium.org>
Reviewed-by: Luis Hector Chavez <lhchavez@chromium.org>

[rename] https://crrev.com/04ffd34753ed1da48d22f7afe3a3b37822e979df/chromeos-base/arc-build-nyc/arc-build-nyc-0.0.1-r5.ebuild
[modify] https://crrev.com/04ffd34753ed1da48d22f7afe3a3b37822e979df/chromeos-base/arc-build-nyc/arc-build-nyc-0.0.1.ebuild

Project Member Comment 15 by bugdroid1@chromium.org, Jul 20
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/be32fc06ea0c4696723b0fba2f1c0f252871ecf8

commit be32fc06ea0c4696723b0fba2f1c0f252871ecf8
Author: Josh Horwich <jhorwich@chromium.org>
Date: Thu Jul 20 00:58:35 2017

Revert "profiles: arch/arm: Sync up make.defaults with upstream Gentoo"

This reverts commit b9a92df6b963f171dbab7b26dcce9d13804cccf5.

Reason for revert:
Unable to start surfaceflinger - /vendor/lib64/egl now has the
libraries when trying to run 32-bit ABI Android.

BUG= chromium:746548 
TEST=Start x86 Android on a 64-bit Intel device

Change-Id: Ie78fb2fec0b7570f7958d65d1bb6e284630d6ad0
Reviewed-on: https://chromium-review.googlesource.com/578565
Reviewed-by: Luis Hector Chavez <lhchavez@chromium.org>
Tested-by: Josh Horwich <jhorwich@chromium.org>

[modify] https://crrev.com/be32fc06ea0c4696723b0fba2f1c0f252871ecf8/profiles/arch/arm/make.defaults

Project Member Comment 16 by bugdroid1@chromium.org, Jul 20
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/517ce487ecf2a173c5b6dd33338a432f5ac77e34

commit 517ce487ecf2a173c5b6dd33338a432f5ac77e34
Author: Josh Horwich <jhorwich@chromium.org>
Date: Thu Jul 20 00:58:53 2017

Revert "profiles: Enable multilib infrastructure for amd64 chromeos profile"

This reverts commit 1b244c91bc938eeec08bf9f2c45be7ed9d826ad5.

Reason for revert:
Unable to start surfaceflinger - /vendor/lib64/egl now has the
libraries when trying to run 32-bit ABI Android.

BUG= chromium:746548 
TEST=Start x86 Android on a 64-bit Intel device

Change-Id: Ia5986fec4be43ae7db031f626d3ffa8407eec582
Reviewed-on: https://chromium-review.googlesource.com/578566
Reviewed-by: Luis Hector Chavez <lhchavez@chromium.org>
Tested-by: Josh Horwich <jhorwich@chromium.org>

[modify] https://crrev.com/517ce487ecf2a173c5b6dd33338a432f5ac77e34/profiles/targets/chromeos/make.defaults
[modify] https://crrev.com/517ce487ecf2a173c5b6dd33338a432f5ac77e34/profiles/default/linux/amd64/10.0/chromeos/use.force
[modify] https://crrev.com/517ce487ecf2a173c5b6dd33338a432f5ac77e34/profiles/default/linux/amd64/10.0/chromeos/make.defaults
[modify] https://crrev.com/517ce487ecf2a173c5b6dd33338a432f5ac77e34/profiles/targets/sdk/make.defaults
[modify] https://crrev.com/517ce487ecf2a173c5b6dd33338a432f5ac77e34/profiles/default/linux/amd64/10.0/chromeos/parent

Project Member Comment 17 by bugdroid1@chromium.org, Jul 20
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/afd7bc6c683725f551c77de24ac3b2ae9b537d9c

commit afd7bc6c683725f551c77de24ac3b2ae9b537d9c
Author: Josh Horwich <jhorwich@chromium.org>
Date: Thu Jul 20 00:59:03 2017

Revert "eclass/arc-build: Enable multilib support"

This reverts commit a9e93c34622bbd833e5e8c59afc853bf0ef6e632.

Reason for revert:
Unable to start surfaceflinger - /vendor/lib64/egl now has the
libraries when trying to run 32-bit ABI Android.

BUG= chromium:746548 
TEST=Start x86 Android on a 64-bit Intel device

Change-Id: I50ada5bd457f47b2252161637fa23c6af8ebb5a4
Reviewed-on: https://chromium-review.googlesource.com/578667
Reviewed-by: Luis Hector Chavez <lhchavez@chromium.org>
Tested-by: Josh Horwich <jhorwich@chromium.org>

[modify] https://crrev.com/afd7bc6c683725f551c77de24ac3b2ae9b537d9c/eclass/arc-build.eclass

Project Member Comment 18 by bugdroid1@chromium.org, Jul 20
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/8b925adfda43ea58e1c8931587e4c587e34f33a3

commit 8b925adfda43ea58e1c8931587e4c587e34f33a3
Author: Josh Horwich <jhorwich@chromium.org>
Date: Thu Jul 20 00:59:12 2017

Revert "x11-libs/arc-libdrm: Enable support for multilib"

This reverts commit d1159dc9b5bef3d9a55e076d5c199a09eaadbf0b.

Reason for revert:
Unable to start surfaceflinger - /vendor/lib64/egl now has the
libraries when trying to run 32-bit ABI Android.

BUG= chromium:746548 
TEST=Start x86 Android on a 64-bit Intel device

Change-Id: I300e0927f0b63ef75439fe3fff0625ff4bd2d24f
Reviewed-on: https://chromium-review.googlesource.com/578668
Reviewed-by: Luis Hector Chavez <lhchavez@chromium.org>
Tested-by: Josh Horwich <jhorwich@chromium.org>

[modify] https://crrev.com/8b925adfda43ea58e1c8931587e4c587e34f33a3/x11-libs/arc-libdrm/arc-libdrm-9999.ebuild

Project Member Comment 19 by bugdroid1@chromium.org, Jul 20
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/f6e49d6ace487dcb7beebb506c41f1d1070067d9

commit f6e49d6ace487dcb7beebb506c41f1d1070067d9
Author: Josh Horwich <jhorwich@chromium.org>
Date: Thu Jul 20 00:59:20 2017

Revert "media-libs/arc-mesa: Add support for multilib"

This reverts commit 502a21576d2ee14e88ade2afab275e2f03da35d8.

Reason for revert:
Unable to start surfaceflinger - /vendor/lib64/egl now has the
libraries when trying to run 32-bit ABI Android.

BUG= chromium:746548 
TEST=Start x86 Android on a 64-bit Intel device

Change-Id: I3e46de994549e266a8f057f11c01a5066063e673
Reviewed-on: https://chromium-review.googlesource.com/578669
Reviewed-by: Luis Hector Chavez <lhchavez@chromium.org>
Tested-by: Josh Horwich <jhorwich@chromium.org>

[modify] https://crrev.com/f6e49d6ace487dcb7beebb506c41f1d1070067d9/media-libs/arc-mesa/arc-mesa-9999.ebuild

Project Member Comment 20 by bugdroid1@chromium.org, Jul 20
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/9c3a15d8a2e0709db578a7779f78d1bbc0b2b2ae

commit 9c3a15d8a2e0709db578a7779f78d1bbc0b2b2ae
Author: Josh Horwich <jhorwich@chromium.org>
Date: Thu Jul 20 00:59:30 2017

Revert "media-libs/arc-cros-gralloc: Enable support for multilib"

This reverts commit ca06c13f221e182f9f23c312d906c8149a4e829f.

Reason for revert:
Unable to start surfaceflinger - /vendor/lib64/egl now has the
libraries when trying to run 32-bit ABI Android.

BUG= chromium:746548 
TEST=Start x86 Android on a 64-bit Intel device

Change-Id: Ib03aaa884ac49dfa9a188910dc61e7fa76e349a4
Reviewed-on: https://chromium-review.googlesource.com/578670
Reviewed-by: Luis Hector Chavez <lhchavez@chromium.org>
Tested-by: Josh Horwich <jhorwich@chromium.org>

[modify] https://crrev.com/9c3a15d8a2e0709db578a7779f78d1bbc0b2b2ae/media-libs/arc-cros-gralloc/arc-cros-gralloc-9999.ebuild

Project Member Comment 21 by bugdroid1@chromium.org, Jul 20
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/006cf3616ae609c0b632d15fd5feae78c9dbbf97

commit 006cf3616ae609c0b632d15fd5feae78c9dbbf97
Author: Josh Horwich <jhorwich@chromium.org>
Date: Thu Jul 20 00:59:49 2017

Revert "x11-libs/arc-libpciaccess: Add support for multilib"

This reverts commit b3ae6619f99498f81b21b59c374546f3f4a3e79b.

Reason for revert:
Unable to start surfaceflinger - /vendor/lib64/egl now has the
libraries when trying to run 32-bit ABI Android.

BUG= chromium:746548 
TEST=Start x86 Android on a 64-bit Intel device

Change-Id: I5f96c561461236df9d664c8cb8f580526196d0da
Reviewed-on: https://chromium-review.googlesource.com/578671
Tested-by: Josh Horwich <jhorwich@chromium.org>
Commit-Queue: Josh Horwich <jhorwich@chromium.org>
Reviewed-by: Luis Hector Chavez <lhchavez@chromium.org>

[rename] https://crrev.com/006cf3616ae609c0b632d15fd5feae78c9dbbf97/x11-libs/arc-libpciaccess/arc-libpciaccess-0.12.902-r4.ebuild

Sign in to add a comment