Last good: http://cros-goldeneye/chromeos/healthmonitoring/buildDetails?buildbucketId=8925873401587522016

First failing: http://cros-goldeneye/chromeos/healthmonitoring/buildDetails?buildbucketId=8925843308638532576

Diff: http://crosland/log/11485.0.0..11486.0.0

There were some new tests, but nothing that I'd necessarily expect to cause a build failure. https://crrev.com/3c0e4666a5d1c44edd2f897ba1e11dd21c937971 added a dependency on dev-go/mdns, but I'm not sure how that would be related either.

Is kevin64 like kevin but with 64-bit userspace?

In all of the build logs that I looked at, there's no actual error message. Caroline, any ideas about why that could be?

I forgot to copy the actual errors:

tast-local-tests-cros-0.0.1-r338: # github.com/hashicorp/go.net/ipv6
tast-local-tests-cros-0.0.1-r338: /build/kevin64/usr/lib/gopath/src/github.com/hashicorp/go.net/ipv6/sockopt_rfc3493_unix.go:105:42: undefined: setsockopt
tast-local-tests-cros-0.0.1-r338: /build/kevin64/usr/lib/gopath/src/github.com/hashicorp/go.net/ipv6/sockopt_rfc3493_unix.go:114:42: undefined: setsockopt
tast-local-tests-cros-0.0.1-r338: /build/kevin64/usr/lib/gopath/src/github.com/hashicorp/go.net/ipv6/sockopt_rfc3542_unix.go:54:12: undefined: getsockopt
tast-local-tests-cros-0.0.1-r338: /build/kevin64/usr/lib/gopath/src/github.com/hashicorp/go.net/ipv6/sockopt_rfc3542_unix.go:75:12: undefined: getsockopt
tast-local-tests-cros-0.0.1-r338: /build/kevin64/usr/lib/gopath/src/github.com/hashicorp/go.net/ipv6/sockopt_rfc3542_unix.go:82:42: undefined: setsockopt
tast-local-tests-cros-0.0.1-r338: # github.com/hashicorp/go.net/ipv4
tast-local-tests-cros-0.0.1-r338: /build/kevin64/usr/lib/gopath/src/github.com/hashicorp/go.net/ipv4/sockopt_asmreqn_unix.go:20:12: undefined: getsockopt
tast-local-tests-cros-0.0.1-r338: /build/kevin64/usr/lib/gopath/src/github.com/hashicorp/go.net/ipv4/sockopt_asmreqn_unix.go:41:42: undefined: setsockopt
tast-local-tests-cros-0.0.1-r338: /build/kevin64/usr/lib/gopath/src/github.com/hashicorp/go.net/ipv4/sockopt_unix.go:29:12: undefined: getsockopt
tast-local-tests-cros-0.0.1-r338: /build/kevin64/usr/lib/gopath/src/github.com/hashicorp/go.net/ipv4/sockopt_unix.go:51:42: undefined: setsockopt

Ah, thanks (and sorry for missing the error). This is probably caused by https://crrev.com/c/1390794, then.

Will a kevin64-paladin builder be added soon?

kevin64-paladin is already in CQ but as experimental. Will try to make it a regular one once chrome pfq issues are resolved.

It looks like the library lacks aarch64 variant. I'll try resolving...

Sent two CLs:

https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/1400328
-> Fixes the build.

https://chromium-review.googlesource.com/c/chromiumos/platform/tast/+/1400327
-> Fix "tast run" on aarch64 boards.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast/+/e2029580751fce0df3818710f499102c50e5f99c

commit e2029580751fce0df3818710f499102c50e5f99c
Author: Shuhei Takahashi <nya@chromium.org>
Date: Tue Jan 08 19:43:59 2019

tast: Support 64bit ARM userland.

Formerly, we always assumed 32bit userland on 64bit CPUs. However
we are introducing 64bit userland on some boards (e.g. kevin64).
This difference is critical on selecting the Go toolchain to
build binaries since we do not support running 32bit binaries on
64bit userland.

This change upates the functions to detect the target architecture
to get the userland architecture, instead of the kernel architecture,
and corrects the toolchain selection.

To get the userland architecture, we run the "file" command against
an arbitrary binary on the target (/sbin/init for now), instead of
"uname -m". I'm skeptical if it's an optimal way to check the
architecture, but I could not find a better way.

BUG= chromium:918245 
TEST=tast run DUT platform.P2PClient  # w/ betty, kevin, kevin64

Change-Id: I8e036200a6cf820ff53d13252d7f0ca277f4cc78
Reviewed-on: https://chromium-review.googlesource.com/1400327
Commit-Ready: Shuhei Takahashi <nya@chromium.org>
Tested-by: Shuhei Takahashi <nya@chromium.org>
Reviewed-by: Hidehiko Abe <hidehiko@chromium.org>

[modify] https://crrev.com/e2029580751fce0df3818710f499102c50e5f99c/src/chromiumos/cmd/tast/build/config.go
[modify] https://crrev.com/e2029580751fce0df3818710f499102c50e5f99c/src/chromiumos/cmd/tast/run/config.go
[modify] https://crrev.com/e2029580751fce0df3818710f499102c50e5f99c/src/chromiumos/cmd/tast/build/build.go
[modify] https://crrev.com/e2029580751fce0df3818710f499102c50e5f99c/src/chromiumos/cmd/tast/run/local.go

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/ab157496004f057b1fa3e1d6cebe0ce836090da2

commit ab157496004f057b1fa3e1d6cebe0ce836090da2
Author: Shuhei Takahashi <nya@chromium.org>
Date: Tue Jan 08 19:44:00 2019

dev-go/mdns: Fix build on 64bit ARM boards.

Simply apply a patch to compile syscall_unix.go on 64bit ARM.

BUG= chromium:918245 
TEST=build_packages --board=kevin64
TEST=fast_build.sh && ~/go/bin/tast run kevin64 platform.P2P{Client,Server}

Change-Id: I670a3cd39bcb171baeb75dde7c62c209826f37a5
Reviewed-on: https://chromium-review.googlesource.com/1400328
Commit-Ready: Shuhei Takahashi <nya@chromium.org>
Tested-by: Shuhei Takahashi <nya@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Hidehiko Abe <hidehiko@chromium.org>

[add] https://crrev.com/ab157496004f057b1fa3e1d6cebe0ce836090da2/dev-go/mdns/files/gonet-arm64.patch
[modify] https://crrev.com/ab157496004f057b1fa3e1d6cebe0ce836090da2/dev-go/mdns/mdns-0.0.1.ebuild
[add] https://crrev.com/ab157496004f057b1fa3e1d6cebe0ce836090da2/dev-go/mdns/mdns-0.0.1-r1.ebuild

I believe tast-local-tests-cros build should pass now. Please feel free to reopen if it's not the case.