Examples of the actual crash output:

[ RUN      ] ProcessUtilTest.SelectivelyClonedDir
[79137:1518475197:0324/015245.420610:816858234:ERROR:launch_fuchsia.cc(89)] fdio_transfer_fd failed: (UNKNOWN)
[79137:1518475197:0324/015245.434944:816872988:FATAL:scoped_file.cc(42)] Check failed: 0 == ret. : Bad file descriptor (9)
#00: base::debug::StackTrace::StackTrace(unsigned long) at ??:?
#01: logging::LogMessage::~LogMessage() at ??:?
#02: logging::ErrnoLogMessage::~ErrnoLogMessage() at ??:?
#03: base::internal::ScopedFDCloseTraits::Free(int) at ??:?
#04: base::LaunchProcess(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, base::LaunchOptions const&) at ??:?
#05: base::LaunchProcess(base::CommandLine const&, base::LaunchOptions const&) at ??:?
#06: base::SpawnMultiProcessTestChild(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, base::CommandLine const&, base::LaunchOptions const&) at ??:?
#07: base::MultiProcessTest::SpawnChildWithOptions(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, base::LaunchOptions const&) at ??:?
#08: base::ProcessUtilTest_SelectivelyClonedDir_Test::TestBody() at ??:?
#09: testing::Test::Run() at ??:?
#10: testing::TestInfo::Run() at ??:?
#11: testing::TestCase::Run() at ??:?
#12: testing::internal::UnitTestImpl::RunAllTests() at ??:?
#13: testing::UnitTest::Run() at ??:?
#14: base::TestSuite::Run() at ??:?
#15: base::(anonymous namespace)::LaunchUnitTestsInternal(base::OnceCallback<int ()>, unsigned long, int, bool, base::OnceCallback<void ()>) at ??:?
#16: base::LaunchUnitTests(int, char**, base::OnceCallback<int ()>) at ??:?
#17: main at ??:?
#18: pc 0x572209dda66e (libc.so,0x1366e)

[2937/2938] ProcessUtilTest.SelectivelyClonedDir (CRASHED)
[ RUN      ] ProcessUtilTest.CloneAlternateDir
[79510:1605400775:0324/015247.521638:818959266:ERROR:launch_fuchsia.cc(89)] fdio_transfer_fd failed: (UNKNOWN)
[79510:1605400775:0324/015247.524983:818962619:FATAL:scoped_file.cc(42)] Check failed: 0 == ret. : Bad file descriptor (9)
#00: base::debug::StackTrace::StackTrace(unsigned long) at ??:?
#01: logging::LogMessage::~LogMessage() at ??:?
#02: logging::ErrnoLogMessage::~ErrnoLogMessage() at ??:?
#03: base::internal::ScopedFDCloseTraits::Free(int) at ??:?
#04: base::LaunchProcess(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, base::LaunchOptions const&) at ??:?
#05: base::LaunchProcess(base::CommandLine const&, base::LaunchOptions const&) at ??:?
#06: base::SpawnMultiProcessTestChild(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, base::CommandLine const&, base::LaunchOptions const&) at ??:?
#07: base::MultiProcessTest::SpawnChildWithOptions(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, base::LaunchOptions const&) at ??:?
#08: base::ProcessUtilTest_CloneAlternateDir_Test::TestBody() at ??:?
#09: testing::Test::Run() at ??:?
#10: testing::TestInfo::Run() at ??:?
#11: testing::TestCase::Run() at ??:?
#12: testing::internal::UnitTestImpl::RunAllTests() at ??:?
#13: testing::UnitTest::Run() at ??:?
#14: base::TestSuite::Run() at ??:?
#15: base::(anonymous namespace)::LaunchUnitTestsInternal(base::OnceCallback<int ()>, unsigned long, int, bool, base::OnceCallback<void ()>) at ??:?
#16: base::LaunchUnitTests(int, char**, base::OnceCallback<int ()>) at ??:?
#17: main at ??:?
#18: pc 0x42529d4da66e (libc.so,0x1366e)

[2938/2938] ProcessUtilTest.CloneAlternateDir (CRASHED)

It looks like pkgfs namespace entries use a different fd transfer policy than farfs. Investigating.

Sorry, didn't mean to derail the bug. This bug should be focused on the result processing infra. 


Here's a bug which discusses the issues on comment #2: https://bugs.chromium.org/p/chromium/issues/detail?id=826018

It looks like the test output JSON is recording the test runs as CRASHED, and the outputs for the crashed tests are exposed as links in the test run summary page. Is there a policy somewhere that tells the trybot whether to handle crashes as failures?

https://isolateserver.appspot.com/browse?namespace=default-gzip&digest=945b3d6f19be5b3b1fcf75ad3e45fa3caa5420c8&as=output.json

That's not quite the issue here; the recipe-side code that parses that JSON is (correctly) picking out the individual test failures. The process is exiting w/ a 0, though, so the recipes think that it has passed despite the contained failures.

Ah... >_<

Thanks John, that's an easy fix: https://chromium-review.googlesource.com/c/chromium/src/+/981438

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c1022fce2628003b41923b7eef64341427c4c7ee

commit c1022fce2628003b41923b7eef64341427c4c7ee
Author: Kevin Marshall <kmarshall@chromium.org>
Date: Tue Mar 27 03:04:36 2018

Fuchsia: return exit code from executables.

The current implementation was masking error return values, making
all test and executable runs return 0. This lead to waterfall errors
being shown as green/passing.

Bug:  825714 
Change-Id: I977faf461fceb5b5cd2d3e8e22d0c0135c77ce71
Reviewed-on: https://chromium-review.googlesource.com/981438
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Kevin Marshall <kmarshall@chromium.org>
Cr-Commit-Position: refs/heads/master@{#545925}
[modify] https://crrev.com/c1022fce2628003b41923b7eef64341427c4c7ee/build/fuchsia/runner_v2/exe_runner.py
[modify] https://crrev.com/c1022fce2628003b41923b7eef64341427c4c7ee/build/fuchsia/runner_v2/test_runner.py

I see test failures on the FYI bot. Hurrah!  (and also: Booo!)