first failure: https://build.chromium.org/p/chromium.android/builders/Marshmallow%20Tablet%20Tester?numbuilds=200

log snippet:

Traceback (most recent call last):
  File "/b/c/b/Marshmallow_Tablet_Tester/src/third_party/android_platform/development/scripts/stack", line 240, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/b/c/b/Marshmallow_Tablet_Tester/src/third_party/android_platform/development/scripts/stack", line 231, in main
    arch_defined)
  File "/b/c/b/Marshmallow_Tablet_Tester/src/third_party/android_platform/development/scripts/stack_core.py", line 187, in ConvertTrace
    ResolveCrashSymbol(list(useful_log), more_info)
  File "/b/c/b/Marshmallow_Tablet_Tester/src/third_party/android_platform/development/scripts/stack_core.py", line 342, in ResolveCrashSymbol
    symbol.TranslateLibPath(lib), code_addresses[lib], more_info)
  File "/b/c/b/Marshmallow_Tablet_Tester/src/third_party/android_platform/development/scripts/symbol.py", line 410, in SymbolInformationForSet
    addr_to_line = CallAddr2LineForSet(lib, unique_addrs)
  File "/b/c/b/Marshmallow_Tablet_Tester/src/third_party/android_platform/development/scripts/symbol.py", line 449, in __call__
    result = self.fn(lib, no_cache)
  File "/b/c/b/Marshmallow_Tablet_Tester/src/third_party/android_platform/development/scripts/symbol.py", line 506, in CallAddr2LineForSet
    symbolizer.Join()
  File "/b/c/b/Marshmallow_Tablet_Tester/src/third_party/android_platform/development/scripts/../../../../build/android/pylib/symbols/elf_symbolizer.py", line 177, in Join
    a2l.WaitForIdle()
  File "/b/c/b/Marshmallow_Tablet_Tester/src/third_party/android_platform/development/scripts/../../../../build/android/pylib/symbols/elf_symbolizer.py", line 252, in WaitForIdle
    self.WaitForNextSymbolInQueue()
  File "/b/c/b/Marshmallow_Tablet_Tester/src/third_party/android_platform/development/scripts/../../../../build/android/pylib/symbols/elf_symbolizer.py", line 270, in WaitForNextSymbolInQueue
    self._RestartAddr2LineProcess()
  File "/b/c/b/Marshmallow_Tablet_Tester/src/third_party/android_platform/development/scripts/../../../../build/android/pylib/symbols/elf_symbolizer.py", line 413, in _RestartAddr2LineProcess
    self._WriteToA2lStdin(addr)
  File "/b/c/b/Marshmallow_Tablet_Tester/src/third_party/android_platform/development/scripts/../../../../build/android/pylib/symbols/elf_symbolizer.py", line 324, in _WriteToA2lStdin
    self._proc.stdin.write('%s\n' % hex(addr))
IOError: [Errno 32] Broken pipe
step returned non-zero exit code: 1



John, could this be related to https://codereview.chromium.org/2851523002? It doesn't really look related, but it's still my best guess at the moment out of the blame list.

no, that failure wouldn't be related. I can look at this later today, though, after dealing with 720103.

 Issue 719975  has been merged into this issue.

 Issue 719973  has been merged into this issue.

Immediately prior to that logcat (below from K tablet, but also appears on M tablet):

/b/c/b/KitKat_Tablet_Tester/src/third_party/android_tools/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-addr2line: /b/c/b/KitKat_Tablet_Tester/src/out/Debug/sandbox_linux_unittests: File format not recognized

Given that, I may well have been wrong -- this might be that CL :|

 Issue 721430  has been merged into this issue.

Yep, definitely is. We're now trying to symbolize failures in the executable gtest binaries (sandbox_linux_unittests and breakpad_unittests) with a python wrapper script named "sandbox_linux_unittests" (e.g.) rather than with an actual arm binary named "sandbox_linux_unittests". Will upload a fix tomorrow.

Issue 721776 has been merged into this issue.

on review: https://chromium-review.googlesource.com/c/504909/

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/46858e064d251775e93c4f8fc7fe2da367ccaf36

commit 46858e064d251775e93c4f8fc7fe2da367ccaf36
Author: John Budorick <jbudorick@chromium.org>
Date: Fri May 12 22:20:50 2017

[android] Only allow ELF files as library candidates in symbolization.

With crrev.com/686094f6b082e2595bc5d429cdcdaa3b76217501, we started
writing a python script to <output dir>/<suite_name>. The stack
symbolization script was picking up that script as a possible library
for symbolization. This change prevents it from doing so.

Bug:  719984 
TBR=agrieve@chromium.org

Change-Id: Id0591ad05a531b7cf2037ced1fef36dd8d767bce
Reviewed-on: https://chromium-review.googlesource.com/504909
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Michael Case <mikecase@chromium.org>
Cr-Commit-Position: refs/heads/master@{#471463}
[modify] https://crrev.com/46858e064d251775e93c4f8fc7fe2da367ccaf36/build/android/pylib/symbols/elf_symbolizer.py
[modify] https://crrev.com/46858e064d251775e93c4f8fc7fe2da367ccaf36/third_party/android_platform/README.chromium
[modify] https://crrev.com/46858e064d251775e93c4f8fc7fe2da367ccaf36/third_party/android_platform/development/scripts/symbol.py