Seems that random tests are flaking. Also I noticed that references are failing. Assigning to a benchmark owner.

The Android One is known,  issue 775242 .

On the failing builds of the other two builders, there are two separate problems:

- Error removing chrome-trace-config.json during atexit, filed  issue 789119 ;
- run_benchmark --upload-results returning exit code 255, this issue.

All the failing builds end with two lines similar to:

Running ['/usr/bin/python', '../../tools/perf/run_benchmark', 'system_health.memory_mobile', '-v', '--upload-results', '--browser=android-chromium', '--output-format=chartjson', '--output-dir', '/b/swarming/w/itqarWON/tmpZfFbAktelemetry', '--output-format=json-test-results'] in [...]
Command ['/usr/bin/python', '../../tools/perf/run_benchmark', 'system_health.memory_mobile', '-v', '--upload-results', '--browser=android-chromium', '--output-format=chartjson', '--output-dir', '/b/swarming/w/itqarWON/tmpZfFbAktelemetry', '--output-format=json-test-results'] returned exit code 255
https://logs.chromium.org/v/?s=chrome%2Fbb%2Fchromium.perf%2FAndroid_Nexus5X_Perf%2F854%2F%2B%2Frecipes%2Fsteps%2Fsystem_health.memory_mobile_on_Android%2F0%2Fstdout

While the successful builds return exit code 0.

Not sure what the failure is while running that command since it looks like we slurp its output entirely.

+ashleymarie could you investigate?

I don't think Ashley is the right person to look at this. Let put this on Telemetry queue

Raising Pri. As this probably means data is not getting through results dashboard and will make it harder for bot sheriffs to figure out system health failures.

Do you have the latest log? The old one has return code 255 due to the atexit problem, I think

I think the two are not related, see e.g.:
https://logs.chromium.org/v/?s=chrome%2Fbb%2Fchromium.perf%2FAndroid_Nexus5X_Perf%2F863%2F%2B%2Frecipes%2Fsteps%2Fsystem_health.memory_mobile_on_Android%2F0%2Fstdout

Where there was no atexit error, but the --upload-results script returned 255.