Issue 624881 link

Starred by 2 users

Issue metadata

Status:	Available
Owner:	----
Cc:	cmt...@chromium.org
Components:	Tools>ChromeOS-Toolchain
EstimatedDays:	----
NextAction:	----
OS:	Chrome
Pri:	2
Type:	Bug
Build-Toolchain

crosperf: Misleading report results when something fails....

Project Member Reported by cmt...@chromium.org, Jun 30 2016

Issue description

If the image_chromeos stage of crosperf fails, for example, instead of failing immediately and telling the user that crosperf could not image the machine, Crosperf will output the Failed to image machine message, but then will appear to keep going and will generate a report like this:

-------------------------------------------
Summary
-------------------------------------------
Benchmark: dromaeo.domcoremodify; Iterations: 3
This benchmark contains no result. Is the benchmark name valid?
Benchmark: smoothness.tough_webgl_cases; Iterations: 3
This benchmark contains no result. Is the benchmark name valid?
Benchmark: page_cycler.typical_25; Iterations: 3
This benchmark contains no result. Is the benchmark name valid?
Benchmark: kraken; Iterations: 3
This benchmark contains no result. Is the benchmark name valid?
Benchmark: dromaeo.domcoreattr; Iterations: 3
This benchmark contains no result. Is the benchmark name valid?
Benchmark: graphics_WebGLAquarium; Iterations: 3
This benchmark contains no result. Is the benchmark name valid?
Benchmark: speedometer; Iterations: 3
This benchmark contains no result. Is the benchmark name valid?
Benchmark: octane; Iterations: 3
This benchmark contains no result. Is the benchmark name valid?

Number re-images: 0

-------------------------------------------
Benchmark Run Status
-------------------------------------------

Keys Status Failing Reason
vanilla_image: octane (1) PENDING
vanilla_image: octane (2) PENDING
vanilla_image: octane (3) PENDING
vanilla_image: kraken (1) PENDING
vanilla_image: kraken (2) PENDING
vanilla_image: kraken (3) PENDING
vanilla_image: speedometer (1) PENDING
vanilla_image: speedometer (2) PENDING
vanilla_image: speedometer (3) PENDING
vanilla_image: dromaeo.domcoreattr (1) PENDING
vanilla_image: dromaeo.domcoreattr (2) PENDING
vanilla_image: dromaeo.domcoreattr (3) PENDING
[snip]

This is very confusing and should be fixed.

Comment 1 by cmt...@chromium.org, Jun 30 2016

This behavior is largely due to the multi-threaded nature of the Crosperf tests.  In particular, crosperf has launched all of the benchmark run threads before discovering the machine isn't usable.  image_chromeos dies with a LogFatal, which does not raise an exception but calls sys.exit(1).  The benchmark threads that were launched then all detect that they are done (might be for any of several reasons), followed by detecting that they have no actual results (might be for many reasons), resulting in the less-than-clear report we got.

Probably the best way to fix this is to have a some kind of global stages/status variable somewhere that gets update as we move along, particularly with respect to downloading images & flashing machines.  When a benchmark run detects that it "finished" with no results, it can check the global status variable(s) to help give a better error message (or to NOT output any kind of report at all).

Comment 2 Deleted

Comment 3 Deleted

Comment 4 Deleted

Comment 5 Deleted

Comment 6 Deleted

Comment 7 Deleted

Comment 8 by yunlian@chromium.org, Feb 8 2018

I had the same error when I try to use crosperf yesterday.
It failed when it tries to call 

mount_gtp_image.sh

Crosperf runs this command outside chroot. I get the same error when I run the command line manually. Should we run this command inside chroot?

Comment 9 by vapier@chromium.org, Feb 8 2018

src/scripts/mount_gpt_image.sh should be safe to run outside the chroot

►