New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 650783 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Nov 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Perf try jobs on Android failing

Project Member Reported by csharrison@chromium.org, Sep 27 2016

Issue description

I'm having trouble getting perf jobs to give me actionable data.

Here is the log for a recent failure (timeout, it looks like):
https://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/1038
 
Cc: dtu@chromium.org robert...@chromium.org pras...@chromium.org
Yep, looks like it's hitting the 1 hour timeout. Prasad, Roberto, Dave, is there any way to extend this timeout?
These mac jobs are also giving invalid results:
https://codereview.chromium.org/2353013002/

gsutil failing?

These might (?) not be related, but since I can't get any perf job to actually work, I thought I might as well post it. 
This is because the bots are set with default timeout of 3600s on the master.tryserver.chromium.perf
https://cs.chromium.org/chromium/build/masters/master.tryserver.chromium.perf/master.cfg?l=81

"|timeout| refers to the maximum number of seconds a build should be allowed
to run without output. After no output for |timeout| seconds, the build is forcibly killed."
https://cs.chromium.org/chromium/build/scripts/master/factory/annotator_factory.py?q=factory_properties+timeout&sq=package:chromium&dr=C&l=41

We can change master.cfg to increase the timeout, what should be the ideal timeout we should set?
#2, Actual failure is due to job timeout.
https://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/820/steps/Performance%20Test%20%28Without%20Patch%29%201%20of%201/logs/stdio

Regarding gsutils error. Perf tryjob or bisect first check if there is any build exists for the revision by executing "gsutils ls" command if build archive doesn't exists then it post a build request to bisect builder.

Comment 5 by dtu@chromium.org, Sep 27 2016

Okay, it timed out, but why did it run for an hour without output? Looks like the test failed to run. What happened?
Android perf jobs worked for me yesterday, but when I tried running some overnight they blew up (along with windows jobs). Links to error output seem to include:
,4672:0@@@STEP_LOG_LINE@Failure Output@WARNING:root:Chrome build location for linux_x86_64 not found. Browser will be run without Flash.@@@
@@@STEP_LOG_LINE@Failure Output@@@@
@@@STEP_LOG_LINE@Failure Output@Traceback (most recent call last):@@@
@@@STEP_LOG_LINE@Failure Output@  <module> at /b/c/b/android_one_perf_bisect/src/tools/perf/run_benchmark:22@@@
@@@STEP_LOG_LINE@Failure Output@    sys.exit(main())@@@
@@@STEP_LOG_LINE@Failure Output@  main at /b/c/b/android_one_perf_bisect/src/tools/perf/run_benchmark:19@@@
@@@STEP_LOG_LINE@Failure Output@    return benchmark_runner.main(config, [trybot_command.Trybot])@@@
@@@STEP_LOG_LINE@Failure Output@  main at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py:468@@@
@@@STEP_LOG_LINE@Failure Output@    return command_instance.Run(options)@@@
@@@STEP_LOG_LINE@Failure Output@  Run at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py:245@@@
@@@STEP_LOG_LINE@Failure Output@    return min(255, self._benchmark().Run(args))@@@
@@@STEP_LOG_LINE@Failure Output@  Run at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/benchmark.py:91@@@
@@@STEP_LOG_LINE@Failure Output@    return story_runner.RunBenchmark(self, finder_options)@@@
@@@STEP_LOG_LINE@Failure Output@  RunBenchmark at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:325@@@
@@@STEP_LOG_LINE@Failure Output@    if (possible_browser.platform.GetOSName() == 'chromeos' and@@@
@@@STEP_LOG_LINE@Failure Output@AttributeError: 'NoneType' object has no attribute 'platform'@@@


See:
https://codereview.chromium.org/2289333003/
Android perf jobs worked for me yesterday, but when I tried running some overnight they blew up (along with windows jobs). Links to error output seem to include:
,4672:0@@@STEP_LOG_LINE@Failure Output@WARNING:root:Chrome build location for linux_x86_64 not found. Browser will be run without Flash.@@@
@@@STEP_LOG_LINE@Failure Output@@@@
@@@STEP_LOG_LINE@Failure Output@Traceback (most recent call last):@@@
@@@STEP_LOG_LINE@Failure Output@  <module> at /b/c/b/android_one_perf_bisect/src/tools/perf/run_benchmark:22@@@
@@@STEP_LOG_LINE@Failure Output@    sys.exit(main())@@@
@@@STEP_LOG_LINE@Failure Output@  main at /b/c/b/android_one_perf_bisect/src/tools/perf/run_benchmark:19@@@
@@@STEP_LOG_LINE@Failure Output@    return benchmark_runner.main(config, [trybot_command.Trybot])@@@
@@@STEP_LOG_LINE@Failure Output@  main at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py:468@@@
@@@STEP_LOG_LINE@Failure Output@    return command_instance.Run(options)@@@
@@@STEP_LOG_LINE@Failure Output@  Run at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py:245@@@
@@@STEP_LOG_LINE@Failure Output@    return min(255, self._benchmark().Run(args))@@@
@@@STEP_LOG_LINE@Failure Output@  Run at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/benchmark.py:91@@@
@@@STEP_LOG_LINE@Failure Output@    return story_runner.RunBenchmark(self, finder_options)@@@
@@@STEP_LOG_LINE@Failure Output@  RunBenchmark at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:325@@@
@@@STEP_LOG_LINE@Failure Output@    if (possible_browser.platform.GetOSName() == 'chromeos' and@@@
@@@STEP_LOG_LINE@Failure Output@AttributeError: 'NoneType' object has no attribute 'platform'@@@


See:
https://codereview.chromium.org/2289333003/

Comment 8 by benhenry@google.com, Nov 22 2016

Status: WontFix (was: Untriaged)
This is old and unlikely we will find a fix anytime soon. This bug cannot be a catch-all for all failing android perf try jobs. Please open actionable bugs if you see failures in the future.

Sign in to add a comment