Perf try jobs on Android failing |
||
Issue descriptionI'm having trouble getting perf jobs to give me actionable data. Here is the log for a recent failure (timeout, it looks like): https://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/1038
,
Sep 27 2016
These mac jobs are also giving invalid results: https://codereview.chromium.org/2353013002/ gsutil failing? These might (?) not be related, but since I can't get any perf job to actually work, I thought I might as well post it.
,
Sep 27 2016
This is because the bots are set with default timeout of 3600s on the master.tryserver.chromium.perf https://cs.chromium.org/chromium/build/masters/master.tryserver.chromium.perf/master.cfg?l=81 "|timeout| refers to the maximum number of seconds a build should be allowed to run without output. After no output for |timeout| seconds, the build is forcibly killed." https://cs.chromium.org/chromium/build/scripts/master/factory/annotator_factory.py?q=factory_properties+timeout&sq=package:chromium&dr=C&l=41 We can change master.cfg to increase the timeout, what should be the ideal timeout we should set?
,
Sep 27 2016
#2, Actual failure is due to job timeout. https://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/820/steps/Performance%20Test%20%28Without%20Patch%29%201%20of%201/logs/stdio Regarding gsutils error. Perf tryjob or bisect first check if there is any build exists for the revision by executing "gsutils ls" command if build archive doesn't exists then it post a build request to bisect builder.
,
Sep 27 2016
Okay, it timed out, but why did it run for an hour without output? Looks like the test failed to run. What happened?
,
Oct 6 2016
Android perf jobs worked for me yesterday, but when I tried running some overnight they blew up (along with windows jobs). Links to error output seem to include: ,4672:0@@@STEP_LOG_LINE@Failure Output@WARNING:root:Chrome build location for linux_x86_64 not found. Browser will be run without Flash.@@@ @@@STEP_LOG_LINE@Failure Output@@@@ @@@STEP_LOG_LINE@Failure Output@Traceback (most recent call last):@@@ @@@STEP_LOG_LINE@Failure Output@ <module> at /b/c/b/android_one_perf_bisect/src/tools/perf/run_benchmark:22@@@ @@@STEP_LOG_LINE@Failure Output@ sys.exit(main())@@@ @@@STEP_LOG_LINE@Failure Output@ main at /b/c/b/android_one_perf_bisect/src/tools/perf/run_benchmark:19@@@ @@@STEP_LOG_LINE@Failure Output@ return benchmark_runner.main(config, [trybot_command.Trybot])@@@ @@@STEP_LOG_LINE@Failure Output@ main at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py:468@@@ @@@STEP_LOG_LINE@Failure Output@ return command_instance.Run(options)@@@ @@@STEP_LOG_LINE@Failure Output@ Run at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py:245@@@ @@@STEP_LOG_LINE@Failure Output@ return min(255, self._benchmark().Run(args))@@@ @@@STEP_LOG_LINE@Failure Output@ Run at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/benchmark.py:91@@@ @@@STEP_LOG_LINE@Failure Output@ return story_runner.RunBenchmark(self, finder_options)@@@ @@@STEP_LOG_LINE@Failure Output@ RunBenchmark at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:325@@@ @@@STEP_LOG_LINE@Failure Output@ if (possible_browser.platform.GetOSName() == 'chromeos' and@@@ @@@STEP_LOG_LINE@Failure Output@AttributeError: 'NoneType' object has no attribute 'platform'@@@ See: https://codereview.chromium.org/2289333003/
,
Oct 6 2016
Android perf jobs worked for me yesterday, but when I tried running some overnight they blew up (along with windows jobs). Links to error output seem to include: ,4672:0@@@STEP_LOG_LINE@Failure Output@WARNING:root:Chrome build location for linux_x86_64 not found. Browser will be run without Flash.@@@ @@@STEP_LOG_LINE@Failure Output@@@@ @@@STEP_LOG_LINE@Failure Output@Traceback (most recent call last):@@@ @@@STEP_LOG_LINE@Failure Output@ <module> at /b/c/b/android_one_perf_bisect/src/tools/perf/run_benchmark:22@@@ @@@STEP_LOG_LINE@Failure Output@ sys.exit(main())@@@ @@@STEP_LOG_LINE@Failure Output@ main at /b/c/b/android_one_perf_bisect/src/tools/perf/run_benchmark:19@@@ @@@STEP_LOG_LINE@Failure Output@ return benchmark_runner.main(config, [trybot_command.Trybot])@@@ @@@STEP_LOG_LINE@Failure Output@ main at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py:468@@@ @@@STEP_LOG_LINE@Failure Output@ return command_instance.Run(options)@@@ @@@STEP_LOG_LINE@Failure Output@ Run at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py:245@@@ @@@STEP_LOG_LINE@Failure Output@ return min(255, self._benchmark().Run(args))@@@ @@@STEP_LOG_LINE@Failure Output@ Run at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/benchmark.py:91@@@ @@@STEP_LOG_LINE@Failure Output@ return story_runner.RunBenchmark(self, finder_options)@@@ @@@STEP_LOG_LINE@Failure Output@ RunBenchmark at /b/c/b/android_one_perf_bisect/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:325@@@ @@@STEP_LOG_LINE@Failure Output@ if (possible_browser.platform.GetOSName() == 'chromeos' and@@@ @@@STEP_LOG_LINE@Failure Output@AttributeError: 'NoneType' object has no attribute 'platform'@@@ See: https://codereview.chromium.org/2289333003/
,
Nov 22 2016
This is old and unlikely we will find a fix anytime soon. This bug cannot be a catch-all for all failing android perf try jobs. Please open actionable bugs if you see failures in the future. |
||
►
Sign in to add a comment |
||
Comment 1 by sullivan@chromium.org
, Sep 27 2016