Chromedriver desktop bots are always success & green in spite of failure |
||||||
Issue descriptionIn spite of test failures, final result on desktop bot is always 'success' and green in color. For example: https://build.chromium.org/p/chromium.chromedriver/builders/Linux Failure build #31208 Success build #31173 Whereas, chromedriver Android bot shows the correct result. For example: https://build.chromium.org/p/chromium.fyi/builders/Android%20ChromeDriver%20Tests%20%28dbg%29?numbuilds=200 Failure build #36440 Success build #36390 Although test failures are displayed inside each build result page and is not blocking our test analysis, it would be nice to have the correct result on the outside builder result page too. Thanks.
,
Sep 26 2017
Unless the recipe is explicitly allowing this somewhere...
,
Sep 26 2017
Took a look at the recipe, and don't see anything obvious. @iannucci, you're listed in OWNERS, so I'll assign to you to take a look.
,
Oct 10 2017
,
Nov 15 2017
Going through old bugs. This recipe should be owned by someone on the chromedriver team, not me (or anyone else in chrome infra foundation).
This recipe actually does everything through an annotation script in build.git. This annotation script should probably move to src.git. The last step in the recipe is, essentially:
api.step("annotated_steps", ["python_slave", api.package_repo_resource("scripts",
"slave", "chromium", "chromedriver_buildbot_run.py"),
'--build-properties=%s' % api.json.dumps(build_properties,
separators=(',', ':')), '--factory-properties={"annotated_script":"chro'+\
'medriver_buildbot_run.py","blink_config":"chromium","gclient_env":'+\
'{"DEPOT_TOOLS_UPDATE":"0","LANDMINES_VERBOSE":"1"},"needs_'+\
'webdriver_java_tests":true}'], allow_subannotations=True)
The overall recipe will not fail unless that step returns something other than 0. My guess is that it returns 0. Someone who cares about these builders should rework this recipe to be not auto-generated.
,
Nov 16 2017
,
Dec 12 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f4aa69223bf74991cf848c7d00554f2e60b736a1 commit f4aa69223bf74991cf848c7d00554f2e60b736a1 Author: John Chen <johnchen@chromium.org> Date: Tue Dec 12 21:16:41 2017 [ChromeDriver] Build script report test failures Currently, the ChromeDriver water shows green status even when some tests have failed. Updating the build script to fix that. Bug: 768880 Change-Id: I5f3ccadd00a119390dd9fbd625172f987a1100eb Reviewed-on: https://chromium-review.googlesource.com/809726 Commit-Queue: John Chen <johnchen@chromium.org> Reviewed-by: Shuotao Gao <stgao@chromium.org> Cr-Commit-Position: refs/heads/master@{#523552} [modify] https://crrev.com/f4aa69223bf74991cf848c7d00554f2e60b736a1/chrome/test/chromedriver/run_buildbot_steps.py
,
Dec 13 2017
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by katthomas@google.com
, Sep 26 2017