[rebaseline-cl] Layout tests are not necessarily run on tryserver.blink try bots for all changes |
|||
Issue descriptionFor changes that don't touch layout tests and only affect builds on some platforms, some try jobs may not run layout tests, e.g.: https://build.chromium.org/p/tryserver.blink/builders/linux_trusty_blink_rel/builds/8033 In this case the correct behavior would to consider all platforms that skipped webkit_tests to be all-passing, not to consider them as having the same failures as other platforms. Rebaseline-cl should also support the possible case where changes really do only affect some platforms (e.g. https://chromium-review.googlesource.com/c/468348/) and we only want to download results for some platforms -- we don't want to try to "fill in" results for platforms with no results.
,
May 22 2017
I think it would also be possible to fix this by changing the behavior of the tryserver.blink bots so that they all run layout tests even if layout tests are not theoretically affected. But, it still might be nicer if rebaseline-cl could consider a finished green job with no webkit_tests step to be all-passing, and it would also be good to able to explicitly rebaseline only platforms with results.
,
May 25 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4ce4987234f2b38382a7f3610ef195a94e5a6b12 commit 4ce4987234f2b38382a7f3610ef195a94e5a6b12 Author: Quinten Yearsley <qyearsley@google.com> Date: Thu May 25 19:37:38 2017 webkitpy: When fetching try jobs, also return try job status. This CL makes it so that the methods in webkitpy.common.net.git_cl return the status of try jobs (whether they're started or finished, successful or failed), so that later this can be used to improve the behavior of rebaseline-cl. Change-Id: I27c5eea571073bd1bf7888863efb3a1c8efec34b Bug: 713265 , 726376 Reviewed-on: https://chromium-review.googlesource.com/514662 Commit-Queue: Quinten Yearsley <qyearsley@chromium.org> Reviewed-by: Walter Korman <wkorman@chromium.org> Cr-Commit-Position: refs/heads/master@{#474756} [modify] https://crrev.com/4ce4987234f2b38382a7f3610ef195a94e5a6b12/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py [modify] https://crrev.com/4ce4987234f2b38382a7f3610ef195a94e5a6b12/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl_unittest.py
,
Jan 2 2018
qyearsley@, did #3 fix this issue, or is there more to be done? (Doing Blink>Infra triage of P2 issues older than 60 days.)
,
Jan 2 2018
This was a while ago, but the status appears to be: since July 2017 (https://crrev.com/c/582272) rebaseline-cl has supported rebaselining from only some bots without "filling in" for missing bots. As far as I know, that solved this problem. |
|||
►
Sign in to add a comment |
|||
Comment 1 by enne@chromium.org
, May 19 2017