New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 658880 link

Starred by 1 user

Issue metadata

Status: Fixed
Merged: issue 673966
Owner:
Closed: Oct 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Feature



Sign in to add a comment

Add a --builders (or --platform) flag to webkit-patch rebaseline-cl.

Project Member Reported by qyears...@chromium.org, Oct 24 2016

Issue description

Current, webkit-patch rebaseline-cl checks for new baselines on all of the Blink try bots.

To enable more fine-grained control of what the tool does, some changes could be made:
 - A set of builders could be specified on the command line, and the tool would only try to download new baselines from this set of builders.
 - Relatedly, if there are results for a subset of the builders, the tool could optionally try to download baselines just for the builders that have results, 

Possibly confusing downside: When downloading a new baseline from a builder, it is downloaded by default to a platform-specific baseline location. Even if a layout test actually has a new non-platform-specific result, the optimize-baselines step would still only de-duplicate the baseline and put the new baseline in a non-platform-specific location if new baselines for all platforms are downloaded.
 
When would you need this, rather than just downloading all of the new baselines?
Labels: -Type-Bug -Pri-2 Pri-3 Type-Feature
Potentially, if you know that you just want to update baselines for a particular builder; then it wouldn't be necessary to wait for all of the try bots to have results before you could download them.

But now I'm inclined to think that this probably unnecessary.

I filed this issue after it was mentioned in chat by zakerinasab@; zakerinasab@, do you think that this feature would be worthwhile?
I think it might get worthy depending on the bug properties. For example, I'm trying to fix a bug related to subscript and superscript in SVG fonts. For some reason the patch works on Linux and Mac (local and try-bots) but when it comes to Windows it works on the local machine but fails on the try-bot (in fact, the behavior is different). In past week I ran rebaseline-cl like 10 times to get the new results (and see the debug info) from try-bots, every time running all android/linux/mac try-bots without needing them.
Ah, I see, so it's important to you to have the debug results locally rather than just clicking through to the build results from the tryserver?
I didn't know that I can download the test results in an archive. So I can either download the results archive and look for my test results in the folder hierarchy to extract for each tryserver, or I can use rebaseline-cl to get the results only for my test. I guess the latter is more convenient and cleaner, but it's not as necessary as I thought.
Owner: ----
Status: Available (was: Assigned)
Yep, once you've run try jobs, the actual results are archived and you can download results for a builder:

e.g. if you
 1. go to a build page: https://build.chromium.org/p/tryserver.blink/builders/mac10.11_blink_rel/builds/1163
 2. click on layout_test_results under archive_webkit_tests_results
 3. click on actual
 4. download, rename, move

then you can get results for an individual builder. "webkit-patch rebaseline-cl" is basically just a tool to download all new baselines for a set of tests at once.
You can download all of the results for a run by clicking on the zip link, too.
Mergedinto: 673966
Status: Duplicate (was: Available)
Owner: szager@chromium.org
Status: Started (was: Duplicate)
Actually, now that I see this again, this is slightly different from  bug 673966 , and Stefan has just uploaded a CL that adds a --builders flag: https://chromium-review.googlesource.com/c/chromium/src/+/724172 :-)
Project Member

Comment 10 by bugdroid1@chromium.org, Oct 18 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0084b9b88ff3c27f43fff5f9b84c53295ed5f4d3

commit 0084b9b88ff3c27f43fff5f9b84c53295ed5f4d3
Author: Stefan Zager <szager@chromium.org>
Date: Wed Oct 18 00:25:36 2017

Add --builders argument to webkit-patch rebaseline_cl

R=qyearsley@chromium.org

Bug:  658880 
Change-Id: Ifd2ea784a589fba1e34cf2289eba488a25528ebd
Reviewed-on: https://chromium-review.googlesource.com/724172
Commit-Queue: Stefan Zager <szager@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#509622}
[modify] https://crrev.com/0084b9b88ff3c27f43fff5f9b84c53295ed5f4d3/third_party/WebKit/Tools/Scripts/webkitpy/tool/commands/rebaseline_cl.py
[modify] https://crrev.com/0084b9b88ff3c27f43fff5f9b84c53295ed5f4d3/third_party/WebKit/Tools/Scripts/webkitpy/tool/commands/rebaseline_cl_unittest.py

After the last change, I'm now getting an error when rebaseline-cl triggers try jobs, fix in https://crrev.com/c/726970
Project Member

Comment 12 by bugdroid1@chromium.org, Oct 19 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0646dd9eb594248414bbed2accf646f0fddbb6e6

commit 0646dd9eb594248414bbed2accf646f0fddbb6e6
Author: Quinten Yearsley <qyearsley@chromium.org>
Date: Thu Oct 19 19:28:43 2017

Allow GitCL.trigger_try_jobs to take a frozenset

Bug:  658880 
Change-Id: I8f03010037dbc45daa9165e9e5ae8c47ade889d0
Reviewed-on: https://chromium-review.googlesource.com/726970
Reviewed-by: Stefan Zager <szager@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#510158}
[modify] https://crrev.com/0646dd9eb594248414bbed2accf646f0fddbb6e6/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py
[modify] https://crrev.com/0646dd9eb594248414bbed2accf646f0fddbb6e6/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl_unittest.py

Status: Fixed (was: Started)

Sign in to add a comment