This bug tracks implementation of the try-flag script described in this explainer: https://docs.google.com/document/d/1nPtn3u0aR6sDlF5R7GYrrTOvIWsBZ5R5PSMGOUXhtfQ/edit
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/668749693c0abc7f00871a2abea73628e6aa0332 commit 668749693c0abc7f00871a2abea73628e6aa0332 Author: Steve Kobes <skobes@chromium.org> Date: Sat Aug 26 00:50:21 2017 Read primary driver flag from LayoutTests/rwt.flag if present. Bug: 755401 Change-Id: Ib0bfabfde45376c92337fb4c940023cde104ffa2 Reviewed-on: https://chromium-review.googlesource.com/634512 Commit-Queue: Steve Kobes <skobes@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Cr-Commit-Position: refs/heads/master@{#497616} [modify] https://crrev.com/668749693c0abc7f00871a2abea73628e6aa0332/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/android.py [modify] https://crrev.com/668749693c0abc7f00871a2abea73628e6aa0332/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/base.py [modify] https://crrev.com/668749693c0abc7f00871a2abea73628e6aa0332/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/base_unittest.py [modify] https://crrev.com/668749693c0abc7f00871a2abea73628e6aa0332/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/driver.py [modify] https://crrev.com/668749693c0abc7f00871a2abea73628e6aa0332/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/linux.py [modify] https://crrev.com/668749693c0abc7f00871a2abea73628e6aa0332/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/win.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7c1932ac8763851e02a633ddec6ddf4c2732e5be commit 7c1932ac8763851e02a633ddec6ddf4c2732e5be Author: Steve Kobes <skobes@chromium.org> Date: Tue Sep 12 00:32:40 2017 Don't mutate _options in additional_driver_flags. With rwt.flag present, this method was adding flags to the array inside the optparse.Values object, so the command line grew extra copies of those flags with repeated calls, until system limits were reached. Bug: 755401 Change-Id: I49ae17d352f6a2a4f4c048e5ef0b488d955a6c64 Reviewed-on: https://chromium-review.googlesource.com/661739 Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Commit-Queue: Steve Kobes <skobes@chromium.org> Cr-Commit-Position: refs/heads/master@{#501119} [modify] https://crrev.com/7c1932ac8763851e02a633ddec6ddf4c2732e5be/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/base.py [modify] https://crrev.com/7c1932ac8763851e02a633ddec6ddf4c2732e5be/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/base_unittest.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4aa01c57644ad9de00d481dddb0ce519b9ba3950 commit 4aa01c57644ad9de00d481dddb0ce519b9ba3950 Author: Steve Kobes <skobes@chromium.org> Date: Thu Oct 05 03:53:10 2017 Ignore Swarming tasks in GitCL.latest_try_jobs. Swarming task IDs aren't comparable to BuildBot build numbers. We'll need something smarter in a post-BuildBot world, but this gets things working for now. Bug: 755401, 771438 Change-Id: Ifa13bc95c3a7527d7272fa3f35cbd385328aa945 Reviewed-on: https://chromium-review.googlesource.com/701916 Commit-Queue: Steve Kobes <skobes@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Cr-Commit-Position: refs/heads/master@{#506632} [modify] https://crrev.com/4aa01c57644ad9de00d481dddb0ce519b9ba3950/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py [modify] https://crrev.com/4aa01c57644ad9de00d481dddb0ce519b9ba3950/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl_unittest.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8b44b01e79b60189d06fc2fb82a4a3f5a4b51f5d commit 8b44b01e79b60189d06fc2fb82a4a3f5a4b51f5d Author: Steve Kobes <skobes@chromium.org> Date: Fri Oct 06 22:49:06 2017 Implement try-flag v0. Currently just gathers results, merges with platform specifiers, and dumps expectation lines to stdout. In action it looks like this: https://pastebin.com/raw/UKsc0d2g Bug: 755401 Change-Id: Ib6f3fa360ae82e10ba540b81194a8b5a86c0a892 Reviewed-on: https://chromium-review.googlesource.com/699658 Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Commit-Queue: Steve Kobes <skobes@chromium.org> Cr-Commit-Position: refs/heads/master@{#507206} [add] https://crrev.com/8b44b01e79b60189d06fc2fb82a4a3f5a4b51f5d/third_party/WebKit/Tools/Scripts/try-flag [modify] https://crrev.com/8b44b01e79b60189d06fc2fb82a4a3f5a4b51f5d/third_party/WebKit/Tools/Scripts/webkitpy/common/net/buildbot.py [modify] https://crrev.com/8b44b01e79b60189d06fc2fb82a4a3f5a4b51f5d/third_party/WebKit/Tools/Scripts/webkitpy/common/net/buildbot_mock.py [modify] https://crrev.com/8b44b01e79b60189d06fc2fb82a4a3f5a4b51f5d/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py [add] https://crrev.com/8b44b01e79b60189d06fc2fb82a4a3f5a4b51f5d/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/try_flag.py [add] https://crrev.com/8b44b01e79b60189d06fc2fb82a4a3f5a4b51f5d/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/try_flag_unittest.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/42b9598debc48402a39ceaa4dc5512ea6380f7b5 commit 42b9598debc48402a39ceaa4dc5512ea6380f7b5 Author: Steve Kobes <skobes@chromium.org> Date: Fri Oct 13 19:32:04 2017 Fix crash in try-flag. Functions marked @memoized cannot receive kwargs. (This broke in patch set 4 of http://crrev.com/c/699658, and wasn't caught by the test because it mocks out the BuildBot object.) Bug: 755401 Change-Id: Ib1515d2680bcc21e0e73951dc36b2f0141307764 Reviewed-on: https://chromium-review.googlesource.com/718483 Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Commit-Queue: Steve Kobes <skobes@chromium.org> Cr-Commit-Position: refs/heads/master@{#508777} [modify] https://crrev.com/42b9598debc48402a39ceaa4dc5512ea6380f7b5/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/try_flag.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f00d23311218cd16ce33ca79754b465ef610ff3c commit f00d23311218cd16ce33ca79754b465ef610ff3c Author: Philip Rogers <pdr@chromium.org> Date: Wed Oct 18 17:53:06 2017 Rename "rwt.flag" to "additional-driver-flag.setting" This patch just renames the additional driver flag setting file. Old: LayoutTests/rwt.flag New: LayoutTests/additional-driver-flag.setting The goal is to reduce confusion about what this file really does. Bug: 755401 Change-Id: I17168e772092874eb5cf839a9d0857d966638a96 Reviewed-on: https://chromium-review.googlesource.com/726180 Reviewed-by: Steve Kobes <skobes@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Commit-Queue: Philip Rogers <pdr@chromium.org> Cr-Commit-Position: refs/heads/master@{#509808} [modify] https://crrev.com/f00d23311218cd16ce33ca79754b465ef610ff3c/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/base.py [modify] https://crrev.com/f00d23311218cd16ce33ca79754b465ef610ff3c/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/base_unittest.py [modify] https://crrev.com/f00d23311218cd16ce33ca79754b465ef610ff3c/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/try_flag.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b63db4aaecfaee13d131892d79ba279428f85c5a commit b63db4aaecfaee13d131892d79ba279428f85c5a Author: Steve Kobes <skobes@chromium.org> Date: Thu Oct 26 01:24:14 2017 Let callers of GitCL.trigger_try_jobs specify the master. Bug: 755401 Change-Id: I4cfb6fcd4a124def9c9bd5861a627939b1c015a0 Reviewed-on: https://chromium-review.googlesource.com/734190 Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Commit-Queue: Steve Kobes <skobes@chromium.org> Cr-Commit-Position: refs/heads/master@{#511682} [modify] https://crrev.com/b63db4aaecfaee13d131892d79ba279428f85c5a/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py [modify] https://crrev.com/b63db4aaecfaee13d131892d79ba279428f85c5a/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl_unittest.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b98ff200379c1147545deaefe719ffef0480b9b5 commit b98ff200379c1147545deaefe719ffef0480b9b5 Author: Steve Kobes <skobes@chromium.org> Date: Wed Nov 01 18:40:30 2017 Implement 'try-flag trigger'. Bug: 755401 Change-Id: Ica41e281b77f3b3625e65f883aedd394470c2a6d Reviewed-on: https://chromium-review.googlesource.com/740292 Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Commit-Queue: Steve Kobes <skobes@chromium.org> Cr-Commit-Position: refs/heads/master@{#513194} [modify] https://crrev.com/b98ff200379c1147545deaefe719ffef0480b9b5/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl_mock.py [modify] https://crrev.com/b98ff200379c1147545deaefe719ffef0480b9b5/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/try_flag.py [modify] https://crrev.com/b98ff200379c1147545deaefe719ffef0480b9b5/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/try_flag_unittest.py
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/fd3e46e8e7e3be4b07482340b3bb7bc3ddfcb2f5 commit fd3e46e8e7e3be4b07482340b3bb7bc3ddfcb2f5 Author: Steve Kobes <skobes@chromium.org> Date: Wed Nov 29 01:45:20 2017 try-flag: Intersect unexpected passes with the flag expectation file. Previously, try-flag reported all unexpected passes, even if the failure expectation came from the main TestExpectations file. This produced a lot of irrelevant noise. Now, try-flag only reports unexpected passes of tests that have flag-specific failure expectations. The --flag argument is no longer optional. Bug: 755401 Change-Id: Ifa507e19159b8cdadf63752f5c8aa2635de014d5 Reviewed-on: https://chromium-review.googlesource.com/794877 Commit-Queue: Steve Kobes <skobes@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Reviewed-by: Philip Rogers <pdr@chromium.org> Cr-Commit-Position: refs/heads/master@{#519950} [modify] https://crrev.com/fd3e46e8e7e3be4b07482340b3bb7bc3ddfcb2f5/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/try_flag.py [modify] https://crrev.com/fd3e46e8e7e3be4b07482340b3bb7bc3ddfcb2f5/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/try_flag_unittest.py
Is there any work left to do here?
Downgrading to P3, as this is blocked on bug 833897, which doesn't seem to be too urgent per comments #4 and #5 there.
Comment 1 by qyears...@chromium.org
, Aug 16 2017Summary: Implement try-flag, a tool for helping to run layout test try jobs with flags (was: implement try-flag)