Precedent: cipd is already a native first class support in upstream recipes-py, this is the basis for replacing swarming_client. Rationale: The swarming recipe module is used for many recipes in many projects. It's currently reimplemented as some projects are using the python implementation, some are using the Go implementation. Yay for maintainability. Blocker: This should be designed for the Go implementation only. Issue 894045 to rewrite the client in Go. This means this task won't be done for a while, as 894045 will take a while to complete. AIs: - Rewrite the chromium swarming recipe module in-place to become sane and the reference implementation. - Make it self-bootstrap with the Go implementation. - The workflow shall be to fetch the tooling via CIPD, for deterministic bootstrapping process. - Reimplement swarming_client to fetch the Go implementations via CIPD. Refs: - https://chromium.googlesource.com/infra/luci/recipes-py/+/master/recipe_modules/cipd - https://chromium.googlesource.com/chromium/tools/build/+/master/scripts/slave/recipe_modules/isolate - https://chromium.googlesource.com/chromium/tools/build/+/master/scripts/slave/recipe_modules/swarming - https://chromium.googlesource.com/chromium/tools/build/+/master/scripts/slave/recipe_modules/swarming_client
Joshua has been working on creating upstream recipe_modules already. A following part is doing the migration, which will probably require more effort.
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/recipes-py/+/a66cf68621e2c514b7684e5404cd700a700f1aba commit a66cf68621e2c514b7684e5404cd700a700f1aba Author: Joshua Seaton <joshuaseaton@google.com> Date: Fri Dec 14 23:45:02 2018 [swarming] Create minimal swarming module. This introduces only a trigger method; collect() will be added in the following change. Bug: 894048 Change-Id: I469a00a919cbd66968ec9035b6aba71605e9262e Reviewed-on: https://chromium-review.googlesource.com/c/1350350 Commit-Queue: Joshua Seaton <joshuaseaton@google.com> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> Reviewed-by: Takuto Ikuta <tikuta@chromium.org> [add] https://crrev.com/a66cf68621e2c514b7684e5404cd700a700f1aba/recipe_modules/swarming/api.py [add] https://crrev.com/a66cf68621e2c514b7684e5404cd700a700f1aba/recipe_modules/swarming/examples/full.expected/override swarming.json [add] https://crrev.com/a66cf68621e2c514b7684e5404cd700a700f1aba/recipe_modules/swarming/__init__.py [add] https://crrev.com/a66cf68621e2c514b7684e5404cd700a700f1aba/recipe_modules/swarming/examples/full.py [add] https://crrev.com/a66cf68621e2c514b7684e5404cd700a700f1aba/recipe_modules/swarming/examples/full.expected/experimental.json [modify] https://crrev.com/a66cf68621e2c514b7684e5404cd700a700f1aba/README.recipes.md [add] https://crrev.com/a66cf68621e2c514b7684e5404cd700a700f1aba/recipe_modules/swarming/test_api.py [add] https://crrev.com/a66cf68621e2c514b7684e5404cd700a700f1aba/recipe_modules/swarming/examples/full.expected/basic.json
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/recipes-py/+/0fad0863959ecf172fb460a7008eeabf8c8f078d commit 0fad0863959ecf172fb460a7008eeabf8c8f078d Author: Joshua Seaton <joshuaseaton@google.com> Date: Thu Dec 20 19:23:23 2018 [swarming] Minor clean-up and typo fixes. Bug: 894048 Change-Id: Iad3eafe15d489a8ffb3b3f3c3d22551dbae15d55 Reviewed-on: https://chromium-review.googlesource.com/c/1382923 Commit-Queue: Joshua Seaton <joshuaseaton@google.com> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/0fad0863959ecf172fb460a7008eeabf8c8f078d/recipe_modules/swarming/api.py [modify] https://crrev.com/0fad0863959ecf172fb460a7008eeabf8c8f078d/recipe_modules/swarming/examples/full.expected/override swarming.json [modify] https://crrev.com/0fad0863959ecf172fb460a7008eeabf8c8f078d/recipe_modules/swarming/examples/full.py [modify] https://crrev.com/0fad0863959ecf172fb460a7008eeabf8c8f078d/recipe_modules/swarming/examples/full.expected/experimental.json [modify] https://crrev.com/0fad0863959ecf172fb460a7008eeabf8c8f078d/README.recipes.md [modify] https://crrev.com/0fad0863959ecf172fb460a7008eeabf8c8f078d/recipe_modules/swarming/examples/full.expected/basic.json
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/recipes-py/+/19c28875c9f1519cac7156e07dbb8e59ff40f7c1 commit 19c28875c9f1519cac7156e07dbb8e59ff40f7c1 Author: Joshua Seaton <joshuaseaton@google.com> Date: Thu Dec 20 19:45:33 2018 [isolated] Add download method Bug: 894048 Change-Id: I4ac0614c49f789586da5c9e6a102556d495a5e4e Reviewed-on: https://chromium-review.googlesource.com/c/1382820 Commit-Queue: Joshua Seaton <joshuaseaton@google.com> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/19c28875c9f1519cac7156e07dbb8e59ff40f7c1/recipe_modules/isolated/examples/full.expected/experimental.json [modify] https://crrev.com/19c28875c9f1519cac7156e07dbb8e59ff40f7c1/recipe_modules/isolated/examples/full.expected/override isolated.json [modify] https://crrev.com/19c28875c9f1519cac7156e07dbb8e59ff40f7c1/recipe_modules/isolated/examples/full.expected/basic.json [modify] https://crrev.com/19c28875c9f1519cac7156e07dbb8e59ff40f7c1/README.recipes.md [modify] https://crrev.com/19c28875c9f1519cac7156e07dbb8e59ff40f7c1/recipe_modules/isolated/examples/full.py [modify] https://crrev.com/19c28875c9f1519cac7156e07dbb8e59ff40f7c1/recipe_modules/isolated/api.py
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/recipes-py/+/354bb02e9c0b74eea43faa883ac1aa134c435e3c commit 354bb02e9c0b74eea43faa883ac1aa134c435e3c Author: Joshua Seaton <joshuaseaton@google.com> Date: Thu Dec 20 20:00:03 2018 [swarming] Add support for environment variables. Bug: 894048 Change-Id: I6ca7e1fb09045e681b4139df1b756d1a9d9ce2c4 Reviewed-on: https://chromium-review.googlesource.com/c/1375072 Commit-Queue: Joshua Seaton <joshuaseaton@google.com> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/354bb02e9c0b74eea43faa883ac1aa134c435e3c/recipe_modules/swarming/api.py [modify] https://crrev.com/354bb02e9c0b74eea43faa883ac1aa134c435e3c/recipe_modules/swarming/examples/full.expected/override swarming.json [modify] https://crrev.com/354bb02e9c0b74eea43faa883ac1aa134c435e3c/recipe_modules/swarming/examples/full.py [modify] https://crrev.com/354bb02e9c0b74eea43faa883ac1aa134c435e3c/recipe_modules/swarming/examples/full.expected/experimental.json [modify] https://crrev.com/354bb02e9c0b74eea43faa883ac1aa134c435e3c/README.recipes.md [modify] https://crrev.com/354bb02e9c0b74eea43faa883ac1aa134c435e3c/recipe_modules/swarming/examples/full.expected/basic.json
Comment 1 by iannu...@google.com
, Oct 18