--swarming-py-path argument to trigger scripts is problematic and should be removed |
||||
Issue descriptionIn Issue 758630 the ability to inject a Swarming trigger script was added, in order to customize how jobs are triggered in the Swarming pool. The mechanism is very useful, but the --swarming-py-path argument that is passed to the trigger script is highly problematic. Here's why: 1) The trigger script absolutely has to filter out this argument before turning around and invoking "swarming.py trigger". Otherwise, the target script or executable will see it and be confused by it. 2) The easiest way to do this is to use argparse to add an argument "--swarming-py-path", and use ArgumentParser.parse_known_args() to chomp this and other expected arguments before calling "swarming.py trigger". 3) Unfortunately, ArgumentParser has built-in functionality to allow arguments to be abbreviated. --swarming-py-path therefore collides with the necessary argument "--swarming" which is passed to swarming.py. This ends up chomping the wrong command line argument. 4) Adding another argument to argparse "--swarming" does not really help, because it turns out that "--swarming-py-path" is placed after the "--" in the command line, and this causes argparse to not consume it, anyway. The best solution is for recipe_modules/swarming/api.py to not pass this argument to the trigger script at all. It's unnecessary, because the trigger script is invoked out of the Chromium checkout, and the script can figure out where src/tools/swarming_client lives relative to it and just invoke it itself.
,
Jan 12 2018
Downgrading to P2. There are other more important issues and I've already worked around this one.
,
Jan 16 2018
Thanks for calling this out Ken. I was planning on dropping it in the src side script, but I also wasn't aware of the abbreviation issue. I can own this since it is necessary to get the trigger script working.
,
Jan 16 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/92f30738b8318982cda243776b6454aa8410bbe9 commit 92f30738b8318982cda243776b6454aa8410bbe9 Author: Stephen Martinis <martiniss@chromium.org> Date: Tue Jan 16 23:49:13 2018 Improve trigger script docs Remove swarming-py-path being passed, and clarifies that an output json file is expected. Bug: 801346 Change-Id: I23009789333e270e483fa8eeb09c2cde858300c9 Reviewed-on: https://chromium-review.googlesource.com/863042 Commit-Queue: Stephen Martinis <martiniss@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> [modify] https://crrev.com/92f30738b8318982cda243776b6454aa8410bbe9/scripts/slave/README.recipes.md [modify] https://crrev.com/92f30738b8318982cda243776b6454aa8410bbe9/scripts/slave/recipe_modules/chromium_tests/tests/steps/generate_isolated_script.expected/swarming_trigger_script.json [modify] https://crrev.com/92f30738b8318982cda243776b6454aa8410bbe9/scripts/slave/recipe_modules/swarming/examples/full.expected/isolated_script_with_custom_trigger_script.json [modify] https://crrev.com/92f30738b8318982cda243776b6454aa8410bbe9/scripts/slave/recipe_modules/swarming/api.py
,
Jan 16 2018
I landed a CL in #4 which should have fixed this. I was working on that last week, but didn't assign this bug to myself, sorry.
,
Jan 17 2018
Thanks Stephen. Sorry Emily for the wasted work, but at least this is fixed now. |
||||
►
Sign in to add a comment |
||||
Comment 1 by martiniss@chromium.org
, Jan 11 2018