Is self.m.path[] ever defined for swarming builds? These are the variations I see in use.
self.m.path['root']
self.m.path['start_dir']
self.m.path.c.dynamic_paths.get('chromite')
Another group is in the process of setting up goma on ChromeOS builders, and the recipe currently installs the binaries via CIDP for every build.
This process hasn't been smooth, so these changes have landed and been reverted multiple times, but we are expecting to have the permissions to fetch and use the goma binaries for every ChromeOS builder now.
That said, some of the chromeos-bot ccompute builders are attached to waterfalls that are also installing goma binaries successfully.
A random recent example:
https://uberchromegw.corp.google.com/i/chromeos/builders/auron_yuna-release/builds/1122
Ah..... DNJ helped me figure this out.
I was doing a Swarming Retry. On a SwarmBucket build that doesn't work, and instead pulls up the UI for the most recent matching build without rerunning anything. But it pulls it up slowly enough to think that it's actually starting a new build.
Sigh.
It's getting further. We are reaching chromite code, before it dies because 'repo' isn't in the path.
Our builders normally have depot_tools pre-checked out and in the path. Is this not the case for swarming builders?
If not, is this best addressed by a change to the recipe?
We are shelling out to non-CrOps scripts that assume various depot_tools commands are in the path like repo and git-cache.
Cleaning this up is non-trivial and far from risk free.
The recipe already checks out a copy of chromite at TOT. Should I do the same thing with depot_tools, then insert that into the path? Or is there a better approach?
Since your recipe already checks out "depot_tools", you can leverage that:
1) Create a recipe module in the "depot_tools" directory, maybe something like "cros_tools" or "cros/tools": https://chromium.googlesource.com/chromium/tools/depot_tools/+/master/recipes/recipe_modules/
2) Have that recipe module simply export:
---
@property
def depot_tool_path(self):
return self.PACKAGE_REPO_ROOT
---
3) Add a flag to "cbuildbot_launcher" that accepts the "depot_tools" path.
4) Pass that to cbuildbot:
...
'--depot-tools-path', self.m.cros_tools.depot_tools_path,
...
At this point, "cbuildbot_launcher" can choose what to do, which (for now) is probably going to be adding it to PATH. You can even have it check out its own HEAD version of "depot_tools" to a temporary directory if the flag is not provided.
If you want to pin "depot_tools", you will have to do your own checkout. That can be done in the recipe or by "cbuildbot_launcher". If you go with the flag approach, pinning in recipe should be easy - just use the "git" module to do a "depot_tools" checkout and pass that path to "cbuildbot_launcher".
Comment 1 by dgarr...@chromium.org
, May 10 2017Is self.m.path[] ever defined for swarming builds? These are the variations I see in use. self.m.path['root'] self.m.path['start_dir'] self.m.path.c.dynamic_paths.get('chromite')