New issue
Advanced search Search tips

Issue 873043 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Aug 15
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug

Blocking:
issue 702625



Sign in to add a comment

android-binary-size trybot doing clobber builds for every build

Project Member Reported by agrieve@chromium.org, Aug 10

Issue description

And it shouldn't be.
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-binary-size

ssh'ing into one of the slaves showed that the out directory did not exist:
chrome-bot@swarm2434-c4:(Linux 14.04):/b/swarming/w/ir/kitchen-workdir/src$ cd out
-bash: cd: out: No such file or directory

Looking at the Swarming bot page, it seems the problem may have to do with the recipe not enabling a build cache.

Recipe:
https://cs.chromium.org/chromium/build/scripts/slave/recipes/binary_size_trybot.py
 
Cc: iannucci@chromium.org no...@chromium.org tandrii@chromium.org
Components: -Infra>Client>Chrome Infra>Platform>Swarming Infra>Platform>Recipes
Owner: mar...@chromium.org
Status: Assigned (was: Available)
maruel@ - can you take a look at this? tandrii@ and I looked at it and it seems like we're not getting the named cache for the builder, but we don't know why not.

nodir / iannucci - for possible ideas as well.
Labels: -Pri-3 Pri-1
Cc: mar...@chromium.org
Owner: agrieve@chromium.org
CWD on LUCI is temporary. It is removed after each build.

To persist state, a build must save it to a cache dir, e.g. builder cache dir
docs: https://chromium.googlesource.com/infra/luci/luci-go/+/0b67d5886e5f420fc1397b1142173201c03f1fa4/buildbucket/proto/config/project_config.proto#101

here is how chromium recipe does this:
https://chromium.googlesource.com/chromium/tools/build/+/439f11fefac35a48cfc72f3fb70d736fca77323e/scripts/slave/recipe_modules/chromium_checkout/api.py#45

this builder uses a different recipe:
https://cs.chromium.org/chromium/build/scripts/slave/recipes/binary_size_trybot.py

assigning to the owner of the recipe
Thanks, nodir@

I see, the binary_size_trybot.py recipe uses bot_update.ensure_checkout() directly, but the chromium recipes use chromium_tests.prepare_checkout(), which calls chromium_checkout.ensure_checkout() instead.

The binary_size_trybot recipe should probably at least call the chromium_checkout entry point instead of bot_update's.
Sorry for needing hand-holding here, but the parameters are a bit different for chromium_checkout.ensure_checkout() and I'm not sure what to pass.

Currently I'm using:
suffix = ' (with patch)'
api.bot_update.ensure_checkout(suffix=suffix, patch=True)

api.chromium_checkout.ensure_checkout() looks like:
def ensure_checkout(
      self, bot_config, root_solution_revision=None,
      disable_syntax_validation=False):

It doesn't accept suffix, nor patch=True.
Should I pass an empty dict for bot_config? Or use:
api.chromium_tests.create_bot_config_object('tryserver.chromium.android', 'android-binary-size')

Is patch=True implicit here?
Should I add a suffix param so the step name can say " (with patch)"?
Are there any steps I now need to wrap with:
  with api.context(cwd=checkout_dir):
Cc: -iannucci@chromium.org -tandrii@chromium.org -mar...@chromium.org
Components: -Infra>Platform>Recipes -Infra>Platform>Swarming Infra>Client>Chrome
Cc: jbudorick@chromium.org bpastene@chromium.org
You should just be able to use bot_config={}. I *think* you don't need "(with patch)" or to set patch=True, but I'm not sure, so I'd try it and see.

Once you have the checkout, the rest of the code can I think stay as-is.

There's so many layers of cruft in this code now :(.


Project Member

Comment 8 by bugdroid1@chromium.org, Aug 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/28d1be6dd9c311c674ac1d0375cea5b2dfb35733

commit 28d1be6dd9c311c674ac1d0375cea5b2dfb35733
Author: Andrew Grieve <agrieve@chromium.org>
Date: Tue Aug 14 18:06:48 2018

Fix binary_size_trybot.py not having a build cache

Without a build cache, it's doing clobber builds every time.

Bug:  873043 
Change-Id: I465ef9f1407fa60e7c4f81b04cb3ba86ec248a6c
Reviewed-on: https://chromium-review.googlesource.com/1174859
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: agrieve <agrieve@chromium.org>

[modify] https://crrev.com/28d1be6dd9c311c674ac1d0375cea5b2dfb35733/scripts/slave/recipes/binary_size_trybot.expected/normal_build.json
[modify] https://crrev.com/28d1be6dd9c311c674ac1d0375cea5b2dfb35733/scripts/slave/recipes/binary_size_trybot.py

Status: Fixed (was: Assigned)
Builds going much faster now :)

Sign in to add a comment