New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 617604 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug

Blocking:
issue 574557



Sign in to add a comment

recipes shouldn't pass --clobber to compile.py

Project Member Reported by thakis@chromium.org, Jun 6 2016

Issue description

Clobbering right before compilation in compile.py has a bunch of issues (it  tends to delete files that were created by earlier steps and that are needed for compiles).

I fixed this for recipe-using bots in  bug 574557 , but the recipe_autogen.py recipe files don't use the normal recipe machinery for some reason (https://cs.chromium.org/search/?q=file:recipe_autogen+clobber+-file:%5C.json&sq=package:chromium&type=cs).

Instead of passing --clobber to compile.py, they should use chromium_apply_configs: ['clobber'] and call chromium.api.runhooks() which will then automatically add a clobber step. (If that's infeasible, then this line https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium/api.py?rcl=0&l=494 should be extracted into its own "clobber()" function and then the recipe should call chromium.api.clobber() before runhooks).

I'm not sure what the overall vision for these generated recipe bots is, so giving this to you rather than doing it myself.

Without this, swarming doesn't work on clobber gn bots (since the clobber-during-compile step deletes the .isolated files gn creates during runhooks), so this blocks gn.
 
Cc: phajdan@google.com estaab@chromium.org iannucci@chromium.org martiniss@chromium.org
Owner: ----
Status: Available (was: Assigned)
Aneesh is away and likely won't get to this so cc'ing recipe owners.
Owner: thakis@chromium.org
Status: Started (was: Available)
:-(
Project Member

Comment 3 by bugdroid1@chromium.org, Jun 8 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/8d2793855e7f3ac1897002b5b8d8fb385f97750b

commit 8d2793855e7f3ac1897002b5b8d8fb385f97750b
Author: thakis <thakis@chromium.org>
Date: Wed Jun 08 16:47:25 2016

Give autogen bots on chromium.fyi and client.nacl.sdk a separate clobber step

Some bots were converted to recipes using some script that automatically
rewrites the old buildbot config.  These bots don't use the regular recipes
machinery, so they didn't get converted as part of the work in  bug 574557 .

On the nacl waterfall, also remove all.sln; these bots have been building using
ninja for a long time.

BUG= 617604 

Review-Url: https://codereview.chromium.org/2047963003

[modify] https://crrev.com/8d2793855e7f3ac1897002b5b8d8fb385f97750b/scripts/slave/recipes/chromium.fyi.misc_block.recipe_autogen.expected/CFI_Linux_CF.json
[modify] https://crrev.com/8d2793855e7f3ac1897002b5b8d8fb385f97750b/scripts/slave/recipes/chromium.fyi.misc_block.recipe_autogen.py
[modify] https://crrev.com/8d2793855e7f3ac1897002b5b8d8fb385f97750b/scripts/slave/recipes/client.nacl.sdk.recipe_autogen.expected/linux_sdk_asan_multi.json
[modify] https://crrev.com/8d2793855e7f3ac1897002b5b8d8fb385f97750b/scripts/slave/recipes/client.nacl.sdk.recipe_autogen.expected/linux_sdk_multi.json
[modify] https://crrev.com/8d2793855e7f3ac1897002b5b8d8fb385f97750b/scripts/slave/recipes/client.nacl.sdk.recipe_autogen.expected/linux_sdk_multirel.json
[modify] https://crrev.com/8d2793855e7f3ac1897002b5b8d8fb385f97750b/scripts/slave/recipes/client.nacl.sdk.recipe_autogen.expected/mac_sdk_multi.json
[modify] https://crrev.com/8d2793855e7f3ac1897002b5b8d8fb385f97750b/scripts/slave/recipes/client.nacl.sdk.recipe_autogen.expected/mac_sdk_multirel.json
[modify] https://crrev.com/8d2793855e7f3ac1897002b5b8d8fb385f97750b/scripts/slave/recipes/client.nacl.sdk.recipe_autogen.expected/windows_sdk_multi.json
[modify] https://crrev.com/8d2793855e7f3ac1897002b5b8d8fb385f97750b/scripts/slave/recipes/client.nacl.sdk.recipe_autogen.expected/windows_sdk_multirel.json
[modify] https://crrev.com/8d2793855e7f3ac1897002b5b8d8fb385f97750b/scripts/slave/recipes/client.nacl.sdk.recipe_autogen.py

Project Member

Comment 4 by bugdroid1@chromium.org, Jun 8 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/474ff687660cdb53c770f676684de790b4973b94

commit 474ff687660cdb53c770f676684de790b4973b94
Author: thakis <thakis@chromium.org>
Date: Wed Jun 08 16:51:22 2016

Give autogen bots on client.drmemory a separate clobber step

Some bots were converted to recipes using some script that automatically
rewrites the old buildbot config. These bots don't use the regular recipes
machinery, so they didn't get converted as part of the work in  bug 574557 .

Also remove all.sln; these bots have been building using ninja for a long time.

BUG= 617604 

Review-Url: https://codereview.chromium.org/2047383002

[modify] https://crrev.com/474ff687660cdb53c770f676684de790b4973b94/scripts/slave/recipes/client.drmemory.recipe_autogen.expected/linux_cr_builder_clobber.json
[modify] https://crrev.com/474ff687660cdb53c770f676684de790b4973b94/scripts/slave/recipes/client.drmemory.recipe_autogen.expected/win7_cr_builder.json
[modify] https://crrev.com/474ff687660cdb53c770f676684de790b4973b94/scripts/slave/recipes/client.drmemory.recipe_autogen.expected/win7_cr_builder_clobber.json
[modify] https://crrev.com/474ff687660cdb53c770f676684de790b4973b94/scripts/slave/recipes/client.drmemory.recipe_autogen.expected/win8_cr_builder.json
[modify] https://crrev.com/474ff687660cdb53c770f676684de790b4973b94/scripts/slave/recipes/client.drmemory.recipe_autogen.expected/win8_cr_builder_clobber.json
[modify] https://crrev.com/474ff687660cdb53c770f676684de790b4973b94/scripts/slave/recipes/client.drmemory.recipe_autogen.py

Project Member

Comment 5 by bugdroid1@chromium.org, Jun 8 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/da41f5b72b6a33c13a69545683321531ef462daf

commit da41f5b72b6a33c13a69545683321531ef462daf
Author: recipe-roller <recipe-roller@chromium.org>
Date: Wed Jun 08 16:59:26 2016

Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

build:
  https://crrev.com/8d2793855e7f3ac1897002b5b8d8fb385f97750b Give autogen bots on chromium.fyi and client.nacl.sdk a separate clobber step (thakis@chromium.org)
  https://crrev.com/474ff687660cdb53c770f676684de790b4973b94 Give autogen bots on client.drmemory a separate clobber step (thakis@chromium.org)

R=thakis@chromium.org
BUG= 617604 

TBR=martiniss@chromium.org,phajdan.jr@chromium.org

Review-Url: https://codereview.chromium.org/2040383003

[modify] https://crrev.com/da41f5b72b6a33c13a69545683321531ef462daf/infra/config/recipes.cfg

Status: Fixed (was: Started)
Project Member

Comment 7 by bugdroid1@chromium.org, Jun 17 2016

Sign in to add a comment