New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 620158 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 0
Type: ----



Sign in to add a comment

recipe_properties.json not found

Project Member Reported by jam@chromium.org, Jun 14 2016

Issue description

See https://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presubmit/builds/200659/steps/steps/logs/stdio

All the steps are green but the try run is red.
>

[W2016-06-14T16:09:15.829227-07:00 7780 0 main.go:268] Failed to write bootstrap result.           {"error":"open /b/build/slave/linux/build/.recipe_runtime/tmpOFwWZZ/bootstrap_result.json: no such file or directory"}
[I2016-06-14T16:09:15.848681-07:00 7767 0 main.go:307] Terminating.                                {"returnCode":0}
WARNING:annotated_run:Could not bootstrap LogDog: Failed to open bootstrap result file [/b/build/slave/linux/build/.recipe_runtime/tmpOFwWZZ/bootstrap_result.json]: [Errno 2] No such file or directory: '/b/build/slave/linux/build/.recipe_runtime/tmpOFwWZZ/bootstrap_result.json'
INFO:annotated_run:Not using LogDog. Invoking `recipes.py` directly.
INFO:root:Freshening repository https://chromium.googlesource.com/chromium/tools/depot_tools.git in /b/build/scripts/slave/.recipe_deps/depot_tools
INFO:root:Running: ['git', '-C', '/b/build/scripts/slave/.recipe_deps/depot_tools', 'config', 'remote.origin.url']
INFO:root:Running: ['git', '-C', '/b/build/scripts/slave/.recipe_deps/depot_tools', 'rev-parse', '-q', '--verify', '26f3e4eecb09d2608b1ac304de9d5d3c68de67ce^{commit}']
INFO:root:Running: ['git', '-C', '/b/build/scripts/slave/.recipe_deps/depot_tools', 'reset', '-q', '--hard', '26f3e4eecb09d2608b1ac304de9d5d3c68de67ce']
INFO:root:Freshening repository https://chromium.googlesource.com/external/github.com/luci/recipes-py.git in /b/build/scripts/slave/.recipe_deps/recipe_engine
INFO:root:Running: ['git', '-C', '/b/build/scripts/slave/.recipe_deps/recipe_engine', 'config', 'remote.origin.url']
INFO:root:Running: ['git', '-C', '/b/build/scripts/slave/.recipe_deps/recipe_engine', 'rev-parse', '-q', '--verify', 'd6fea3edecb31f2f87e8551f7ca4cdaafef731f8^{commit}']
INFO:root:Running: ['git', '-C', '/b/build/scripts/slave/.recipe_deps/recipe_engine', 'reset', '-q', '--hard', 'd6fea3edecb31f2f87e8551f7ca4cdaafef731f8']
INFO:root:Freshening repository https://chromium.googlesource.com/external/github.com/luci/recipes-py.git in /b/build/scripts/slave/.recipe_deps/recipe_engine
INFO:root:Running: ['git', '-C', '/b/build/scripts/slave/.recipe_deps/recipe_engine', 'config', 'remote.origin.url']
INFO:root:Running: ['git', '-C', '/b/build/scripts/slave/.recipe_deps/recipe_engine', 'rev-parse', '-q', '--verify', 'd6fea3edecb31f2f87e8551f7ca4cdaafef731f8^{commit}']
INFO:root:Running: ['git', '-C', '/b/build/scripts/slave/.recipe_deps/recipe_engine', 'reset', '-q', '--hard', 'd6fea3edecb31f2f87e8551f7ca4cdaafef731f8']
Traceback (most recent call last):
  File "/b/build/scripts/slave/.recipe_deps/recipe_engine/recipes.py", line 443, in <module>
    ret = main()
  File "/b/build/scripts/slave/.recipe_deps/recipe_engine/recipes.py", line 412, in main
    return run(package_deps, args)
  File "/b/build/scripts/slave/.recipe_deps/recipe_engine/recipes.py", line 117, in run
    properties = get_properties_from_file(args.properties_file)
  File "/b/build/scripts/slave/.recipe_deps/recipe_engine/recipes.py", line 102, in get_properties_from_file
    properties_file = sys.stdin if filename == '-' else open(filename)
IOError: [Errno 2] No such file or directory: '/b/build/slave/linux/build/.recipe_runtime/tmpOFwWZZ/recipe_properties.json'
program finished with exit code 1
elapsedTime=338.290756
 
Cc: phajdan.jr@chromium.org d...@chromium.org
 Issue 620176  has been merged into this issue.

Comment 2 by d...@chromium.org, Jun 15 2016

Cc: hinoka@chromium.org
"bot_update" is nuking all directories underneath "/b/build/slave/<slavename>/build", including the recipe tempdir (".../build/.recipe_runtime"). AFAIK this is new.

+hinoka@, any idea why this would be happening? None of the relevant code bases have had checkins recently...
Project Member

Comment 3 by bugdroid1@chromium.org, Jun 15 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/ed82e67abdd60ce13f7bdae56d648bd62776f0e3

commit ed82e67abdd60ce13f7bdae56d648bd62776f0e3
Author: dnj <dnj@google.com>
Date: Wed Jun 15 01:46:48 2016

annotated_run: Use builder directory for temp.

bot_update assumes that it owns the build directory
(/b/build/slave/<slavename>/build), and occasionally nukes directories
underneath of it. We are currently using the build directory for
temporary files, and they are being destroyed.

Instead, use the builder directory for temporary files. This should
protect it from bot_update.

NOPRESUBMIT=true
NOTRY=true
BUG= chromium:620158 
TEST=unittest
  - Ran unittest, seems reasonable.

Review-Url: https://codereview.chromium.org/2062293002

[modify] https://crrev.com/ed82e67abdd60ce13f7bdae56d648bd62776f0e3/scripts/slave/annotated_run.py
[modify] https://crrev.com/ed82e67abdd60ce13f7bdae56d648bd62776f0e3/scripts/slave/unittests/annotated_run_test.py

Comment 4 by d...@chromium.org, Jun 15 2016

Labels: -Pri-1 Pri-0
Owner: d...@chromium.org
Status: Fixed (was: Untriaged)
Well, the patch is doing what it's supposed to be doing. No new builds have failed. It's hard to tell if this fixed anything, but I'm going to optimistically mark that it did. Please re-open if this is observed again.

Comment 5 by jam@chromium.org, Jun 15 2016

Thanks for the quick fix.

Curious when this bug started? i.e. if this has been going on for a while, how could we have caught it earlier?
Project Member

Comment 6 by bugdroid1@chromium.org, Jun 15 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/4f0984967563cb3f9a625855d8b73105775afde1

commit 4f0984967563cb3f9a625855d8b73105775afde1
Author: recipe-roller <recipe-roller@chromium.org>
Date: Wed Jun 15 10:32:02 2016

Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

build:
  https://crrev.com/ed82e67abdd60ce13f7bdae56d648bd62776f0e3 annotated_run: Use builder directory for temp. (dnj@google.com)

R=dnj@google.com
BUG= chromium:620158 

TBR=martiniss@chromium.org,phajdan.jr@chromium.org

Review-Url: https://codereview.chromium.org/2068973002

[modify] https://crrev.com/4f0984967563cb3f9a625855d8b73105775afde1/infra/config/recipes.cfg

Project Member

Comment 7 by bugdroid1@chromium.org, Jun 15 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/bed5ced56d75a4a63d5b4e4241e4e9aa51e2de31

commit bed5ced56d75a4a63d5b4e4241e4e9aa51e2de31
Author: recipe-roller <recipe-roller@chromium.org>
Date: Wed Jun 15 10:46:14 2016

Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

build:
  https://crrev.com/ed82e67abdd60ce13f7bdae56d648bd62776f0e3 annotated_run: Use builder directory for temp. (dnj@google.com)
  https://crrev.com/3ff29095a3ded4744b3dc5c7367956471a9010ba libyuv: Set android_full_debug=1 in GYP_DEFINES for Android x86. (kjellander@chromium.org)
  https://crrev.com/98122d8810af966ebd7da62629c688c1ab6c8c9a remote_run: use /b as base directory for temporary files when available (phajdan.jr@chromium.org)

R=kjellander@chromium.org,dnj@google.com,fbarchard@chromium.org,phajdan.jr@chromium.org
BUG= 593999 , libyuv:602 , chromium:620158 

TBR=martiniss@chromium.org,phajdan.jr@chromium.org

Review-Url: https://codereview.chromium.org/2068043003

[modify] https://crrev.com/bed5ced56d75a4a63d5b4e4241e4e9aa51e2de31/infra/config/recipes.cfg

Comment 8 by jam@chromium.org, Jun 15 2016

ping re question in comment 5 (it's obscured by the recipe roller automated cls. anyway to not show these in bugs (i.e. by not listing bugs in their desc)? i find them only noise)

Comment 9 by hinoka@chromium.org, Jun 15 2016

Re #2: Bot_update assumes it owns /b/build/slave/<builder>/build, and will nuke it at the first sign of detecting trouble (ie something is unrecoverably corrupt).  
re #8, comment about recipe roller cls, I've made  bug 620511 
Project Member

Comment 11 by bugdroid1@chromium.org, Jun 17 2016

Sign in to add a comment