Random (?) rules.gni error in android builds in buildbot |
|||||||
Issue description
I've had this error or similar ones a couple of times the last day on bots (links below):
compute_grit_inputs_for_analyze=true""" to /b/c/b/android_n5x_swarming_rel/src/out/Release/args.gn.
/b/c/b/android_n5x_swarming_rel/src/buildtools/linux64/gn gen //out/Release --check
-> returned 1
ERROR at //build/config/android/rules.gni:3161:32: Could not read file.
_scanned_files = read_file(_info_path, "scope")
^---------
I resolved this to "/b/c/b/android_n5x_swarming_rel/src/third_party/android_deps/repository/android_arch_lifecycle_runtime/android_arch_lifecycle_runtime.info".
See //third_party/android_deps/BUILD.gn:149:1: whence it was called.
android_aar_prebuilt("android_arch_lifecycle_runtime_java") {
^------------------------------------------------------------
See //build/config/android/internal_rules.gni:117:23: which caused the file to be included.
deps += [ "${_target_label}__build_config" ]
^-----------------------------
Example: https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/418142
Example #2: https://ci.chromium.org/buildbot/tryserver.chromium.android/android_compile_dbg/495013
Example #3: https://ci.chromium.org/buildbot/tryserver.chromium.android/cast_shell_android/490127
Three different builders, but they were all android related.
,
May 11 2018
It looks like android_arch_lifecycle_runtime.info is a symlink to a cipd file, meaning that cipd was on the hook for fetching it and ensuring that it exists. I'm not sure why that wouldn't have been the case. +jbudorick, +vadimsh ?
,
May 11 2018
I looked at slave943-c4 and slave1047-c4 and I see android_arch_lifecycle_runtime.info to be just missing. Where have you seen it as a broken symlink (it would indicate a slightly different failure mode)? I think something has deleted it from the checkout (do we run "git clean"?), and cipd currently isn't aggressively restoring such deleted files ( Issue 794764 ). Broken symlink would indicate that something has deleted <gclient_root>/.cipd directory with cipd guts. This should not be happening either (and CIPD will be able to recover if the directory was fully deleted. It won't handle well if it got partially corrupted).
,
May 11 2018
I didn't log into the machines, so I don't know if the file was missing or if the symlink was missing.
,
May 15 2018
Got this on slave1047-c4 as well in https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/422278. Maybe we can reset slave943-c4 and slave1047-c4 to a good state?
,
May 18 2018
slave1047-c4 once again https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/425578 Maybe a trooper can take care of this?
,
May 18 2018
I'll respawn slave943-c4 and slave1047-c4. I'll also look into fixing it for real.
,
May 19 2018
Assign to vadimsh@ who is looking into it.
,
Jun 5 2018
I'm not longer actively looking at this.
,
Jun 7 2018
Can't find any more reoccurrences of this. Going to close it out. Pls reopen if I missed something.
,
Jun 7 2018
We did start nuking .cipd prior to bot_update earlier this week pending the addition of cipd paranoia, so we shouldn't see that particular failure mode again. |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by tandrii@chromium.org
, May 10 2018