chroot fails to unmount in quawks and nyan_big |
|
Issue descriptionThis partly broke the last CQ run: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8923991602867193632 Error: 06:26:11: INFO: Cleaning up chroot at /b/swarming/w/ir/cache/cbuild/repository/chroot umount: /b/swarming/w/ir/cache/cbuild/repository/chroot: device is busy. (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1)) 06:26:11: INFO: Waiting for ts_mon flushing process to finish... 06:26:53: INFO: Finished waiting for ts_mon process. cbuildbot_launch: Unhandled exception: Traceback (most recent call last): File "/b/swarming/w/ir/kitchen-workdir/chromite/scripts/cbuildbot_launch", line 170, in <module> DoMain() File "/b/swarming/w/ir/kitchen-workdir/chromite/scripts/cbuildbot_launch", line 166, in DoMain commandline.ScriptWrapperMain(FindTarget) File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/commandline.py", line 912, in ScriptWrapperMain ret = target(argv[1:]) File "/b/swarming/w/ir/kitchen-workdir/chromite/scripts/cbuildbot_launch.py", line 498, in main return _main(options, argv) File "/b/swarming/w/ir/kitchen-workdir/chromite/scripts/cbuildbot_launch.py", line 481, in _main CleanupChroot(buildroot) File "/b/swarming/w/ir/kitchen-workdir/chromite/scripts/cbuildbot_launch.py", line 72, in wrapped_functor result = functor(*args, **kwargs) File "/b/swarming/w/ir/kitchen-workdir/chromite/scripts/cbuildbot_launch.py", line 399, in CleanupChroot cros_sdk_lib.CleanupChrootMount(chroot_dir, delete=False) File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/timeout_util.py", line 191, in TimeoutWrapper return func(*args, **kwargs) File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/cros_sdk_lib.py", line 394, in CleanupChrootMount osutils.UmountTree(chroot) File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/osutils.py", line 901, in UmountTree UmountDir(mount_pt, lazy=False, cleanup=False) File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/osutils.py", line 863, in UmountDir runcmd(cmd, print_cmd=False) File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/cros_build_lib.py", line 321, in SudoRunCommand return RunCommand(sudo_cmd, **kwargs) File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/cros_build_lib.py", line 646, in RunCommand raise RunCommandError(msg, cmd_result) chromite.lib.cros_build_lib.RunCommandError: return code: 1; command: sudo -- umount -d /b/swarming/w/ir/cache/cbuild/repository/chroot cmd=['sudo', '--', 'umount', '-d', '/b/swarming/w/ir/cache/cbuild/repository/chroot']
,
Jan 19
(3 days ago)
prob should enumerate chromite.lib.osutils.UmountDir to try and collect details if it failed with EBUSY like this. glancing at the logs, nothing stuck out as what had leaked and caused the issue. |
|
►
Sign in to add a comment |
|
Comment 1 by semenzato@chromium.org
, Jan 18 (4 days ago)