New issue
Advanced search Search tips

Issue 923432 link

Starred by 1 user

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

chroot fails to unmount in quawks and nyan_big

Project Member Reported by evanhernandez@google.com, Jan 18 (4 days ago)

Issue description

This partly broke the last CQ run:
https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8923991602867193632

Error:

06:26:11: INFO: Cleaning up chroot at /b/swarming/w/ir/cache/cbuild/repository/chroot
umount: /b/swarming/w/ir/cache/cbuild/repository/chroot: device is busy.
        (In some cases useful info about processes that use
         the device is found by lsof(8) or fuser(1))

06:26:11: INFO: Waiting for ts_mon flushing process to finish...
06:26:53: INFO: Finished waiting for ts_mon process.
cbuildbot_launch: Unhandled exception:
Traceback (most recent call last):
  File "/b/swarming/w/ir/kitchen-workdir/chromite/scripts/cbuildbot_launch", line 170, in <module>
    DoMain()
  File "/b/swarming/w/ir/kitchen-workdir/chromite/scripts/cbuildbot_launch", line 166, in DoMain
    commandline.ScriptWrapperMain(FindTarget)
  File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/commandline.py", line 912, in ScriptWrapperMain
    ret = target(argv[1:])
  File "/b/swarming/w/ir/kitchen-workdir/chromite/scripts/cbuildbot_launch.py", line 498, in main
    return _main(options, argv)
  File "/b/swarming/w/ir/kitchen-workdir/chromite/scripts/cbuildbot_launch.py", line 481, in _main
    CleanupChroot(buildroot)
  File "/b/swarming/w/ir/kitchen-workdir/chromite/scripts/cbuildbot_launch.py", line 72, in wrapped_functor
    result = functor(*args, **kwargs)
  File "/b/swarming/w/ir/kitchen-workdir/chromite/scripts/cbuildbot_launch.py", line 399, in CleanupChroot
    cros_sdk_lib.CleanupChrootMount(chroot_dir, delete=False)
  File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/timeout_util.py", line 191, in TimeoutWrapper
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/cros_sdk_lib.py", line 394, in CleanupChrootMount
    osutils.UmountTree(chroot)
  File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/osutils.py", line 901, in UmountTree
    UmountDir(mount_pt, lazy=False, cleanup=False)
  File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/osutils.py", line 863, in UmountDir
    runcmd(cmd, print_cmd=False)
  File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/cros_build_lib.py", line 321, in SudoRunCommand
    return RunCommand(sudo_cmd, **kwargs)
  File "/b/swarming/w/ir/kitchen-workdir/chromite/lib/cros_build_lib.py", line 646, in RunCommand
    raise RunCommandError(msg, cmd_result)
chromite.lib.cros_build_lib.RunCommandError: return code: 1; command: sudo -- umount -d /b/swarming/w/ir/cache/cbuild/repository/chroot
cmd=['sudo', '--', 'umount', '-d', '/b/swarming/w/ir/cache/cbuild/repository/chroot']

 

Comment 1 by semenzato@chromium.org, Jan 18 (4 days ago)

Labels: -Pri-3 Pri-2
Raising priority since it happened in two builds.

Comment 2 by vapier@chromium.org, Jan 19 (3 days ago)

prob should enumerate chromite.lib.osutils.UmountDir to try and collect details if it failed with EBUSY like this.  glancing at the logs, nothing stuck out as what had leaked and caused the issue.

Comment 3 Deleted

Sign in to add a comment