New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 764150 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: May 2018
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Leaking loopbacks locally

Project Member Reported by dgarr...@chromium.org, Sep 12 2017

Issue description

I keep finding loopback devices that appear to be unused, but which I can't delete.


$sudo losetup -a /dev/loop0: [fc02]:15204431 (/usr/local/google/home/dgarrett/sand/quick/chroot.img)
$sudo losetup -d /dev/loop0
$sudo losetup -a /dev/loop0: [fc02]:15204431 (/usr/local/google/home/dgarrett/sand/quick/chroot.img)

$cros_sdk --delete

18:05:47: NOTICE: Deleting chroot.

$sudo losetup -a
/dev/loop0: [fc02]:15204431 (/usr/local/google/home/dgarrett/sand/quick/chroot.img)

 
Are you able to reproduce this reliably?  I have a feeling it's the same underlying cause as crbug.com/752562, but I haven't been able to get it to happen consistently enough to understand what's going on.
Most likely, it happens when I start run_tests, realize I meant run_tests
-q, then kill it (which is hard to do). I'm pretty sure there are leaked
processes, but I don't seem to be able to find them.
Project Member

Comment 3 by bugdroid1@chromium.org, May 24 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/04b73c195727bb3bc8e77a9c48b0a2cb6680ac0d

commit 04b73c195727bb3bc8e77a9c48b0a2cb6680ac0d
Author: Benjamin Gordon <bmgordon@chromium.org>
Date: Thu May 24 22:44:35 2018

cros_sdk_lib: Rescan loopbacks during chroot setup

lvmetad is supposed to rescan devices for LVM metadata as they come and
go, but sometimes it misses an event.  When this happens, it can keep
stale data about our chroot loopback devices.  This later shows up as
mysterious device-mapper errors when trying to re-mount an existing
chroot image or re-use a loopback device.

To correct for this, force rescanning loopback devices when we attach or
detach them during chroot setup and teardown.  pvscan --cache will fail
if use_lvmetad isn't turned on in the global config, but it fails
quickly without any side effects, so we just run it and ignore the exit
code rather than trying to determine if it's strictly necessary.

BUG=chromium:752562,  chromium:764150 ,  chromium:839581 
TEST=cros_sdk --replace 100 times in a loop; cros_sdk_unittest passes

Change-Id: I7504036a3af8273a27bf279b907a01f2087b99be
Reviewed-on: https://chromium-review.googlesource.com/1058071
Commit-Ready: Benjamin Gordon <bmgordon@chromium.org>
Tested-by: Benjamin Gordon <bmgordon@chromium.org>
Reviewed-by: Benjamin Gordon <bmgordon@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/04b73c195727bb3bc8e77a9c48b0a2cb6680ac0d/lib/cros_sdk_lib.py
[modify] https://crrev.com/04b73c195727bb3bc8e77a9c48b0a2cb6680ac0d/lib/cros_sdk_lib_unittest.py

Project Member

Comment 4 by bugdroid1@chromium.org, May 25 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/935b926c05a8870c2a155a2b9bb5a512b57529b9

commit 935b926c05a8870c2a155a2b9bb5a512b57529b9
Author: chromite-chromium-autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri May 25 01:32:28 2018

Roll src/third_party/chromite/ b887ecda5..04b73c195 (2 commits)

https://chromium.googlesource.com/chromiumos/chromite.git/+log/b887ecda57d6..04b73c195727

$ git log b887ecda5..04b73c195 --date=short --no-merges --format='%ad %ae %s'
2018-05-11 bmgordon cros_sdk_lib: Rescan loopbacks during chroot setup
2018-05-23 ayatane Run tests when building infra Go binaries

Created with:
  roll-dep src/third_party/chromite
BUG=chromium:752562, chromium:764150 , chromium:839581 ,chromium:None


The AutoRoll server is located here: https://chromite-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=chrome-os-gardeners@chromium.org

Change-Id: I9a2dd44468169b73ae835707be14812a876adaeb
Reviewed-on: https://chromium-review.googlesource.com/1072805
Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#561725}
[modify] https://crrev.com/935b926c05a8870c2a155a2b9bb5a512b57529b9/DEPS

Status: Fixed (was: Untriaged)
I think this is fixed.  With crrev.com/c/1058071, I've been able to run the cros_sdk unit tests a couple of dozen times without any stuck loopbacks.  Feel free to reopen if it happens again.

Sign in to add a comment