Could you check if you have any device-mapper logs in /var/log/messages?

In the meantime, to get going again, you can either reboot or delete ~/tryjob/repository/chroot.img.  Either one should clean up whatever is wedged.

A number of messages like this that seems to coincide temporally with the times I tried running the tryjobs, to the best of my memory:

May  2 18:40:01 sharanohiar.mtv.corp.google.com kernel: [1987936.816729] device-mapper: table: 253:12: thin: Couldn't open thin internal device
May  2 18:40:01 sharanohiar.mtv.corp.google.com kernel: [1987936.816735] device-mapper: ioctl: error adding target to table
May  2 18:40:01 sharanohiar.mtv.corp.google.com kernel: [1987936.960710] device-mapper: table: 253:12: thin: Couldn't open thin internal device
May  2 18:40:01 sharanohiar.mtv.corp.google.com kernel: [1987936.960716] device-mapper: ioctl: error adding target to table
May  2 18:40:01 sharanohiar.mtv.corp.google.com kernel: [1987937.076503] device-mapper: table: 253:12: thin: Couldn't open thin internal device
May  2 18:40:01 sharanohiar.mtv.corp.google.com kernel: [1987937.076509] device-mapper: ioctl: error adding target to table

Deleting chroot.img doesn't help

Will try rebooting later,  I feel like that has a high chance of helper.

I have gotten into a state where a reboot was required, but didn't have time to try and debug.

Since then, things have been stable for me.

Issue persists across a reboot.  cros_sdk works normally for my normal chroot.

Hopefully someone has some insight, because it sounds like I'm going to actually have to take some time to dig into this myself.

I have this on my list to investigate, but it's been stubbornly hard to narrow down.  If you do spend some time on it, please share whatever you learn here.

Moving this out of the untriaged queue

I have spotted this error on a builder now.

https://luci-milo.appspot.com/buildbot/chromeos/nyan-full-compile-paladin/12641

https://logs.chromium.org/v/?s=chromeos%2Fbb%2Fchromeos%2Fnyan-full-compile-paladin%2F12641%2F%2B%2Frecipes%2Fsteps%2FCleanUp%2F0%2Fstdout

I am also still experiencing this randomly, but I haven't had time or the need to dig into it.

I have crrev.com/c/1058071 out for review that I think will address this.

Re #8 upon closer inspection, the error that appeared in the build seems to be different.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/04b73c195727bb3bc8e77a9c48b0a2cb6680ac0d

commit 04b73c195727bb3bc8e77a9c48b0a2cb6680ac0d
Author: Benjamin Gordon <bmgordon@chromium.org>
Date: Thu May 24 22:44:35 2018

cros_sdk_lib: Rescan loopbacks during chroot setup

lvmetad is supposed to rescan devices for LVM metadata as they come and
go, but sometimes it misses an event.  When this happens, it can keep
stale data about our chroot loopback devices.  This later shows up as
mysterious device-mapper errors when trying to re-mount an existing
chroot image or re-use a loopback device.

To correct for this, force rescanning loopback devices when we attach or
detach them during chroot setup and teardown.  pvscan --cache will fail
if use_lvmetad isn't turned on in the global config, but it fails
quickly without any side effects, so we just run it and ignore the exit
code rather than trying to determine if it's strictly necessary.

BUG=chromium:752562,  chromium:764150 ,  chromium:839581 
TEST=cros_sdk --replace 100 times in a loop; cros_sdk_unittest passes

Change-Id: I7504036a3af8273a27bf279b907a01f2087b99be
Reviewed-on: https://chromium-review.googlesource.com/1058071
Commit-Ready: Benjamin Gordon <bmgordon@chromium.org>
Tested-by: Benjamin Gordon <bmgordon@chromium.org>
Reviewed-by: Benjamin Gordon <bmgordon@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/04b73c195727bb3bc8e77a9c48b0a2cb6680ac0d/lib/cros_sdk_lib.py
[modify] https://crrev.com/04b73c195727bb3bc8e77a9c48b0a2cb6680ac0d/lib/cros_sdk_lib_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/935b926c05a8870c2a155a2b9bb5a512b57529b9

commit 935b926c05a8870c2a155a2b9bb5a512b57529b9
Author: chromite-chromium-autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri May 25 01:32:28 2018

Roll src/third_party/chromite/ b887ecda5..04b73c195 (2 commits)

https://chromium.googlesource.com/chromiumos/chromite.git/+log/b887ecda57d6..04b73c195727

$ git log b887ecda5..04b73c195 --date=short --no-merges --format='%ad %ae %s'
2018-05-11 bmgordon cros_sdk_lib: Rescan loopbacks during chroot setup
2018-05-23 ayatane Run tests when building infra Go binaries

Created with:
  roll-dep src/third_party/chromite
BUG=chromium:752562, chromium:764150 , chromium:839581 ,chromium:None


The AutoRoll server is located here: https://chromite-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=chrome-os-gardeners@chromium.org

Change-Id: I9a2dd44468169b73ae835707be14812a876adaeb
Reviewed-on: https://chromium-review.googlesource.com/1072805
Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#561725}
[modify] https://crrev.com/935b926c05a8870c2a155a2b9bb5a512b57529b9/DEPS