New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 754014 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Aug 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

cros_sdk failing on thin pool creation

Project Member Reported by bmgordon@chromium.org, Aug 9 2017

Issue description

$ cros_sdk -- ./build_packages --board=${BOARD}
09:32:31: NOTICE: Downloading SDK tarball...
09:32:32: NOTICE: Creating chroot. This may take a few minutes...
  /dev/mapper/cros_home+zzh+chromium+chroot_001-thinpool_tmeta: open failed: No such file or directory
  Failed to activate new LV.
Running ['/home/zzh/chromium/src/scripts/sdk_lib/make_chroot.sh', '--stage3_path', '/home/zzh/chromium/.cache/sdks/cros-sdk-2017.08.08.001011.tar.xz', '--chroot', '/home/zzh/chromium/chroot', '--cache_dir', '/home/zzh/chromium/.cache', '--useimage'] failed!

The line that failed is
# lvcreate -q -L 499G -T "${chroot_vg}/thinpool" -V500G -n chroot --noudevsync >/dev/null
in make_chroot.sh.

 
Could you provide the following from your host machine:

* OS name and version
* Output from sudo lvs --version
* Output from thin_check --verson

For example:

Ubuntu 14.04

$ sudo lvs --version
  LVM version:     2.02.98(2) (2012-10-15)
  Library version: 1.02.77 (2012-10-15)
  Driver version:  4.34.0

$ thin_check --version
0.2.8

Comment 2 by perley@google.com, Aug 9 2017

Goobuntu rodete 201707RD5

$ sudo lvs --version
  LVM version:     2.02.168(2) (2016-11-30)
  Library version: 1.02.137 (2016-11-30)
  Driver version:  4.35.0

$ thin_check --version
0.6.1

Comment 3 by zz...@cornell.edu, Aug 9 2017

OS Name and version:
CentOS 7.0
Linux kernel 3.10

$ sudo lvs --version
  LVM version:     2.02.166(2)-RHEL7 (2016-09-28)
  Library version: 1.02.135-RHEL7 (2016-09-28)
  Driver version:  4.34.0

]$ thin_check --version
0.6.3-1.el7

Hmm, those are both quite a bit newer than mine.  I'll get myself an updated VM and make sure newer LVM isn't behaving differently.  As a workaround, you should be able to use --nouse-image until this is fixed.
I've reproduced this on CentOS and Fedora VMs.  It looks like on newer versions of lvm, the --noudevsync flag causes more trouble than it does on Ubuntu 14.04.  I've been working on a different implementation of the chroot setup that should take care of this problem as well.

It's not quite done, but I think it's getting close enough to be worth wider visibility.  If you'd like to patch https://crrev.com/c/608585 and re-run cros_sdk, I think it will work correctly for you.  If you give it a try, please confirm one way or the other.

Comment 6 by zz...@cornell.edu, Aug 11 2017

I'm trying this patch, but it hangs forever in
09:37:50: INFO: chroot lock: blocking while taking write lock
Anyway to get around this?

Thanks!
Do you have another shell sitting inside the chroot (in a different terminal or screen session)?  That's supposed to happen if one cros_sdk is trying to modify the chroot while another one has it open.  If so, just exit from the shell inside the chroot and the blocked cros_sdk should proceed.

Comment 8 by zz...@cornell.edu, Aug 11 2017

I killed the process inside the chroot, and now everything is good. The patch works.

Thanks!
Project Member

Comment 9 by bugdroid1@chromium.org, Aug 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/abb3e3768ca5b6a7ea6b03c8995bfdf09f9624bb

commit abb3e3768ca5b6a7ea6b03c8995bfdf09f9624bb
Author: Benjamin Gordon <bmgordon@chromium.org>
Date: Thu Aug 17 18:31:48 2017

cros_sdk: Switch chroot image maintenance to python

The python code to create/mount/delete chroot.img was introduced in
https://crrev.com/c/611156.  This CL uses this to move the chroot.img
setup from make_chroot.sh into cros_sdk.  Having the setup in
make_chroot.sh was originally desirable to keep all the chroot
maintenance together, but the complexity of keeping the outer and inner
mount namespaces synced up had too many problems.

By moving chroot.img setup into cros_sdk, we eliminate:
  1. Tracking the parent namespace and exiting from the child namespace.
  2. Moving mounts between namespaces.
  3. Passing --noudevsync to the lvm commands.
  4. The whole chroot.build temporary directory.

I will send another CL to remove chroot.img setup from make_chroot.sh
once this CL has gone through the CQ.

BUG=chromium:730144,  chromium:754014 , chromium:752562
TEST=Created and removed a bunch of chroots; trybots.

Change-Id: Ie461def2b9d3dbb3215e844ceb4e281e905d5ff8
Reviewed-on: https://chromium-review.googlesource.com/614761
Commit-Ready: Benjamin Gordon <bmgordon@chromium.org>
Tested-by: David Schneider <dnschneid@chromium.org>
Tested-by: Benjamin Gordon <bmgordon@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>

[modify] https://crrev.com/abb3e3768ca5b6a7ea6b03c8995bfdf09f9624bb/scripts/cros_sdk.py

Status: Fixed (was: Assigned)

Sign in to add a comment