New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 676528 link

Starred by 1 user

Issue metadata

Status: Archived
Owner:
Last visit > 30 days ago
Closed: Jan 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug



Sign in to add a comment

udev-225 adds 6+ MB file /etc/udev/hwdb.bin

Project Member Reported by mka@chromium.org, Dec 22 2016

Issue description

We are in the process of upgrading udev to v225 (https://crosbug.com/p/59259). A CQ run failed due to the image size being exceeded ( https://crbug.com/676152 ) with udev-225. udev isn't the primary problem here, however it was found that it adds the 6+ MB file /etc/udev/hwdb.bin.

It is not entirely clear how hwdb is used on CrOS and if this large database is needed. There are a few platforms (ninja, buddy and sumo) which explicitly add an hwdb file to create a special keyboard mapping. 

It seems we can't get rid completely of hwdb, but lets see if we can reduce the size of /etc/udev/hwdb.bin. Potentially the largest databases with PCI and USB information are not needed.
 

Comment 1 by mka@chromium.org, Dec 22 2016

Cc: vapier@chromium.org
Some interesting findings:

1) The hwdb.bin size doesn't really depend on udev-225 being used by the target, but by the udevadm version on the build host. With udev-225 unmasked apparently udev in the chroot gets updated under certain conditions.

2) The reason for 1) is that udevadm-208 looks for .hwdb files in /lib64/udev/hwdb.d/ (which does not exist), while udevadm-225 scans /lib/udev/hwdb.d/ which contains .hwdb files (from package sys-apps/hwids)

3) hwdb.bin is not only generated/updated by the udev package but also by the hwids one

Since the information from the hwids .hwdb files doesn't make it into hwdb.bin with udev-208 I deduce it isn't actually used by CrOS.

The following packages depend on hwids (for gru):

sys-apps/lshw
sys-apps/pciutils
sys-apps/usbutils
sys-fs/udev

Besides the hwdb info hwids provides /usr/share/misc/pci.ids
/usr/share/misc/usb.ids which are used by (at least) lsusb and lspci. This means removing hwids from the dependency list is not an option.

If we are concerned about the use of disk space by hwdb.bin (and /lib/udev/hwdb.d/) I suggest to remove the .hwdb files from the hwids package. The current hwids ebuild comes from portage-stable, to remove the files I suppose the right thing to do would be to create a hwids ebuild in chromiumos-overlay.

Comment 2 by vapier@chromium.org, Dec 22 2016

the hwdb provides a lot of human readable vendor/product strings that we don't care about and could punt on.  but they also provide some details which i think might matter to us.

PCI/USB devices get class/subclass/interface details.  these might also be human readable strings though ... not 100% sure.

input devices (mouse/keyboard/etc...) include attributes that i don't think exist anywhere else.

i think we should figure out how to customize hwids so that it omits vendor/product details, preferably by a USE flag.  this would cascade down into now just the hwdb files, but also the pci/usb ids files.  that would mean truncating them, but oh well.

Comment 3 by mka@chromium.org, Dec 22 2016

vapier@, do you have more information on the actual use of hwdb information in CrOS?

My point is that apparently nobody noticed that it wasn't there in the first place, if we enable it selectively it would be good to know it is actually used. At least for lsusb and lspci the information gathered from sysfs and usb/pci ids looks sufficient to me.

I'm not sure I would personally want to get into filtering pci/usb id files at this point. My main goal is to switch to udev-225, on that course I already got side-tracked quite a bit with rather unrelated stuff and would prefer not to perpetuate that eternally :) We can create a separate task for that if it is deemed worthwhile.

Comment 4 by vapier@chromium.org, Dec 22 2016

lack of bug reports of misbehavior of devices isn't evidence that bugs don't exist.  based on the attrs i saw on the inputs, i'd hypothesis that those devices are not working well with CrOS when absent, although some basic functionality might exist in there.  considering we're talking about USB/hotplug devices, it's not like we can just say "no Chromebook ships with those devices, so we don't care if they work".
Project Member

Comment 5 by bugdroid1@chromium.org, Jan 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/47448cfb938f9e31a8157a3840864f24b8cc7abf

commit 47448cfb938f9e31a8157a3840864f24b8cc7abf
Author: Matthias Kaehlcke <mka@chromium.org>
Date: Thu Dec 22 19:05:03 2016

sys-apps/hwids: Clone hwids-20150717 from portage-stable

BUG= chromium:676528 
TEST=emerge-gru hwids
  "Emerging (1 of 1) sys-apps/hwids-20150717::chromiumos"

Change-Id: Iee48b5ba9a7a6b5a267bb3787836bad1ba569226
Reviewed-on: https://chromium-review.googlesource.com/423194
Commit-Ready: Matthias Kaehlcke <mka@chromium.org>
Tested-by: Matthias Kaehlcke <mka@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[add] https://crrev.com/47448cfb938f9e31a8157a3840864f24b8cc7abf/sys-apps/hwids/metadata.xml
[add] https://crrev.com/47448cfb938f9e31a8157a3840864f24b8cc7abf/sys-apps/hwids/hwids-20150717.ebuild
[add] https://crrev.com/47448cfb938f9e31a8157a3840864f24b8cc7abf/sys-apps/hwids/Manifest

Project Member

Comment 6 by bugdroid1@chromium.org, Jan 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/94901adf4c99f6e6a99380595206dde8e1a4b891

commit 94901adf4c99f6e6a99380595206dde8e1a4b891
Author: Matthias Kaehlcke <mka@chromium.org>
Date: Fri Dec 23 00:23:59 2016

sys-apps/hwids: Add use flag 'hwdb-lite' to reduce hwdb size

With our current udev-208 the hwdb information from this
package isn't actually included in the binary database
/etc/udev/hwdb.bin. This changes with udev-225 and the file
now occupies 6+ MB of storage. Some larger items like PCI/USB
vendor and model information or OUI isn't really needed on
CrOS.

Add the new use flag 'hwdb-lite' which excludes some
informations from hwdb. The flag is enabled by default.

BUG= chromium:676528 
TEST=emerge-gru hwids
  equery-gru f hwids | grep 20-usb-vendor-model.hwdb
    => no match
  USE="-hwids-lite" emerge-gru hwids
  equery-gru f hwids | grep 20-usb-vendor-model.hwdb
    => /lib/udev/hwdb.d/20-usb-vendor-model.hwdb

Change-Id: I1ab405b952d4251028422e7f8458c5bcc9e40e67
Reviewed-on: https://chromium-review.googlesource.com/423195
Commit-Ready: Matthias Kaehlcke <mka@chromium.org>
Tested-by: Matthias Kaehlcke <mka@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/94901adf4c99f6e6a99380595206dde8e1a4b891/sys-apps/hwids/metadata.xml
[modify] https://crrev.com/94901adf4c99f6e6a99380595206dde8e1a4b891/sys-apps/hwids/hwids-20150717.ebuild
[add] https://crrev.com/94901adf4c99f6e6a99380595206dde8e1a4b891/sys-apps/hwids/hwids-20150717-r1.ebuild

Comment 7 by mka@chromium.org, Jan 11 2017

Status: Fixed (was: Assigned)

Comment 8 by dchan@google.com, Mar 4 2017

Labels: VerifyIn-58

Comment 9 by dchan@google.com, Apr 17 2017

Labels: VerifyIn-59

Comment 10 by dchan@google.com, May 30 2017

Labels: VerifyIn-60
Labels: VerifyIn-61

Comment 12 by dchan@chromium.org, Oct 14 2017

Status: Archived (was: Fixed)

Sign in to add a comment