Actually, as I was thinking about this more I realized that we probably need to go back to having the kernel install firmware files in kernel 4.4 until we solve this better.  See  bug #770234 

https://chromium.googlesource.com/chromiumos/third_party/linux-firmware/+log/3fe01af27849c4 seems to be some old branch(es).

This shows 160 additional commits, including many recent ones:
https://chromium.googlesource.com/chromiumos/third_party/linux-firmware/+log/3fe01af27849c4..master

I'm starting on this as payment for wanting to eat 1.7 MB to get stack crawls symbolized.  It actually looks like we're stupider than I originally thought.  We purposely tell the kernel to install firmware even if we have no driver to support that firmware.  Awesome.

...but there's at least one case (keyspan USB devices) where we appear to have a driver and we really do need the firmware.  I'll go through them one by one, but I sorta expect that the in-kernel firmware that we actually need to be near zero.

We are running out of root file system space for a number of images ... samus (crbug.com/775061), cyan (crbug.com/775872), and possibly eve. Unless there are other easy targets to drop this may become a bit more important.

I'll try to have something to review for at least kernel 4.4 tomorrow. I
expect several MB of savings.

Wonder how much we could save by enabling module compression (CONFIG_MODULE_COMPRESS). Any idea ?

Follow-up on #4: Free space in rootfs is actually 50-100 MB. The running-out-of-space problem is in the factory image, and there the problem is different. For whatever reason the root fs in the factory image has 250MB of 390 MB free space in build r10030, and ran out of space in r10032.

Module compression forked to bug #777204

...and firmware CL gong in at <https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/729700>

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/bdf884f930b616eef6bbf93b1a00ccb2ff28b805

commit bdf884f930b616eef6bbf93b1a00ccb2ff28b805
Author: Douglas Anderson <dianders@chromium.org>
Date: Sun Oct 22 19:34:30 2017

linux-firmware: Clean linux-firmware (removing most of the files)

Some components on a computer (like a display controller, or a GPU, or
an Ethernet adapter, or ...) need firmware loaded onto them before
they can function properly.  This firmware is known in our build
system as "linux-firmware".

Except for some special cases[1], linux-firmware is stored on the root
filesystem in "/lib/firmware" and is loaded on demand when the driver
for the component needs it.

Historically, linux-firmware got installed onto the root filesystem
from 3 primary places:
1. The Linux kernel has had some firmware checked into its git tree
   under the "firmware/" directory.  During the kernel we ran "make
   firmware_install" to get the kernel to install firmware into
   "/lib/firmware".
2. There is a separate git tree called "linux-firmware" that is a fork
   of the firmware directory from the kernel.  We cloned that to
   "src/third_party/linux-firmware" back in 2013 and then we've been
   picking specific commits back.  We don't install _all_ firmware
   here, we pick the ones to install using USE flags.  Actually, if we
   did install everything we'd collide with the firmware that the
   kernel installs since this is a fork of the kernel's firmware
   directory.  We only install very specific things.
3. Some of our packages install firmware into /lib/firmware directly.
   This firmware isn't located anywhere else.

One major thing has changed recently: the Linux kernel has removed the
firmware from its git tree as of v4.14.  This means we need to figure
out what to do.  When we're using a kernel v4.14 or newer we need to
find out how we're going to get all that firmware that used to be
installed by the kernel.  ...and we need to resolve versioning since
we only have one linux-firmware directory and lots of kernel versions.
Yikes, we're in trouble...

...or so we thought...

I spent a bunch of time looking at all the firmware files that were
installed, and here's what I found:

The first thing I realized was that "make firmware_install" was
probably the wrong thing to be doing all along.  Running "make
firmware_install" installs _every single_ firmware file even if there
is no driver enabled for it (!).  If we only want firmware files that
are actually needed then "make modules_install" is the right thing to
do.  It turns out that for ARM devices this installs _nothing_.
...and for x86 devices this installs "e100" and "tigon" firmware.
That's it.

I then crawled through _all_ of the firmware files on kernel 4.4 and
found that, for the most part, the kernel's build system was doing the
right thing.  No files were installed on ARM and very few on x86
because, for the most part, we don't have the CONFIG options turned on
for all those drivers that need firmware.  So we were installing a lot
of firmware that we had no drivers for (!).

I say "for the most part" because I found that the keyspan USB driver
had firmware associated with it and was configured (as a module) but
"make modules_install" wasn't installing firmware for it.  As far as I
can understand this was due to someone being confused when the
firmware was added to the kernel.  I believe that we _do_ still need
the keyspan USB firmware in order to make the keyspan USB driver
useful.  I have no idea if anyone _needs_ this driver, but we've got
it turned on so we should probably keep the firmware.

Aside from the keyspan USB firmware (and the e100 and tigon firmware
for x86), I confirmed that all the rest of the drivers that used
firmware from the kernel "firmware" directory were disabled on kernel
4.4 and thus useless.

I dug a little more and I believe that on all our older kernels (3.8,
3.10, 3.14, 3.18) all the (non-keyspan) firmware is either properly
tagged to get installed with "modules_install", comes from the
linux-firmware.git, or is builtin to the kernel.  AKA: aside from the
keyspan issue we've never needed "modules_install".

From all the above, I believe that it's safe to stop running
"firmware_install" for all of our kernels and simply install the
keyspan USB firmware (which hasn't changed since 2008) from the
linux-firmware git tree.  This should make all our existing kernels
happy and save a bunch of disk space.  It's also really simple.

Long term:
- On kernel 4.14+ if someone needs a firmware that used to come from
  the kernel ebuild, they'll need to:
  -> Check that the firmware in linux-firmware.git is up to date.
  -> Add a USE flag to that board enabling the firmware to get
     installed.  Possibly we'd want this USE flag to add a config
     fragment to the kernel (from the cros-kernel2.eclass) so we don't
     enable modules without having the actual firmware present.
- If we uprev a firmware that was used on old kernels, we should be
  able to find (fairly easily) the set of boards we need to test with
  because they'll have the USE flag set.  Then we can make sure we
  didn't break anything.
- We will likely not do a wholesale uprev of the linux-firmware.git.
  Right now we are only picking a very small number of firmware files
  to install and we'll manage those manually.  If this changes and we
  find some reason to install a whole pile of firmware files that are
  uprevving all the time, we can re-evaluate.

[1] Some firmware is actually built-in to the kernel.  See
FIRMWARE_BINARIES in the eclass.  We won't worry about these right now
as they're un-affected by this change.

BUG= chromium:770230 
TEST=trybots on lots of boards

Change-Id: Ia2b54e9fbf91e1fd39a3fc0d06349ad2ab816353
Reviewed-on: https://chromium-review.googlesource.com/729700
Commit-Ready: Douglas Anderson <dianders@chromium.org>
Tested-by: Douglas Anderson <dianders@chromium.org>
Reviewed-by: Chirantan Ekbote (OOO 10/23 - 11/03) <chirantan@chromium.org>

[modify] https://crrev.com/bdf884f930b616eef6bbf93b1a00ccb2ff28b805/sys-kernel/linux-firmware/linux-firmware-9999.ebuild
[add] https://crrev.com/bdf884f930b616eef6bbf93b1a00ccb2ff28b805/licenses/LICENSE.keyspan_usb
[modify] https://crrev.com/bdf884f930b616eef6bbf93b1a00ccb2ff28b805/profiles/targets/chromeos/package.use

All good folks, thanks!  Not suggesting a merge to M-63.  Although I'm pretty confident in the above analysis, there's still _some_ chance that there's some firmware somewhere that someone was relying on.  I'd rather give as much bake time as possible.

"We purposely tell the kernel to install firmware even if we have no driver to support that firmware.  Awesome."
" We don't install _all_ firmware here, we pick the ones to install using USE flags."
It seems like trimming the USE flags and the files which get installed with them is an easier solution to reclaim rootfs space than to muck with the linux-firmware directory itself? 

> "We purposely tell the kernel to install firmware even if we have no driver to support that firmware.  Awesome."

That is no longer true after the CL from comment #10.  Now we only install firmware that's requested.  Note that this is not an open bug, it's in "Fixed" state.


> " We don't install _all_ firmware here, we pick the ones to install using USE flags."
> It seems like trimming the USE flags and the files which get installed with them is an easier 
> solution to reclaim rootfs space than to muck with the linux-firmware directory itself? 

If you know of boards saying that they need firmware but they don't need it, then yes you'll save space by fixing that bug.

===

If you happen to just be trying to save a bit of rootfs space, you could volunteer to own bug #777204.