New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 765499 link

Starred by 5 users

vboot_reference: stop statically linking crossystem/futility

Project Member Reported by vapier@chromium.org, Sep 15 2017

Issue description

we're forcing crossystem to be statically linked in vboot because of AU.  the only place i can see crossystem being used is in `chromeos-install` ... it doesn't seem to be in postinst.  chromeos-install is a shell script and executes many many other programs that aren't statically linked.  so why do we still need this ?
 

Comment 1 Deleted

Comment 2 by vapier@chromium.org, Sep 15 2017

Hung-Te mentioned firmware updater.  looking at random boards:
  bob         static
  butterfly   static
  caroline    dynamic xxd
  cave        dynamic xxd
  chell       dynamic ectool xxd
  coral       static
  cyan        static
  daisy       static
  edgar       static
  elm         static
  eve         dynamic dbus-send iotools xxd
  falco       static
  fizz        static
  gale        static
  glados      static
  hana        static
  kevin       static
  kip         static
  leon        static
  link        static
  lumpy       static
  oak         static
  parrot      static
  peppy       static
  poppy       static
  quawks      static
  samus       dynamic ectool
  scarlet     static
  sentry      dynamic ectool xxd
  winky       static
  wizpig      static
  wolf        static

so either dynamic isn't a big deal, or we're already failing at it ;).  where is the code that creates the firmware updater ?  seems like it's time to convert them to dynamic/lddtree too.

keep in mind we're talking about ToT here which isn't where (afaik) firmware-updaters come from.  we can always land a patch in the firmware branch to force static builds of userland tools.

Comment 3 by hungte@chromium.org, Sep 15 2017

src/platform/firmware/pack_firmware.py:

   parser.add_argument(
        '--tools', type=str,
        default='flashrom mosys crossystem gbb_utility vpd dump_fmap',
        help='List of tool programs to be bundled into updater')

This tracks programs we found that usually cause compatibility problems in history.

> ToT here which isn't where (afaik) firmware-updaters come from.  we can always land a patch in the firmware branch to force static builds of userland tools.

No. Firmware branch maintains only the firmware "image binary blob". The updater (code) comes from where the OS image is built, i.e., release branch or ToT.

Comment 4 by hungte@chromium.org, Sep 15 2017

We had futility built in a special way: both dynamic (futility) and static (futility_s).

Firmware updater will select the _s automatically if that is available.

So maybe the first step can be changing crossystem to same way that futility was built?

Comment 5 by hungte@chromium.org, Sep 15 2017

Change to crossystsem:
 https://chromium-review.googlesource.com/#/c/chromiumos/platform/vboot_reference/+/668274/

crossytem size changes:

16K  dyn.tbz2
336K static.tbz2
Project Member

Comment 6 by bugdroid1@chromium.org, Sep 15 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/vboot_reference/+/c8e3d27c814634c629b46dab0b708d3cf375c641

commit c8e3d27c814634c629b46dab0b708d3cf375c641
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri Sep 15 17:50:29 2017

Makefile: Build utils for both dynamic and static version.

The auto update process (especially firmware updater) needs static vboot
utilitys but normal OS images, including recovery images, don't need
that. We should build both dynamic and static binaries at the same time
for images to choose what they need.

Currently only `crossystem` will build static version. And after this
change is merged:

(cd /build/reef/usr/bin; file crossystem*)
crossystem:   ELF 64-bit LSB shared object
crossystem_s: ELF 64-bit LSB executable

(cd /build/reef/usr/bin; du -sh crossystem*)
40K  crossystem
808K crossystem_s

BUG= chromium:764753 , chromium:765499 
TEST=emerge-reef vboot_reference
BRANCH=None

Change-Id: Ibd66c87bb44c5593767aeb710240e0165103f016
Reviewed-on: https://chromium-review.googlesource.com/668274
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/c8e3d27c814634c629b46dab0b708d3cf375c641/Makefile

Project Member

Comment 7 by bugdroid1@chromium.org, Sep 15 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/d6617d2297afb0a8577c1715a7d0731705d948c9

commit d6617d2297afb0a8577c1715a7d0731705d948c9
Author: Mike Frysinger <vapier@chromium.org>
Date: Fri Sep 15 17:50:31 2017

crosutil_unittest: fix tests to run from any cwd

Fix running the tests from any working directory.  The current code
assumes it's run from the unittests/ directory.

BUG= chromium:765499 
TEST=unittests pass

Change-Id: I95cdb109f1429d2f25c49a8ca5621267471010ca
Reviewed-on: https://chromium-review.googlesource.com/667734
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/d6617d2297afb0a8577c1715a7d0731705d948c9/unittests/crosutil_unittest

Project Member

Comment 8 by bugdroid1@chromium.org, Sep 16 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/98e9abb1faa1810fb10338dc693e7c7a0e95f132

commit 98e9abb1faa1810fb10338dc693e7c7a0e95f132
Author: Mike Frysinger <vapier@chromium.org>
Date: Fri Sep 15 23:59:24 2017

osutils.Which: add support for diff root path

BUG= chromium:765499 
TEST=unittests pass

Change-Id: I66cf5cbaac8bb315dad62212656e214fe5d26797
Reviewed-on: https://chromium-review.googlesource.com/667012
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/98e9abb1faa1810fb10338dc693e7c7a0e95f132/lib/osutils_unittest.py
[modify] https://crrev.com/98e9abb1faa1810fb10338dc693e7c7a0e95f132/lib/osutils.py

Project Member

Comment 9 by bugdroid1@chromium.org, Sep 16 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/80d2f19368813a9e560e61af4f36e74085365984

commit 80d2f19368813a9e560e61af4f36e74085365984
Author: Mike Frysinger <vapier@chromium.org>
Date: Fri Sep 15 23:59:22 2017

pack_firmware_unittest: use cros_test_lib

BUG= chromium:765499 
TEST=unittests pass

Change-Id: I511381ad87725ebbfd33a09a3617f335dc22b7d7
Reviewed-on: https://chromium-review.googlesource.com/667735
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/80d2f19368813a9e560e61af4f36e74085365984/pack_firmware_unittest.py

Project Member

Comment 10 by bugdroid1@chromium.org, Sep 16 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/30bc745ffd49bb6ccbc4856e9e1a6d04d06de99f

commit 30bc745ffd49bb6ccbc4856e9e1a6d04d06de99f
Author: Mike Frysinger <vapier@chromium.org>
Date: Fri Sep 15 23:59:22 2017

pack_firmware_unittest: fix temporary output handling

The unittest has been writing to an "out" file in the cwd instead of
using a tempdir.  Clean that up.

BUG= chromium:765499 
TEST=unittests pass

Change-Id: I8a07feb5864c11077d259f684a0ee5e4344a86b9
Reviewed-on: https://chromium-review.googlesource.com/668316
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/30bc745ffd49bb6ccbc4856e9e1a6d04d06de99f/pack_firmware_unittest.py

Comment 11 Deleted

Comment 12 Deleted

Project Member

Comment 13 by bugdroid1@chromium.org, Sep 16 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/61647a5f0045eae1dd3c371b2b4f9da7b9713fac

commit 61647a5f0045eae1dd3c371b2b4f9da7b9713fac
Author: Mike Frysinger <vapier@chromium.org>
Date: Sat Sep 16 06:39:25 2017

pack_firmware: use osutils.Which

Rather than reimplement this func in two different ways. use
the existing utility function that does the same thing.

BUG= chromium:765499 
TEST=unittests pass

Change-Id: If199c188516d7502b06df8341f6669355d55de00
Reviewed-on: https://chromium-review.googlesource.com/668317
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/61647a5f0045eae1dd3c371b2b4f9da7b9713fac/pack_firmware.py
[modify] https://crrev.com/61647a5f0045eae1dd3c371b2b4f9da7b9713fac/local_flashrom/flashrom
[modify] https://crrev.com/61647a5f0045eae1dd3c371b2b4f9da7b9713fac/pack_firmware_unittest.py
[modify] https://crrev.com/61647a5f0045eae1dd3c371b2b4f9da7b9713fac/test/dump_fmap
[modify] https://crrev.com/61647a5f0045eae1dd3c371b2b4f9da7b9713fac/test/flashrom

Project Member

Comment 14 by bugdroid1@chromium.org, Sep 23 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/vboot_reference/+/5337484629c87d0fa6a1fa0aa598f3cad5a19f37

commit 5337484629c87d0fa6a1fa0aa598f3cad5a19f37
Author: Mike Frysinger <vapier@chromium.org>
Date: Sat Sep 23 05:25:40 2017

crossystem: support standard --help flag

BUG= chromium:765499 
TEST=unittests pass
BRANCH=None

Change-Id: I5c5118c44897d89e5116a9fce49bacbf16704dd8
Reviewed-on: https://chromium-review.googlesource.com/668658
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Aaron Durbin <adurbin@chromium.org>

[modify] https://crrev.com/5337484629c87d0fa6a1fa0aa598f3cad5a19f37/utility/crossystem.c

Project Member

Comment 15 by bugdroid1@chromium.org, Sep 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/180416528bd137d9556b0b0da1f9a8a712b3236f

commit 180416528bd137d9556b0b0da1f9a8a712b3236f
Author: Mike Frysinger <vapier@chromium.org>
Date: Wed Sep 27 05:35:20 2017

convert to futility

We have a single futility binary now and compat symlinks for the old
program names.  pack_firmware though has been copying the old names
still though which means we ended up with duplicates.  Switch all the
code over to futility to shrink things down.

BUG= chromium:765499 
TEST=unittests pass

Change-Id: I5994bdd3e2be779ee02637d7c1e643cee82622b2
Reviewed-on: https://chromium-review.googlesource.com/668318
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/180416528bd137d9556b0b0da1f9a8a712b3236f/pack_dist/updater4.sh
[add] https://crrev.com/180416528bd137d9556b0b0da1f9a8a712b3236f/test/futility
[modify] https://crrev.com/180416528bd137d9556b0b0da1f9a8a712b3236f/pack_firmware_unittest.py
[modify] https://crrev.com/180416528bd137d9556b0b0da1f9a8a712b3236f/pack_dist/updater3.sh
[modify] https://crrev.com/180416528bd137d9556b0b0da1f9a8a712b3236f/pack_stub
[modify] https://crrev.com/180416528bd137d9556b0b0da1f9a8a712b3236f/pack_dist/crosfw.sh
[modify] https://crrev.com/180416528bd137d9556b0b0da1f9a8a712b3236f/pack_firmware_functest.py
[delete] https://crrev.com/3330e07b88f36c3e678cdddc1edd9ed0fb227fff/test/dump_fmap
[modify] https://crrev.com/180416528bd137d9556b0b0da1f9a8a712b3236f/pack_firmware.py
[modify] https://crrev.com/180416528bd137d9556b0b0da1f9a8a712b3236f/pack_dist/crosutil.sh

Project Member

Comment 16 by bugdroid1@chromium.org, Sep 27 2017

Project Member

Comment 17 by bugdroid1@chromium.org, Sep 28 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/aa0a5034bd31b43c0e4081c42da5409a53ed1292

commit aa0a5034bd31b43c0e4081c42da5409a53ed1292
Author: Mike Frysinger <vapier@chromium.org>
Date: Thu Sep 28 02:03:02 2017

pack_firmware: switch to chromite.lib.commandline

This provides a bit more of a standard chromite CLI, includes more
helper argparse types (notably "path"), takes care of setting up the
logging system for us, and allows us to freeze the parsed object.

BUG= chromium:765499 
TEST=unittests pass

Change-Id: Id364d303fafe67da9c39a9e2cd32cbc59cfb1913
Reviewed-on: https://chromium-review.googlesource.com/668320
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/aa0a5034bd31b43c0e4081c42da5409a53ed1292/pack_firmware_unittest.py
[modify] https://crrev.com/aa0a5034bd31b43c0e4081c42da5409a53ed1292/pack_firmware.py

Project Member

Comment 18 by bugdroid1@chromium.org, Sep 28 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/2c5835c6dceb7abf6f47e75395ead3b629959e14

commit 2c5835c6dceb7abf6f47e75395ead3b629959e14
Author: Mike Frysinger <vapier@chromium.org>
Date: Thu Sep 28 04:12:55 2017

pack_firmware_functest: switch to cros_test_lib

BUG= chromium:765499 
TEST=unittests pass

Change-Id: Id1f940c16e0abcdddc234c32c3c77ece7eb0c143
Reviewed-on: https://chromium-review.googlesource.com/688534
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>

[modify] https://crrev.com/2c5835c6dceb7abf6f47e75395ead3b629959e14/pack_firmware_functest.py

Project Member

Comment 19 by bugdroid1@chromium.org, Sep 28 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/fa0215c3dd3637b3a2d8e38482df46f77a59b34a

commit fa0215c3dd3637b3a2d8e38482df46f77a59b34a
Author: Mike Frysinger <vapier@chromium.org>
Date: Thu Sep 28 17:14:35 2017

invoke: drop need for eval

BUG= chromium:765499 
TEST=unittests pass

Change-Id: I1f2c2b04421a5388b0a82ce7bece16def512a789
Reviewed-on: https://chromium-review.googlesource.com/688554
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/fa0215c3dd3637b3a2d8e38482df46f77a59b34a/pack_dist/crosfw.sh
[modify] https://crrev.com/fa0215c3dd3637b3a2d8e38482df46f77a59b34a/pack_dist/common.sh

Cc: sdantul...@chromium.org gkihumba@chromium.org josa...@chromium.org mkarkada@chromium.org abod...@chromium.org mxt@chromium.org ka...@chromium.org matthewjoseph@chromium.org dchan@chromium.org sontis@chromium.org waihong@chromium.org dhadd...@chromium.org dchan@google.com
 Issue 769536  has been merged into this issue.
Issue(Recovery is failing with an Unexpected error message while installing Via USB) is seen on latest M-63 Build i.e. on 63.0.3230.0/1003.0.0 dev-channel Reks and Candy.
please file a new bug with full logs so it can be properly triaged

Comment 23 by ajha@chromium.org, Oct 5 2017

 Issue 769602  is already filed for this, which was duped into  Issue 769536 (duped here in C#20) 
that bug was filed for an older version that is already fixed (that's what comment #19 here is about).  you're talking about something newer.
Project Member

Comment 25 by bugdroid1@chromium.org, Oct 27 2017

Project Member

Comment 26 by bugdroid1@chromium.org, Oct 27 2017

Project Member

Comment 27 by bugdroid1@chromium.org, Oct 28 2017

Project Member

Comment 28 by bugdroid1@chromium.org, Oct 30 2017

Labels: merge-merged-release-R63-10032.B
The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/overlay-samus-private/+/4b074bcd85dd22a8cbdbd8ae75f8a0b784acec39

commit 4b074bcd85dd22a8cbdbd8ae75f8a0b784acec39
Author: Mike Frysinger <vapier@chromium.org>
Date: Mon Oct 30 18:06:20 2017

Project Member

Comment 29 by bugdroid1@chromium.org, Oct 30 2017

Project Member

Comment 30 by bugdroid1@chromium.org, Nov 15 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/5d69102b5506fed2d6b1e11e6c0c37dcf4af2ce9

commit 5d69102b5506fed2d6b1e11e6c0c37dcf4af2ce9
Author: Mike Frysinger <vapier@chromium.org>
Date: Wed Nov 15 05:52:42 2017

convert to more chromite helpers

BUG= chromium:765499 
TEST=unittests pass

Change-Id: I9bf0719bcae16181a6cc11070b7f07af5856e177
Reviewed-on: https://chromium-review.googlesource.com/731029
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>

[modify] https://crrev.com/5d69102b5506fed2d6b1e11e6c0c37dcf4af2ce9/pack_firmware_functest.py
[modify] https://crrev.com/5d69102b5506fed2d6b1e11e6c0c37dcf4af2ce9/pack_firmware.py
[modify] https://crrev.com/5d69102b5506fed2d6b1e11e6c0c37dcf4af2ce9/pack_firmware_unittest.py

Project Member

Comment 31 by bugdroid1@chromium.org, Dec 1 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/08014fe0379a78680dac74bbf593d9fb9de5732f

commit 08014fe0379a78680dac74bbf593d9fb9de5732f
Author: Mike Frysinger <vapier@chromium.org>
Date: Fri Dec 01 04:30:13 2017

functest: be less clever about test inputs

There's no need to declare a string and split it into an array when we
can just declare the array directly.  It's arguably easier to update.

BUG= chromium:765499 
TEST=unittests pass

Change-Id: I3b85b47208f7536ebff8c53165055afc06df5e67
Reviewed-on: https://chromium-review.googlesource.com/798771
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>

[modify] https://crrev.com/08014fe0379a78680dac74bbf593d9fb9de5732f/pack_firmware_functest.py

Project Member

Comment 32 by bugdroid1@chromium.org, Dec 1 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/aec9923163ad5ce80d35e45f5e3900e1adb29472

commit aec9923163ad5ce80d35e45f5e3900e1adb29472
Author: Mike Frysinger <vapier@chromium.org>
Date: Fri Dec 01 04:30:24 2017

functest: fix unbalanced-tuple-unpacking lint warning

Simplify this function with a list comprehension to fix a lint warning:
W:700, 4: Possible unbalanced tuple unpacking with sequence defined at line 686:
          left side has 2 label(s), right side has 0 value(s) (unbalanced-tuple-unpacking)

BUG= chromium:765499 
TEST=unittests pass

Change-Id: I7f1a9a8e7ceecd2aa4259b318ec771801d32bafd
Reviewed-on: https://chromium-review.googlesource.com/798970
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>

[modify] https://crrev.com/aec9923163ad5ce80d35e45f5e3900e1adb29472/pack_firmware_functest.py

Project Member

Comment 33 by bugdroid1@chromium.org, Dec 8 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/4462382d82815024b0674846ab1288fcefd26d30

commit 4462382d82815024b0674846ab1288fcefd26d30
Author: Mike Frysinger <vapier@chromium.org>
Date: Fri Dec 08 11:59:20 2017

pack_firmware: switch to osutils.Which

Replace our custom for loop with chromite's Which helper.  This isn't
strictly necessary now, but will help when we add $ROOT support.

We have to +x the helper tools because Which enforces that in results.

BUG= chromium:765499 
TEST=unittests pass

Change-Id: I4c7c482e672c4a65b0f1197d6232d77edb30c118
Reviewed-on: https://chromium-review.googlesource.com/798770
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/4462382d82815024b0674846ab1288fcefd26d30/functest/crossystem
[modify] https://crrev.com/4462382d82815024b0674846ab1288fcefd26d30/functest/flashrom
[modify] https://crrev.com/4462382d82815024b0674846ab1288fcefd26d30/functest/mosys
[modify] https://crrev.com/4462382d82815024b0674846ab1288fcefd26d30/pack_firmware.py
[modify] https://crrev.com/4462382d82815024b0674846ab1288fcefd26d30/functest/vpd

Project Member

Comment 34 by bugdroid1@chromium.org, Dec 9 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/51afc0c34508c4dcc43e7ad73ccd991da854cbd1

commit 51afc0c34508c4dcc43e7ad73ccd991da854cbd1
Author: Mike Frysinger <vapier@chromium.org>
Date: Sat Dec 09 16:50:03 2017

pack_firmware: move helper programs to a bin/ subdir

All bundled files get dumped into the same flat directory currently.
This includes data files and executables and helper scripts.  If we
want to bundle libraries, it's going to get even messier in here.
Lets relocate all the programs to a bin/ subdir to keep things a bit
neater in here.

A packed updater currently looks something like:
    updater/
    |-- bios.bin
    |-- common.sh
    |-- crosfw.sh
    |-- crossystem
    |-- crosutil.sh
    |-- ec.bin
    |-- ectool
    |-- flashrom
    |-- futility
    |-- mosys
    |-- pd.bin
    |-- shflags
    |-- updater4.sh
    |-- updater_custom.sh
    |-- VERSION
    |-- VERSION.md5
    `-- vpd

After this change it looks like:
    updater/
    |-- bin
    |   |-- crossystem
    |   |-- ectool
    |   |-- flashrom
    |   |-- futility
    |   |-- mosys
    |   `-- vpd
    |-- bios.bin
    |-- common.sh
    |-- crosfw.sh
    |-- crosutil.sh
    |-- ec.bin
    |-- pd.bin
    |-- shflags
    |-- updater4.sh
    |-- updater_custom.sh
    |-- VERSION
    `-- VERSION.md5

BUG= chromium:765499 
TEST=unittests pass

Change-Id: I26ddffa070ddcbac5af8804f91f09ef9f3028591
Reviewed-on: https://chromium-review.googlesource.com/798773
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/51afc0c34508c4dcc43e7ad73ccd991da854cbd1/functest/fmap.bin
[rename] https://crrev.com/51afc0c34508c4dcc43e7ad73ccd991da854cbd1/functest/bin/vpd
[rename] https://crrev.com/51afc0c34508c4dcc43e7ad73ccd991da854cbd1/functest/bin/flashrom
[rename] https://crrev.com/51afc0c34508c4dcc43e7ad73ccd991da854cbd1/functest/bin/crossystem
[modify] https://crrev.com/51afc0c34508c4dcc43e7ad73ccd991da854cbd1/pack_firmware.py
[rename] https://crrev.com/51afc0c34508c4dcc43e7ad73ccd991da854cbd1/functest/bin/mosys
[modify] https://crrev.com/51afc0c34508c4dcc43e7ad73ccd991da854cbd1/pack_firmware_functest.py
[modify] https://crrev.com/51afc0c34508c4dcc43e7ad73ccd991da854cbd1/pack_dist/crosutil.sh

Project Member

Comment 35 by bugdroid1@chromium.org, Dec 11 2017

Labels: merge-merged-factory-coral-10122.B
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/ad09fdbe22bc24bd87068523c57976a39745412d

commit ad09fdbe22bc24bd87068523c57976a39745412d
Author: Mike Frysinger <vapier@chromium.org>
Date: Mon Dec 11 09:06:34 2017

convert to more chromite helpers

BUG= chromium:765499 
TEST=unittests pass

Change-Id: I9bf0719bcae16181a6cc11070b7f07af5856e177
Reviewed-on: https://chromium-review.googlesource.com/731029
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/794609
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Jason Clinton <jclinton@chromium.org>
Commit-Queue: Marco Chen <marcochen@chromium.org>
Tested-by: Marco Chen <marcochen@chromium.org>

[modify] https://crrev.com/ad09fdbe22bc24bd87068523c57976a39745412d/pack_firmware_functest.py
[modify] https://crrev.com/ad09fdbe22bc24bd87068523c57976a39745412d/pack_firmware.py
[modify] https://crrev.com/ad09fdbe22bc24bd87068523c57976a39745412d/pack_firmware_unittest.py

Cc: jclinton@chromium.org
Are we very close to having lddtree being used by Pack Firmware? Working on mosys and trying to determine whether we want to continue to generate static binaries. If lddtree is close, we can consider dropping in the medium term.

Blocking: 679858
Project Member

Comment 38 by bugdroid1@chromium.org, May 16 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/f43fac8082946a4c4349a74628690cf21901230f

commit f43fac8082946a4c4349a74628690cf21901230f
Author: Mike Frysinger <vapier@chromium.org>
Date: Wed May 16 12:08:31 2018

cros-firmware: update PATH for access to lddtree

BUG= chromium:765499 
TEST=unittests pass

Change-Id: Ib03552447b710dd19bccaa50218848c3f010b96d
Reviewed-on: https://chromium-review.googlesource.com/1059308
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/f43fac8082946a4c4349a74628690cf21901230f/eclass/cros-firmware.eclass

Project Member

Comment 39 by bugdroid1@chromium.org, May 16 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/b168a20180bfbfe98e580aee51b9fc28a9c09276

commit b168a20180bfbfe98e580aee51b9fc28a9c09276
Author: Mike Frysinger <vapier@chromium.org>
Date: Wed May 16 23:43:03 2018

pack_firmware: support lddtree & dynamic ELFs

Trying to produce static binaries for the firmware updater is often at
odds with other system desires: keeping dynamic ELFs around for size in
the rootfs and recovery image.  It is also often leaky: we can pull in
ELFs that are dynamically linked which means the updater isn't actually
standalone like we'd want.

Lets convert the firmware updater over to using dynamic ELFs normally
via lddtree.  This way we use the same package everywhere and we no
longer have to manually track/deal with static binaries.

A packed updater currently looks something like:
    updater/
    |-- bin
    |   |-- crossystem
    |   |-- ectool
    |   |-- flashrom
    |   |-- futility
    |   |-- mosys
    |   `-- vpd
    |-- bios.bin
    |-- common.sh
    |-- crosfw.sh
    |-- crosutil.sh
    |-- ec.bin
    |-- pd.bin
    |-- shflags
    |-- updater4.sh
    |-- updater_custom.sh
    |-- VERSION
    `-- VERSION.md5

After this change it looks like:
    updater/
    |-- bin
    |   |-- crossystem
    |   |-- crossystem.elf
    |   |-- ectool
    |   |-- ectool.elf
    |   |-- flashrom
    |   |-- flashrom.elf
    |   |-- futility
    |   |-- futility.elf
    |   |-- mosys
    |   |-- mosys.elf
    |   |-- vpd
    |   `-- vpd.elf
    |-- bios.bin
    |-- common.sh
    |-- crosfw.sh
    |-- crosutil.sh
    |-- ec.bin
    |-- lib
    |   |-- ld-linux-x86-64.so.2
    |   |-- libcrypto.so.1.0.0
    |   |-- libc.so.6
    |   |-- libdl.so.2
    |   |-- libfdt.so.1
    |   |-- libfmap.so.0
    |   |-- libpci.so.3
    |   |-- libuuid.so.1
    |   `-- libz.so.1
    |-- pd.bin
    |-- shflags
    |-- updater4.sh
    |-- updater_custom.sh
    |-- VERSION
    `-- VERSION.md5

The reason for all the ".elf" files is that those are the actual ELFs
while the other one is a wrapper script.  e.g. "vpd" is a small shell
script that sets up the execution environment so that it uses the libs
under the local ./lib/ directory before running the real "vpd.elf".
If lddtree does find a static ELF, it will copy it over directly too.

File size wise, they're about equal, although the latter is now more
reliable in terms of completeness (always dynamic), and scales better
if we want to add more programs.

This CL alone doesn't fully convert things.  If the --root option isn't
used, we copy files over like we always have.  But if we specify the
--root option, we use the new lddtree code paths.  Once the eclass and
any other users update, we can delete old support.

BUG= chromium:765499 
TEST=unittests pass
CQ-DEPEND=CL:1059308

Change-Id: If33473eff5301d3134aed2182321f3e75ce6909f
Reviewed-on: https://chromium-review.googlesource.com/668321
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/b168a20180bfbfe98e580aee51b9fc28a9c09276/pack_firmware_functest.py
[modify] https://crrev.com/b168a20180bfbfe98e580aee51b9fc28a9c09276/pack_firmware.py
[modify] https://crrev.com/b168a20180bfbfe98e580aee51b9fc28a9c09276/pack_firmware_unittest.py

Project Member

Comment 40 by bugdroid1@chromium.org, May 23 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/618ee0d58d424d62a7979014f3e90dba0bafad32

commit 618ee0d58d424d62a7979014f3e90dba0bafad32
Author: Mike Frysinger <vapier@chromium.org>
Date: Wed May 23 13:47:27 2018

cros-firmware: switch to --root when packing the firmware

This allows us to bundle programs when built dynamically w/out having
to worry about all their library deps.  Otherwise we need to manually
build programs statically, and hope that we remember to mark all such
programs as static in the build (we often forget to).

BUG= chromium:765499 
TEST=precq passes
CQ-DEPEND=CL:668321

Change-Id: I6734521d4aaf81179e2ee8a304c910bb8ed6ba12
Reviewed-on: https://chromium-review.googlesource.com/818706
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Jason Clinton <jclinton@chromium.org>

[modify] https://crrev.com/618ee0d58d424d62a7979014f3e90dba0bafad32/eclass/cros-firmware.eclass

Project Member

Comment 41 by bugdroid1@chromium.org, May 31 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/vboot_reference/+/04e3f3125a7ab6c1d104515c6ab788da4bb3f657

commit 04e3f3125a7ab6c1d104515c6ab788da4bb3f657
Author: Mike Frysinger <vapier@chromium.org>
Date: Thu May 31 19:26:21 2018

stop statically linking crossystem

Now that initramfs, firmware updater, and auto-updater are all using
dynamically linked programs, there's no need to produce a static
build of crossystem anymore.

BUG= chromium:765499 
TEST=precq passes (includes vmtests w/AU)
BRANCH=None

Change-Id: I5aa123e662040ff5d9f2328c0f036b648fc629fb
Reviewed-on: https://chromium-review.googlesource.com/667881
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/04e3f3125a7ab6c1d104515c6ab788da4bb3f657/Makefile

Owner: vapier@chromium.org
Summary: vboot_reference: stop statically linking crossystem/futility (was: vboot_reference: stop statically linking crossystem)
still have to clean up futility_s building
Status: Assigned (was: Available)
Project Member

Comment 44 by bugdroid1@chromium.org, Sep 7

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/vboot_reference/+/9fa18683c9b55d47d0ef530b5bd5a057d04b782a

commit 9fa18683c9b55d47d0ef530b5bd5a057d04b782a
Author: Mike Frysinger <vapier@chromium.org>
Date: Fri Sep 07 21:29:03 2018

futility: drop static build

We shouldn't need this anywhere anymore, so drop it.

BUG= chromium:765499 
TEST=precq passes
BRANCH=none

Change-Id: I0f1adb2bf120e1a20c79d2641a0d2fe96d8e6908
Reviewed-on: https://chromium-review.googlesource.com/1210342
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/9fa18683c9b55d47d0ef530b5bd5a057d04b782a/Makefile

Status: Fixed (was: Assigned)
Project Member

Comment 46 by bugdroid1@chromium.org, Sep 9

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/0e3c59515fcce2931d5503bf6d89237b10f2ca52

commit 0e3c59515fcce2931d5503bf6d89237b10f2ca52
Author: Hung-Te Lin <hungte@chromium.org>
Date: Sun Sep 09 17:33:52 2018

pack_firmware: Change default tools to empty for updater5

Since chromium:765753, the postinst now always runs updater inside
the new rootfs (using chroot) so bundling tool programs (flashrom,
crossystem, mosys, ...) is no longer needed for AU and recovery.

There may be still developers grabbing an older/newer version
of chromeos-firmwareupdate and run it directly on different rootfs,
where the related programs may be not compatible and cause updating to
fail -- especially in early proto stages. However, this may be not super
critical to support, especially since, we have zero testing/validation
making sure we actually bundle all the binaries we care about (for
example, mosys calls other programs like vpd_get_value). Plus, currently
we are moving updating logic into futility itself (chromium:875551).

As a result, in this patch we are going to experiment "no tools bundled"
on new devices using updater5 (which will run the futility based
updater) and then decide if we should remove all tools for all devices.

BUG= chromium:765499 , chromium:765753 , chromium:875551 
TEST=./pack_firmware_unittest.py
     ./pack_firmware_functest.py
     Set CROS_FIRMWARE_SCRIPT=updater5.sh in ebuild and then
     emerge-$BOARD chromeos-firmware-$BOARD;
     /build/$BOARD/usr/sbin/chromeos-firmwareupdate -V # no tools

Change-Id: I5b388129ac3961ef5a52767b861cb70bbee596c9
Reviewed-on: https://chromium-review.googlesource.com/1215102
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/0e3c59515fcce2931d5503bf6d89237b10f2ca52/pack_firmware_unittest.py
[modify] https://crrev.com/0e3c59515fcce2931d5503bf6d89237b10f2ca52/pack_firmware.py

Project Member

Comment 47 by bugdroid1@chromium.org, Sep 21

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/vboot_reference/+/cc77bb21ebf1e131a6a8333e89709a1b8f195817

commit cc77bb21ebf1e131a6a8333e89709a1b8f195817
Author: Hung-Te Lin <hungte@chromium.org>
Date: Fri Sep 21 04:30:02 2018

futility: Remove FUTIL_STATIC build rules

After CL:1210342, the 'futility_s' is no longer needed so we may drop
the related build rules. People who wants to build static version of
futility can do:
  make STATIC=true futil

BUG= chromium:765499 
TEST=precq passes; make futil; make clean; make STATIC=1 futil
BRANCH=none

Change-Id: I80e83a80eaa273f09288f850c59a52494dc5bec9
Signed-off-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1235795
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/cc77bb21ebf1e131a6a8333e89709a1b8f195817/Makefile

Project Member

Comment 48 by bugdroid1@chromium.org, Nov 1

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/firmware/+/2cd6f5a4299d77480847f5f27676e20aabd719d7

commit 2cd6f5a4299d77480847f5f27676e20aabd719d7
Author: Hung-Te Lin <hungte@chromium.org>
Date: Thu Nov 01 17:02:15 2018

pack_firmware: Remove --tool and deprecate --tool_base

The design of bundling programs (especially those exist inside rootfs,
i.e., flashrom, crossystem, mosys, ...) in firmware updater was
introduced long ago when the underlying programs were still changing
rapidly, and updater was full of shell scripts.

Recently we are seeing challenges that
 - The duplicated programs have increased updater size rapidly
 - There is no easy way to ensure all needed programs (and all
   dependencies, including library, data and other executable files)
   are included.

Since then, there were few issues created to solve the problem,
including:
 - chromium:765753 changed the postinst to always run inside chroot
 - chromium:765499 tries to copy dynamically linked programs with lddtree

This change has removed --tool and deprecated --tool_base (which is
currently still used by cros-firmware.eclass, and can be removed after
this is merged), so the updater script will always find programs from
rootfs. If really needed, a device may still include additional programs
using --extra.

- Auto update and recovery will run the binaries from new rootfs.
- Factory installer may need to upgrade related tools (flashrom, mosys,
  etc) but that's probably not a big problem since there are other
  programs using mosys, so the factory branch needs to cherry-pick
  related (especailly unibuild) changes as well.
- Developers working on early builds may suffer if there are known
  issues in flashrom or crossystem, but they can always flash using
  servo, or boot a fresh new disk image and run new tools from there.

BUG= chromium:765499 , chromium:765753 
TEST=./pack_firmware_unittest.py
     ./pack_firmware_functest.py

Change-Id: Icbc98953f6a13d3f1a4f248f3643b6c77e8e4956
Reviewed-on: https://chromium-review.googlesource.com/1212684
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[delete] https://crrev.com/85a7b3eb388b1fa826179617eaae6593dd290cf6/test/flashrom
[delete] https://crrev.com/85a7b3eb388b1fa826179617eaae6593dd290cf6/test/futility
[modify] https://crrev.com/2cd6f5a4299d77480847f5f27676e20aabd719d7/functest/bin/flashrom
[modify] https://crrev.com/2cd6f5a4299d77480847f5f27676e20aabd719d7/pack_firmware_unittest.py
[modify] https://crrev.com/2cd6f5a4299d77480847f5f27676e20aabd719d7/pack_firmware.py
[modify] https://crrev.com/2cd6f5a4299d77480847f5f27676e20aabd719d7/pack_firmware_functest.py
[delete] https://crrev.com/85a7b3eb388b1fa826179617eaae6593dd290cf6/local_flashrom/flashrom

Sign in to add a comment