It is also failing the appid part:
...
13:07:52: INFO: Waiting for recovery image...
13:21:28: INFO: RunCommand: /b/cbuild/internal_master/chromite/bin/cros_sdk 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmpfdDasR' -- ./security_test_image '--board=amd64-generic-goofy' in /b/cbuild/internal_master
INFO    security_test_image: Loading baselines from /mnt/host/source/cros-signing/security_test_baselines
INFO    security_test_image: Using /mnt/host/source/src/build/images/amd64-generic-goofy/R52-8224.0.0/recovery_image.bin
INFO    security_test_image: Using vboot_reference.git rev 65f61f90ccae5f0b5f37ea3bdf86735bc58709a0
INFO    security_test_image: Running ensure_no_nonrelease_files.sh
INFO    security_test_image: Running ensure_sane_lsb-release.sh
Loading config from /mnt/host/source/cros-signing/security_test_baselines/ensure_sane_lsb-release.config... Done.
CHROMEOS_RELEASE_APPID mismatch. Expected '', image contains '{166986C6-8CB1-D180-6E5F-D62EA9730C83}'
ERROR   security_test_image: sane_lsb-release: test failed
INFO    security_test_image: Running ensure_secure_kernelparams.sh
Kernel dm= parameter does not match any expected values!
...

The image reports the correct appid, but the test is somehow looking for a blank appid.

This implies there might be something wrong with https://chrome-internal-review.googlesource.com/#/c/247092/4 but I am really not sure what, the script is not complaining it cannot find the board, it is just not finding the appid or expected kernel configuration, as if the board is not recognized somehow.

Any ideas? I guess this needs to be reproed locally with some added instrumentation.

security_test_image is in src/scripts/ so you should be able to do a local build for this board and then run/debug that script

i don't see a problem at a glance in that CL

Hung-te is anyone on your team leveraging this image?

Last time when I was looking at that, I thought it may take some time for the CL to take effect (either wait some cache to expire or some bot to be updated), then the target build starts and we stopped fixing the build. Instead we simply take the artifacts.

For a long term goal I think the factory team still want to get the build fixed. I'll take a look when available.

 Issue 610470  has been merged into this issue.

Ok, I think I've tracked down and found what happened.

 ./build_image --board amd64-generic-goofy base
 cd ../build/images/amd64-generic-goofy/latest
 mv chromiumos_base_image.bin chromiumos_image.bin
 cd ~/trunk/src/scripts
 ./mod_image_for_recovery.sh --board amd64-generic-goofy
 ./security_test_image --board amd64-generic-goofy

The board name in image is
 CHROMEOS_RELEASE_BOARD=amd64-generic-goofy

However, in ensure_sane_lsb-release.sh , the rule to derive board is:
 
 # Pick the right set of test-expectation data to use. The cuts
 # turn e.g. x86-foo-pvtkeys into x86-foo. 
  local board=$(lsbval $lsb CHROMEOS_RELEASE_BOARD | cut -d = -f 2 | cut -d - -f 1,2)

which converts the board name to amd64-generic .

I'm not sure which is the best solution here, since...
(1) We can change the name of appid var to amd64_generic, but that may also add appid for real amd64_generic.
(2) We can change ensure_sane_lsb-release, but that may create additional complicated rules and need a signer update
(3) We can change the board name of amd64-generic-goofy to amd64-generic_goofy, but I'm not sure if the name itself is more confusing

Any suggestions?

If we don't intend on pushing AUs to these, then I am partial to 1, if not then 2 is the technically correct fix, as it seems this logic makes assumptions which are not always true. 

If we did 3 that might break other assumptions (_ implies an old style variant), but 3.5 might be make it amd64generic-goofy as a bit of a hack, but that means we have to change it all over the place. 

For these goofy projects we won't push AU, so I think I'll do a CL as 1, with a comment that it's actually for amd64-generic-goofy.

Fixing 2 may be a good idea but I don't know what should be the new rule for deriving board names :)

I'm interested in 3. When you say _ is old style variant, does that mean we're not going to do _ anymore? What's the - and _ mean today? I mean, we have a lot of inconsistent board names like

 x86-zgb-he
 veyron_minnie
 samus-cheets

i'm not sure the splitting on dashes is still needed.  the signer runs ensure_sane_lsb-release *before* signing the image, and *before* tagging lsb-release with the signed keys.

image_signer.py:
  SignImage
    self._PreSignRecovery
      self.SecurityTestImage
        ensure_sane_lsb-release
      self._ModifyLSBRelease    # CHROMEOS_RELEASE_BOARD has "signed-xxxkeys" added
    sign_official_build.sh

the initial CHROMEOS_RELEASE_BOARD value is added when we build the image:
src/scripts/
  ./build_image
    create_base_image()
  ./build_library/base_image_util.sh
    create_base_image calls cros_set_lsb_release --board=${BOARD}
chromite/scripts/
  ./cros_set_lsb_release.py
    CHROMEOS_RELEASE_BOARD is set directly to the value of --board

so the comment referencing "x86-foo-pvtkeys" seems to refer to a time when we signed the image first and then ran the security tests against the image.  imo we should simply drop that cut.

https://chromium-review.googlesource.com/371678

as for breaking any existing boards, i'm not worried about that as we don't have any that use more than one dash and have gotten as far as signing before

as for variants, Bernie is right that we don't want to use them anymore.  some clarifications:
 - the board name is *not* x86-zgb-he, it's x86-zgb_he (it is the "he" variant of the "x86-zgb" board).  some build/signer code might normalize all underscores into dashes in gs:// paths in a naive attempt to try and standardize, but that's about it.  would be nice to just drop that logic, but it has questionable gain, and we know the code we have now is working.  it's not like we'd ever permit two diff boards that only differ in dashes & underscores.
 - the veyron project really shouldn't have used the variant system, but it's too late now to clean it up.  we should have had a "veyron" baseboard with all the derivatives being just "minnie" and such.  see all the intel examples like jecht & tidus & rikku.  it's even more unfortunate because the veyron platform has been so fsckin popular :).
 - "samus-cheets" is not a variant, it's purely a naming convention we used to say "it's the samus board, but including the cheets addon".  there is no logic derived from its name (unlike the variants which causes the build to change behavior and stack things).  it could just as easily been called "chopsuey" or whatever with the same result.

ok, then let's try vapier's approach and see if that would fix the problem.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/vboot_reference/+/a929c2755eee3dec286585c58f3ac93690c7d270

commit a929c2755eee3dec286585c58f3ac93690c7d270
Author: Mike Frysinger <vapier@chromium.org>
Date: Wed Aug 17 01:22:22 2016

image_signing: drop board hacking for lsb appid checks

BUG= chromium:605595 
TEST=None
BRANCH=None

Change-Id: I8104d80d151440bdd8f419c88bd98592d9f44612
Reviewed-on: https://chromium-review.googlesource.com/371678
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/a929c2755eee3dec286585c58f3ac93690c7d270/scripts/image_signing/ensure_sane_lsb-release.sh

the canaries are fixed.  the change isn't on the signer, but i'm guessing you don't actually want to sign this image (at this time).  so it'll get pushed out eventually at some point.