This change seems like the smoking gun: https://chromium-review.googlesource.com/c/483342/

It's possible that the PFQ builders aren't set up correctly?

I'll revert above CL speculatively and see whether it helps.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/00b062e732629c252673b4cba77a9206edf5cbce

commit 00b062e732629c252673b4cba77a9206edf5cbce
Author: Mattias Nissler <mnissler@chromium.org>
Date: Fri Apr 28 10:14:39 2017

Revert "chromeos-chrome: add host_pkg_config for cross-compilation"

This reverts commit c27bea7ccb346760ee278b2d42ab3cd6188467c5.

Reason for revert: Likely the culprit for TestSimpleChromeWorkflow breakage in Chrome PFQ builders, see  http://crbug.com/716399 .

Original change's description:
> chromeos-chrome: add host_pkg_config for cross-compilation
> 
> This new build variable ensures that host-only compilation helpers
> aren't incorrectly linked against libraries in the current board
> sysroot.
> 
> BUG= chromium:710841 
> TEST=cbuildbot kevin-release
> 
> Change-Id: Ifc667fd55cfaa00e2721c7894684909b6bd08b46
> Reviewed-on: https://chromium-review.googlesource.com/483342
> Commit-Ready: Richard Townsend <richard.townsend@arm.com>
> Tested-by: Richard Townsend <richard.townsend@arm.com>
> Reviewed-by: Mike Frysinger <vapier@chromium.org>
> 

TBR=vapier@chromium.org,richard.townsend@arm.com
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= chromium:710841 ,  chromium:716399 

Change-Id: If877ffdca42ce697957d7a438e4675a9cf7541ee
Reviewed-on: https://chromium-review.googlesource.com/489948
Reviewed-by: Mattias Nissler <mnissler@chromium.org>
Tested-by: Mattias Nissler <mnissler@chromium.org>

[modify] https://crrev.com/00b062e732629c252673b4cba77a9206edf5cbce/chromeos-base/chromeos-chrome/chromeos-chrome-9999.ebuild

This was the right thing to do, I'm not sure why x86_64-pc-linux-gnu-pkg-config might not be available in this environment, or why it passed the commit queue.

The revert fixed the TestSimpleChromeWorkflow step.

hrm, simple chrome.  the build pkg-config that is available inside the chroot is not guaranteed to be available outside.  we prob need to update the chromite/cli/cros/cros_chrome_sdk.py script first to set this new knob, and then reland that chromeos-chrome ebuild change.

OK, so I wasn't able to replicate the precise build, but I was able to the error via the cros shell. It seems to relate from the use of $(tc-getBUILD_PKG_CONFIG), which seems to return 'x86_64-pc-linux-gnu-pkg-config', which isn't provided in the arm-generic toolchain inside the cros shell. Patchset #2 of https://chromium-review.googlesource.com/483342 would probably correct this, but it's not a very clean solution.

Oops, guess vapier beat me to it. I'll add a follow up task inside https://bugs.chromium.org/p/chromium/issues/detail?id=710841

OK, so I still haven't been able to replicate the problem in the simple chrome environment. Would it be possible to get more of the build log so I can assess exactly what it's doing?

FWIW, attached is the full output from the build step. I'm not sure whether it's an actual problem or some build config issue. Do you ensure that pkg-config is installed in the build root?

Deleted: pkg_config_build_error.txt 27.0 KB

pkg_config_build_error.txt 27.0 KB View Download

we don't ship a pkg-config for your active build system

just update cros_chrome_sdk.py as i mentioned earlier to set the variable and clobber whatever the ebuild has set up

So replicating the build steps as closely as possible and modifying the simple chrome environment a bit to simulate the unreverted ebuild, I think I have managed to come up with a patch for this. All it involves is adding 'gn_args["host_pkg_config"] = "pkg-config"' in cli/cros/cros_chrome_sdk.py at around line 703. If I can get it signed off internally, I'll be able to push it quite soon.

The current fix is available at https://chromium-review.googlesource.com/#/c/496446/. I don't quite understand yet the relationship between the ebuild's configuration and simple chrome's environment, but adding it into the the SDK script does seem to override the pkg_config setting in the environment's GN_ARGS.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/4e87bfa4ee18aa9d1799ea350eea3a0a406470b9

commit 4e87bfa4ee18aa9d1799ea350eea3a0a406470b9
Author: Richard Townsend <Richard.Townsend@arm.com>
Date: Mon May 08 19:43:43 2017

Add host_pkg_config for simple chrome builds.

host_pkg_config is used to make sure that compiled_actions targets
find the right libraries/include headers required for compilation on the
host machine for use elsewhere in the build process. It's related with
 crbug.com/710841  and  crbug.com/710006 , which both block  crbug.com/595493 

BUG= chromium:716399 
TEST=Built with simple-chrome environment

Change-Id: I2daaf5d7f89d7f5229946e0f19fe44ece530e714
Reviewed-on: https://chromium-review.googlesource.com/496446
Commit-Ready: Richard Townsend <richard.townsend@arm.com>
Tested-by: Richard Townsend <richard.townsend@arm.com>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/4e87bfa4ee18aa9d1799ea350eea3a0a406470b9/cli/cros/cros_chrome_sdk.py