New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 611319 link

Starred by 7 users

Issue metadata

Status: Fixed
Owner: ----
Closed: Mar 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug

Blocking:
issue 617013



Sign in to add a comment

Flaky compile failures on "third_party/kasko/kasko_features.h"

Project Member Reported by st...@chromium.org, May 12 2016

Issue description

Some build examples:
https://build.chromium.org/p/chromium/builders/Win/builds/43317
https://build.chromium.org/p/chromium/builders/Win/builds/43224
https://build.chromium.org/p/chromium/builders/Win/builds/43247
https://build.chromium.org/p/chromium/builders/Win/builds/43088
https://build.chromium.org/p/chromium.chrome/builders/Google%20Chrome%20Win/builds/6852
https://build.chromium.org/p/chromium.chrome/builders/Google%20Chrome%20Win/builds/6906


One error example:
FAILED: obj/chrome/installer/setup/setup_lib.installer_crash_reporting.obj 
ninja -t msvc -e environment.x86 -- C:\b\build\slave\cache\cipd\goma/gomacc "C:\b\depot_tools\win_toolchain\vs_files\95ddda401ec5678f15eeed01d2bee08fcbc5ee97\VC\bin\amd64_x86\cl.exe" /nologo /showIncludes /FC @obj\chrome\installer\setup\setup_lib.installer_crash_reporting.obj.rsp /c ..\..\chrome\installer\setup\installer_crash_reporting.cc /Foobj\chrome\installer\setup\setup_lib.installer_crash_reporting.obj /Fdobj\chrome\setup_lib.cc.pdb 
c:\b\build\slave\win\build\src\components\crash\content\app\crashpad.h(15): fatal error C1083: Cannot open include file: 'third_party/kasko/kasko_features.h': No such file or directory


Below is my analysis and findings:

chrome/installer/setup/installer_crash_reporting.cc in "chrome_installer.gypi:setup_lib" depends on components/crash/content/app/crashpad.h, and then indirectly depends on third_party/kasko/kasko_features.h generated by "third_party/kasko/kasko.gyp:kasko_features".

There is a indirect dependency path "chrome_installer.gypi:setup_lib" -> "components/crash.gypi:crash_component" -> "third_party/kasko/kasko.gyp:kasko" -> "third_party/kasko/kasko.gyp:kasko_features", but "third_party/kasko/kasko.gyp:kasko_features" and others are not set 'hard_dependency': 1.

This is very similar to another flaky compile failure in https://bugs.chromium.org/p/chromium/issues/detail?id=606829.
 

Comment 1 by st...@chromium.org, May 12 2016

Cc: gab@chromium.org robertshield@chromium.org grt@chromium.org
cc owners of chrome/installer/ for further triage :)

Comment 2 by st...@chromium.org, May 12 2016

Components: Tools>Test>FindIt

Comment 3 by grt@chromium.org, May 12 2016

Owner: pmonette@chromium.org
Status: Assigned (was: Available)
Yo Patrick: you added feature support for Kasko, right? Could you look into this?

Comment 4 by st...@chromium.org, May 14 2016

It seems this happened again.

https://build.chromium.org/p/chromium/builders/Win/builds/43379

Comment 5 by est...@chromium.org, May 14 2016

Cc: ananta@chromium.org blundell@chromium.org pmonette@chromium.org
 Issue 611849  has been merged into this issue.

Comment 6 by est...@chromium.org, May 14 2016

I have a possible fix here: https://codereview.chromium.org/1976113003/

Comment 7 by roc...@chromium.org, May 19 2016

Still seeing this flake

Comment 8 by est...@chromium.org, May 19 2016

Owner: blundell@chromium.org
I suspect that the problem is that the kasko GYP target is doing export_dependent_settings on kasko_features only when kasko is enabled, but dependees include kasko_features.h regardless of whether kasko is enabled (they have to in order to use the buildflag that tells them whether kasko is enabled :).

Hopefully https://codereview.chromium.org/2007513002/ fixes the problem.
Project Member

Comment 10 by bugdroid1@chromium.org, May 23 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/af480cf00ad993ac39e9e3253701879b17d2140b

commit af480cf00ad993ac39e9e3253701879b17d2140b
Author: blundell <blundell@chromium.org>
Date: Mon May 23 14:07:37 2016

GYP: In kasko target, export dependent settings of kasko_features

Many targets depend on the kasko target for a dependency on the generated
header kasko_features.h. However, when kasko is not enabled, the kasko
target was not exporting the fact that kasko_features is a hard
dependency, causing flaky build failures. This CL fixes the bug.

BUG= 611319 
TBR=pmonette

Review-Url: https://codereview.chromium.org/2007513002
Cr-Commit-Position: refs/heads/master@{#395314}

[modify] https://crrev.com/af480cf00ad993ac39e9e3253701879b17d2140b/third_party/kasko/kasko.gyp

Status: Fixed (was: Assigned)
Speculatively marking this bug fixed; reopen if necessary.

Comment 12 by st...@chromium.org, May 23 2016

Status: Assigned (was: Fixed)
The flake still happens after the above CL, so reopen.
blundell@, mind another look?

https://build.chromium.org/p/chromium.chrome/builders/Google%20Chrome%20Win/builds/7359
Labels: Sheriff-Chromium
There have been 3 failures in the past 50 runs.

Labels: -Sheriff-Chromium
This has an assignee, so taking out of the Sheriff queue.
This is pretty much my last idea:

https://codereview.chromium.org/2038473002/
Status: Started (was: Assigned)
Project Member

Comment 17 by bugdroid1@chromium.org, Jun 2 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d507af25f7281161a3eb0d44a0deef057e46c8dc

commit d507af25f7281161a3eb0d44a0deef057e46c8dc
Author: blundell <blundell@chromium.org>
Date: Thu Jun 02 14:02:45 2016

Specify direct dependency of //components/crash on kasko_features.h

//components/crash/content/app has a public dependency on the generated
header kasko_features.h. However, the GYP and GN files specify this
dependency only transitively via a dependency on //third_party/kasko
(which //components/crash/content/app does not actually depend on). This
CL changes the dependency to be directly on
//third_party/kasko:kasko_features as a speculative fix for the build
failures listed in  crbug.com/611319 . Note that in any case this change
is correct (whether it ends up resolving the flaky build failures or no).

BUG= 611319 
TBR=pmonette

Review-Url: https://codereview.chromium.org/2038473002
Cr-Commit-Position: refs/heads/master@{#397390}

[modify] https://crrev.com/d507af25f7281161a3eb0d44a0deef057e46c8dc/components/crash.gypi
[modify] https://crrev.com/d507af25f7281161a3eb0d44a0deef057e46c8dc/components/crash/content/app/BUILD.gn
[modify] https://crrev.com/d507af25f7281161a3eb0d44a0deef057e46c8dc/components/crash/content/app/DEPS

Status: Fixed (was: Started)
Marking this speculatively fixed again. Reopen if the failure still occurs, but I don't have any more ideas in that case :P.
Status: Available (was: Fixed)
This issue still occurs: https://build.chromium.org/p/chromium/builders/Win/builds/44040/steps/compile/logs/stdio.

I don't have any more ideas :\.

 Issue 617434  has been merged into this issue.
Owner: ----
Project Member

Comment 22 by bugdroid1@chromium.org, Jun 6 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0983a7c38cb33066a2ad57f6da0dc21cd7376493

commit 0983a7c38cb33066a2ad57f6da0dc21cd7376493
Author: sebmarchand <sebmarchand@chromium.org>
Date: Mon Jun 06 18:10:31 2016

Revert of Specify direct dependency of //components/crash on kasko_features.h (patchset #1 id:1 of https://codereview.chromium.org/2038473002/ )

Reason for revert:
It's breaking the official win-asan builder (https://uberchromegw.corp.google.com/i/official.desktop/builders/win-asan/builds/268)

Original issue's description:
> Specify direct dependency of //components/crash on kasko_features.h
>
> //components/crash/content/app has a public dependency on the generated
> header kasko_features.h. However, the GYP and GN files specify this
> dependency only transitively via a dependency on //third_party/kasko
> (which //components/crash/content/app does not actually depend on). This
> CL changes the dependency to be directly on
> //third_party/kasko:kasko_features as a speculative fix for the build
> failures listed in  crbug.com/611319 . Note that in any case this change
> is correct (whether it ends up resolving the flaky build failures or no).
>
> BUG= 611319 
> TBR=pmonette
>
> Committed: https://crrev.com/d507af25f7281161a3eb0d44a0deef057e46c8dc
> Cr-Commit-Position: refs/heads/master@{#397390}

TBR=mark@chromium.org,pmonette@chromium.org,blundell@chromium.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG= 611319 

Review-Url: https://codereview.chromium.org/2047473002
Cr-Commit-Position: refs/heads/master@{#398063}

[modify] https://crrev.com/0983a7c38cb33066a2ad57f6da0dc21cd7376493/components/crash.gypi
[modify] https://crrev.com/0983a7c38cb33066a2ad57f6da0dc21cd7376493/components/crash/content/app/BUILD.gn
[modify] https://crrev.com/0983a7c38cb33066a2ad57f6da0dc21cd7376493/components/crash/content/app/DEPS

Blocking: 617013
Cc: brettw@chromium.org sebmarchand@chromium.org chrisha@chromium.org danakj@chromium.org
 Issue 626796  has been merged into this issue.
Yes, Kasko is being ripped out of SyzyAsan, so I'll just remove the Kasko dependencies in Crashpad.
Have GN folks seen similar flake with other generated feature-headers?
danakj@: Just to be clear, the issue here is with GYP. My understanding is that the way that GN handles auto-generated headers makes it much less sensitive to this kind of issue by design with the tradeoff of it being more conservative about inserting serialization into the build process; I'm not familiar with the details there though.
Status: Fixed (was: Available)
Marking this as fixed, as kasko has long since been removed.

Sign in to add a comment