New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 619083 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug

Blocking:
issue 431177



Sign in to add a comment

GN: tool implemented via a script does not have a dependency on the script

Project Member Reported by sdefresne@chromium.org, Jun 10 2016

Issue description

Some tool on iOS/Mac are implemented as python script. If the script is updated, the targets build via the tool are not updated. I think this can cause problem when a bug is found in one of those tools.

brettw: is there a way to specify those dependency that we forgot to use or is this unsupported? In case we need to change a tool, do we force a clobber to ensure the build is correct?




 
Labels: OS-All
actions() should automatically be considering the script as a dependency. 

Can you point me at a specific example so I can look into it?
I was not talking about "action" but about tools, for example "copy_bundle_data" tool uses the script build/config/mac/copy_bundle_data.py (see build/toolchain/mac/BUILD.gn).

Today I found a bug in that script on iOS in how it deals with .strings file, but even after updating the script, ninja did not notice the need to rebuild anything.

My question is this: is it possible to have "tool" declare dependencies on the script they use? If yes, how? If no, should they? 
Components: Build
Labels: -Pri-3 Build-Tools-GN Pri-1
Ah. Good question. Yeah, I don't think we do anything to track those as dependencies today, but we should. Bumping the priority, since this could produce incorrect builds.

Comment 5 by brettw@chromium.org, Jun 10 2016

GN has the concept of toolchain deps (though this will affect everything). But a tool dependency, like the toolchain deps, is inefficient to express because it requires a dependency be listed on every single file in the build, which is inefficient (bloats the ninja files).

If this was only for bundle data, it's fine. Maybe there should be a new variable on a tool like inputs (to mirror actions), and then we use that for copy_bundle_data.

A much more efficient way to implement this in the short term is to put a version number on the command line to the tool, and increment it when the tool changes. Ninja will recompile things when their command line changes.

This is what we do for rolling Clang. Ovsiouly this is annoying and manual, but we can do it today if the problem comes up.
Labels: -Pri-1 Pri-3
When building chrome on iOS we have tens of thousands of copy_bundle_data "build" steps so inputs will probably be too expensive. I'll use version numbers on the command line trick to get this working then. Thank you for the tips.

Lowering priority as there are workarounds. 

Comment 7 by rsesek@chromium.org, Jun 16 2016

 Issue 620732  has been merged into this issue.

Comment 8 by rsesek@chromium.org, Jun 16 2016

Blocking: 431177
Labels: -Pri-3 Pri-1
Owner: rsesek@chromium.org
Status: Started (was: Available)
Since several of our scripts pass sys.argv through to a subtool, I tested if setting env var instead would work instead of a flag argument, e.g. |command = "SCRIPT_VERSION=1 python build/toolchain/mac/copy_bundle_data.py"|. And that works, so I'll do that for now.

However I think this is pretty fragile, as it requires someone to know to increment the version whenever they change the script. I'll do this now so we unbork things, but expressing this as a dependency is the right thing to do from a correctness perspective.
Project Member

Comment 9 by bugdroid1@chromium.org, Jun 17 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f2eb108ab1e65962104bfde275743369b0e7665b

commit f2eb108ab1e65962104bfde275743369b0e7665b
Author: Robert Sesek <rsesek@chromium.org>
Date: Fri Jun 17 22:02:54 2016

[Mac/iOS/GN] Place a version in the command of toolchain tools implemented using scripts.

When a tool is implemented as a script, modification of the script does not
cause any dependent build steps to be dirtied and rebuilt, since the script
isn't listed as an input. To hack around this, use another script to get the
tool script's modification time and use that as the command line tool version.
This still won't cause rebuild if just the script changes, but when the build
files are regenerated (like after apply_patch and generate_build_files on the
bots), the dependent build steps will get rebuilt.

BUG=619083
R=brettw@chromium.org, sdefresne@chromium.org

Review URL: https://codereview.chromium.org/2075703002 .

Cr-Commit-Position: refs/heads/master@{#400519}

[modify] https://crrev.com/f2eb108ab1e65962104bfde275743369b0e7665b/build/toolchain/mac/BUILD.gn
[add] https://crrev.com/f2eb108ab1e65962104bfde275743369b0e7665b/build/toolchain/mac/get_tool_mtime.py

Cc: rsesek@chromium.org
Labels: -Pri-1 Pri-2
Owner: ----
Status: Available (was: Started)
Making this back available so that we can fix this the proper way, by specifying inputs to tools.
Project Member

Comment 11 by sheriffbot@chromium.org, Jun 21 2017

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available. If you change it back, also remove the "Hotlist-Recharge-Cold" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: -Hotlist-Recharge-Cold
Status: Available (was: Untriaged)
Project Member

Comment 13 by sheriffbot@chromium.org, Jun 21 2018

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Status: Available (was: Untriaged)

Sign in to add a comment