New issue
Advanced search Search tips

Issue 863028 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Feature



Sign in to add a comment

provide a way to compile the targets the CQ buildbots would build

Project Member Reported by jochen@chromium.org, Jul 12

Issue description

sometimes I change a ton of files, and then I guess which combination of _unittests and _browsertests targets I have to compile to see that nothing broke.

It would be nice if there was a short-cut that compiles the targets that depend changed files, similar to how the CQ bots don't compile everything by analyze the CL
 
Labels: Pri-3
Status: Available (was: Untriaged)
I'm tentatively marking this as available P3 (going by "it would be nice" phrasing), please update the priority if it's more important than that.

+dpranke and jbudorick FYI, in case you have opinions.

In my possibly naive view, CQ's "analyze" step is just a command that (probably?) can be run locally? Something like:

src/tools/mb/mb.py analyze -m tryserver.chromium.linux -b linux_chromium_rel_ng -f src/tools/mb/mb_config.pyl -v //out/Release input.json output.json
yeah, however, that command overwrites my args.gn in out/Release and I have no idea what goes into input.json and output.json
Cc: dpranke@chromium.org jbudorick@chromium.org
Actually +dpranke and jbudorick FYI, in case you have opinions.
Components: Build
Are you looking to do this locally, or on the bots?

Locally, I'm guessing you don't want to build "all", because that would do it but it would probably build other stuff you care less about?

The list of targets you might care about varies from builder to builder, so you'd either have to specify a particular builder or builders to match, or have MB or something try to infer a match from your GN args. Once you had that, it would also be nice to have a way to easily extract the list of targets from the //testing/buildbot/*.json files.

(Basically this would mean moving the logic the recipe uses to figure this out client-side).

I agree this would be a nice thing to be able to do and is something that I've wanted for some time (bonus points for actually running the tests the way the builder(s) would run them as well, double bonus for isolating and running them all in parallel under swarming ;), and all of this is pretty tractable work.
Cc: w...@chromium.org
+wez as well since I know I've talked to him about this as well.
Labels: -Restrict-View-Google
looking for a way to do this locally, yes.

I'm mostly interested in compiling, so I don't end up uploading a CL that doesn't even compiled because I forgot about updating some obscure callsite
+1 to what Dirk describes in comment #4.

In general our builders all build 'all', with the exception of e.g. Fuchsia, which builds 'gn_all', don't they?

IIUC what jochen@ is looking for is essentially the exact build processing that ninja normally does when it sees that a file is newer than a dependent, only rebuilding affected dependents, but with the additional property that targets with no dependency at all on the modified file are not built, if not present, unless an affected target depends on them.  Sort of an 'all-affected' meta-target for ninja.

GN also has its 'desc' tool, which I think could be easily extended to provide a mode where given a set of top-level <targets> and a list of <files>, returns the subset of <target> with the any dependency on <files>. The down-side to that would be that GN doesn't have a concept of 'all', though.
Cc: martiniss@chromium.org
+martiniss and I discussed this some yesterday, though we were more focused on the bonus point features mentioned in #4.
> GN also has its 'desc' tool, which I think could be easily extended to
> provide a mode where given a set of top-level <targets> and a list of
> <files>, returns the subset of <target> with the any dependency on <files>. 

GN also has an `analyze` tool that does exactly that (this is how analyze is implemented on the bots).

> The down-side to that would be that GN doesn't have a concept of 'all', though.

`gn analyze` actually does (the other GN commands don't), but there are some subtleties to the current implementation where it works for what to compile but not quite for what to test. That would be easy to fix if need be.


Sign in to add a comment