New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 776679 link

Starred by 1 user

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

[jumbo] Feature: Exclude locally modified files from jumbo

Project Member Reported by brat...@opera.com, Oct 20 2017

Issue description

Idea based on an old Opera jumbo build system: We could try to skip those files that are locally modified when building jumbo units.

Pro: Quicker turnaround when compiling that file.

Con: Can be slower if many files are locally modified every compilation cycle.
Con: Will not see jumbo collisions.
Con: Will still force recompilation of that jumbo unit the first time since it will have changed when a file was removed from it.

 

Comment 1 by brat...@opera.com, Oct 20 2017

Mostyn and jl and I talked about this the other day and it seems it would be most likely to work well if using the output from git status, though the performance of that might be not good enough. Especially if we do as today, and build one jumbo file at a time (that is a lot of git status calls).

The old minimake depended on timestamps and a git hook which is a bit too cumbersome and wouldn't scale well with a project the size of Chromium.
One case that this will miss is when a header file is modified. If modifying that header file causes many .cc files to be recompiled then jumbo is probably a net win but if the header is included by just a single .cc file then jumbo will probably hurt rebuild times. That said, I'm not sure trying to handle the header-file case is worthwhile.

"git status" is slow enough on Windows to make me hesitant to have this run automatically. And, since v8 is jumbified we'd really need to run "git status" on that repo as well.

On the other hand, making it easier to exclude the files being edited could be pretty darned useful. Maybe have a script to run the process on demand? That is, something like this:

    python third_party\depot_tools\update_jumbo_exclude.py out\debug

I haven't looked into how the jumbo file excluding works but it is important that it not perturb any other jumbo file sets. That is, the boundaries between sets must stay static or else extra compilation will be required when the exclusions change.

Comment 3 by brat...@opera.com, Oct 20 2017

A minimal change would be to have jumbo.gni prepare a "recently_modified_files_jumbo.cc which is normally empty but where merge_files_for_jumbo.py inserts recently changed files. They would still not be compiled individually and it could cause a build time regression from all the empty files, but it could be done without changing the structure of the current jumbo implementation.



Components: Build

Sign in to add a comment