New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 821982 link

Starred by 2 users

Issue metadata

Status: Duplicate
Owner:
Closed: Apr 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug

Blocked on:
issue 526154



Sign in to add a comment

Code search xrefs to generated files only work for linux

Project Member Reported by jamescook@chromium.org, Mar 14 2018

Issue description

Click "LoginScreenPtr" here: https://cs.chromium.org/chromium/src/chrome/browser/ui/ash/login_screen_client.h?l=108

Tries to load this file:
https://cs.chromium.org/chromium/src/out/Debug/gen/ash/public/interfaces/login_screen.mojom.h?type=cs&sq=package:chromium&l=56

Fails:
Can't load 'chromium/src/out/Debug/gen/ash/public/interfaces/login_screen.mojom.h': Could not resolve path src/out/Debug/gen/ash/public/interfaces/login_screen.mojom.h.

Makes it hard to explore generated code.

 
(Also, chopsdash does not show a code search service degradation today, though there was one yesterday.)

Comment 2 by aga...@chromium.org, Mar 27 2018

Blockedon: 526154
Cc: j...@chromium.org
Status: Available (was: Untriaged)
Summary: Code search xrefs to generated files only work for linux (was: Code search not loading generated mojom header files)
This is due to the odd mashup of Linux and ChromeOS information that our codesearch currently surfaces.

Codesearch currently does the following things:
1) Compile chromium for linux, and commit the resulting src/out/ directory to a git repo
2) Compile chromium for linux, and use the resulting objects to construct cross-references for codesearch
3) Compile chromium for chromeos, drop all the resulting objects which overlap with objects produced by the linux build, and use the remaining ones to construct additional crossreferences for codesearch

So the result is that, while we do surface some chromeos-specific crossreferences (yay, that's a win!), if those xrefs point to generated files, they end up going nowhere because the genfiles are only the linux ones.

It's not clear how best to fix this, even once we do have a better multi-platform cross-references story. Blocking this bug on that, and CCing Joey so he can be thinking about this.
Wacky idea: Just compile for chromeos. It's mostly a superset of linux (e.g. OS_LINUX is defined for target_os="chromeos" builds).

Comment 4 by aga...@chromium.org, Mar 28 2018

Yep, we're considering this in the short term, depending on how hard it looks to do things "right".

Comment 5 by j...@chromium.org, Apr 11 2018

Cc: -j...@chromium.org
Owner: j...@chromium.org
Status: Started (was: Available)
Project Member

Comment 6 by bugdroid1@chromium.org, Apr 16 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/28b929f17a06949c2315e1a4256ff297293d2bd7

commit 28b929f17a06949c2315e1a4256ff297293d2bd7
Author: Joey Scarr <jsca@google.com>
Date: Mon Apr 16 00:47:20 2018

Exclude some files from the generated output repo.

Many of these files are generated by the Android build, and they clog up
the repo.

Bug:  821982 
Change-Id: I84f3dbe46aca88100be8a64e4c0fbb2ff9a1aa4d
Reviewed-on: https://chromium-review.googlesource.com/1004891
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Joey Scarr <jsca@chromium.org>

[modify] https://crrev.com/28b929f17a06949c2315e1a4256ff297293d2bd7/scripts/slave/sync_generated_files_codesearch.py

Project Member

Comment 8 by bugdroid1@chromium.org, Apr 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/9ce5d5fb741326161582463bf052324a72935d6c

commit 9ce5d5fb741326161582463bf052324a72935d6c
Author: Joey Scarr <jsca@google.com>
Date: Wed Apr 18 00:39:28 2018

Start pushing Android generated files to Git again.

As a precautionary measure, I've switched this back to the 'android' branch until I can measure the impact on the size of the Git repo (in case there were some large binaries I missed). That way if something goes wrong, we can just delete the branch again.

Bug:  821982 
Change-Id: Ia9ea683f1af706b257f819e300ada5aa9b67490a
Reviewed-on: https://chromium-review.googlesource.com/1013673
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Alan Bram <flyboy@chromium.org>
Commit-Queue: Joey Scarr <jsca@chromium.org>

[modify] https://crrev.com/9ce5d5fb741326161582463bf052324a72935d6c/scripts/slave/recipes/chromium_codesearch.py
[modify] https://crrev.com/9ce5d5fb741326161582463bf052324a72935d6c/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_android_with_revision.json
[modify] https://crrev.com/9ce5d5fb741326161582463bf052324a72935d6c/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_android.json

Project Member

Comment 9 by bugdroid1@chromium.org, Apr 24 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/6afd5d57bfe3818b7a111c7e42c3689db2dc5111

commit 6afd5d57bfe3818b7a111c7e42c3689db2dc5111
Author: Joey Scarr <jsca@google.com>
Date: Tue Apr 24 00:18:13 2018

Rsync generated files using a whitelist instead of a blacklist.

This removes a lot of unnecessary junk from src/out in Code Search. I've
verified that this whitelist is sane by analysing the filetype counts of the
current repo:

   6587 h
   4708 cc
   1298 js
    598 p
    539 v
    271 json
    244 js_library
    196 sources_list
    196 deps_sources_list
    162 strings
     99 html
     65 stamp
     63 png
     46 cpp
     17 inc
     17 idl
     15 tmp
     15 py
     15 c
     14 txt
     12 log
     11 proto
     10 pickle
     10 pb
      3 svg
      2 rc
      2 json5
      2 grd
      2 bro
      1 xml
      1 mac
      1 css

Bug:  821982 
Change-Id: I8bbb9ff8f0feaa0a2635e2a21e65ade845434f28
Reviewed-on: https://chromium-review.googlesource.com/1023629
Commit-Queue: Joey Scarr <jsca@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>

[modify] https://crrev.com/6afd5d57bfe3818b7a111c7e42c3689db2dc5111/scripts/slave/sync_generated_files_codesearch.py

Project Member

Comment 10 by bugdroid1@chromium.org, Apr 30 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/386361d8d704bdac9561616ce9c924837c74136e

commit 386361d8d704bdac9561616ce9c924837c74136e
Author: Joey Scarr <jsca@google.com>
Date: Mon Apr 30 00:27:34 2018

Rsync generated files to the right directory.

This CL fixes a bug where the generated files for Android were being
committed to the wrong directory in Git (and also being pulled from the
results of an old compile).

Bug:  821982 
Change-Id: I9758dacb940f6b176e352c545ab8205461f5e960
Reviewed-on: https://chromium-review.googlesource.com/1023521
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Joey Scarr <jsca@chromium.org>

[modify] https://crrev.com/386361d8d704bdac9561616ce9c924837c74136e/scripts/slave/recipe_modules/codesearch/api.py
[modify] https://crrev.com/386361d8d704bdac9561616ce9c924837c74136e/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_android.json
[modify] https://crrev.com/386361d8d704bdac9561616ce9c924837c74136e/scripts/slave/recipes/chromium_codesearch.py
[modify] https://crrev.com/386361d8d704bdac9561616ce9c924837c74136e/scripts/slave/recipe_modules/codesearch/config.py
[modify] https://crrev.com/386361d8d704bdac9561616ce9c924837c74136e/scripts/slave/sync_generated_files_codesearch.py
[modify] https://crrev.com/386361d8d704bdac9561616ce9c924837c74136e/scripts/slave/README.recipes.md
[modify] https://crrev.com/386361d8d704bdac9561616ce9c924837c74136e/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_android_with_revision.json
[modify] https://crrev.com/386361d8d704bdac9561616ce9c924837c74136e/scripts/slave/recipe_modules/codesearch/tests/checkout_generated_files_repo_and_sync.py
[add] https://crrev.com/386361d8d704bdac9561616ce9c924837c74136e/scripts/slave/recipe_modules/codesearch/tests/checkout_generated_files_repo_and_sync.expected/specified_branch_and_out_dir.json

Comment 11 by j...@chromium.org, Apr 30 2018

Mergedinto: 665761
Status: Duplicate (was: Started)
Marking this as a dupe of an older issue and will post further updates to that one.
Project Member

Comment 13 by bugdroid1@chromium.org, May 15 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/dde7306967ecbaafdf9fe9381a9c81885255875e

commit dde7306967ecbaafdf9fe9381a9c81885255875e
Author: Joey Scarr <jsca@chromium.org>
Date: Tue May 15 03:55:15 2018

Revert "Write Windows generated files to out/win/Debug."

This reverts commit 8c386c9f842817d3ebc855d536767ca49169d183.

Reason for revert: Turns out you can't have a platform name in the output directory.  http://crbug.com/548283 

Original change's description:
> Write Windows generated files to out/win/Debug.
> 
> Bug:  821982 
> Change-Id: I4971bf6e8be33af363c6692b2bb47f61e94bea7f
> Reviewed-on: https://chromium-review.googlesource.com/1051186
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Joey Scarr <jsca@chromium.org>

TBR=agable@chromium.org,jsca@chromium.org

Change-Id: I90f9e2c9faba06b1716ccd0fe294eb38ba2f7556
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  821982 
Reviewed-on: https://chromium-review.googlesource.com/1058948
Reviewed-by: Joey Scarr <jsca@chromium.org>
Commit-Queue: Joey Scarr <jsca@chromium.org>

[modify] https://crrev.com/dde7306967ecbaafdf9fe9381a9c81885255875e/scripts/slave/README.recipes.md
[modify] https://crrev.com/dde7306967ecbaafdf9fe9381a9c81885255875e/scripts/slave/recipes/chromium_codesearch.py
[modify] https://crrev.com/dde7306967ecbaafdf9fe9381a9c81885255875e/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_win.json
[modify] https://crrev.com/dde7306967ecbaafdf9fe9381a9c81885255875e/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_win_delete_generated_files_fail.json
[modify] https://crrev.com/dde7306967ecbaafdf9fe9381a9c81885255875e/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_win_with_revision.json

Project Member

Comment 14 by bugdroid1@chromium.org, May 15 2018

Sign in to add a comment