New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 718658 link

Starred by 2 users
Status: Fixed
Owner:
█ shenghua...@chromium.org
Last visit > 30 days ago
Closed: Jun 2017
Cc:
jbudorick@chromium.org
j...@chromium.org
flyboy@chromium.org
rauch+monorail-auto@google.com
█ orodley+autocc@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 724721



Sign in to add a comment

[codesearch] Refactor chromium_codesearch.py

Project Member Reported by shenghua...@chromium.org, May 5 2017 
For clankium codesearch migration, the 'codesearch-gen-clankium-linux' bot needs a recipe which has similar steps in the chromium codesearch gen recipe chromium_codesearch.py:
https://cs.corp.google.com/chromium_build/scripts/slave/recipes/chromium_codesearch.py

Need to refactor the recipe to:
1. make use of mb config for gn args
2. split out steps and create recipe modules s.t. can be called by downstreams.
Project Member

Comment 1 by bugdroid1@chromium.org, May 5 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6bd1fc8a43faee33e11dafd0b9e62e0533ee3324

commit 6bd1fc8a43faee33e11dafd0b9e62e0533ee3324
Author: shenghuazhang <shenghuazhang@chromium.org>
Date: Fri May 05 02:59:00 2017

[codesearch] Add codesearch chromium gen args to mb_config

Will use mb for gn args of chromium_codesearch.py. Add gn configs in
mb_config.pyl
(https://chromium.googlesource.com/chromium/tools/build/+/master/scripts/slave/recipes/chromium_codesearch.py#49)

BUG= 718658 

Review-Url: https://codereview.chromium.org/2865593002
Cr-Commit-Position: refs/heads/master@{#469577}

[modify] https://crrev.com/6bd1fc8a43faee33e11dafd0b9e62e0533ee3324/tools/mb/mb_config.pyl

Comment 2 by emso@chromium.org, May 10 2017

Components: Infra>Codesearch
Project Member

Comment 3 by bugdroid1@chromium.org, May 10 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/290f2bede4df8363122591f27baf118f227841a5

commit 290f2bede4df8363122591f27baf118f227841a5
Author: shenghuazhang <shenghuazhang@chromium.org>
Date: Wed May 10 19:30:48 2017

[codesearch] Add phase to builders in 'chromium.infra.codesearch'

Builder 'codesearch-gen-chromium-chromiumos' has duplicated compilation
(chromiumos, linux). Add phases to mb config to enable double compile in
recipe.

BUG= 718658 

Review-Url: https://codereview.chromium.org/2871673005
Cr-Commit-Position: refs/heads/master@{#470668}

[modify] https://crrev.com/290f2bede4df8363122591f27baf118f227841a5/tools/mb/mb_config.pyl
Project Member

Comment 4 by bugdroid1@chromium.org, May 10 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/a75ef95539f72ced4d30ad85ea35b6664a2ffa3c

commit a75ef95539f72ced4d30ad85ea35b6664a2ffa3c
Author: Shenghua Zhang <shenghuazhang@google.com>
Date: Wed May 10 20:07:39 2017

[codesearah] Refactor chromium_codesearch.py to use run_mb

Bug:718658
Change-Id: I2a560a1a362127b3cb97374369cfd1bff1502c40
Reviewed-on: https://chromium-review.googlesource.com/498094
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Shenghua Zhang <shenghuazhang@google.com>

[modify] https://crrev.com/a75ef95539f72ced4d30ad85ea35b6664a2ffa3c/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_chromiumos_fail.json
[modify] https://crrev.com/a75ef95539f72ced4d30ad85ea35b6664a2ffa3c/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_chromiumos_gen_compile_fail.json
[modify] https://crrev.com/a75ef95539f72ced4d30ad85ea35b6664a2ffa3c/scripts/slave/recipes/chromium_codesearch.py
[modify] https://crrev.com/a75ef95539f72ced4d30ad85ea35b6664a2ffa3c/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_android.json
[modify] https://crrev.com/a75ef95539f72ced4d30ad85ea35b6664a2ffa3c/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_linux.json
[modify] https://crrev.com/a75ef95539f72ced4d30ad85ea35b6664a2ffa3c/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_chromiumos.json
Project Member

Comment 5 by bugdroid1@chromium.org, May 18 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/5eb5de9c8c3da08fa9a1d546955e237ac719f912

commit 5eb5de9c8c3da08fa9a1d546955e237ac719f912
Author: Shenghua Zhang <shenghuazhang@google.com>
Date: Thu May 18 06:15:53 2017

[codesearch] Refactor chromium_codesearch.py by adding recipe modules

chromium_codesearch.py is the recipe of chromium codesearch gen bots.
Most of the recipe flow could be same to clankium codesearch gen bots.
Now refactor this recipe by moving functions as modules, s.t can be reused
by downstream.

Bug:718658
Change-Id: Ibf21ee4cd27a4f2193586f0b5ddaf174efb439e7
Reviewed-on: https://chromium-review.googlesource.com/501129
Commit-Queue: Shenghua Zhang <shenghuazhang@google.com>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Emma Söderberg <emso@chromium.org>
Reviewed-by: Dave Sansome <dsansome@chromium.org>

[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/checkout_generated_files_repo_and_sync.expected/generated_repo_not_set_failed.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/config.py
[modify] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_chromiumos_fail.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/generate_compilation_database.expected/generate_compilation_database_for_linux_failed.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/api.py
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/example.expected/codesearch_gen_chromium_linux_test_basic.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/filter_compilation.py
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/create_and_upload_kythe_index_pack.py
[modify] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipes/chromium_codesearch.py
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/OWNERS
[modify] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_linux.json
[modify] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_android.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/copy_compilation_output.expected/basic.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/create_and_upload_kythe_index_pack.expected/bucket_name_not_set_failed.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/checkout_generated_files_repo_and_sync.expected/basic.json
[modify] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_chromiumos.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/generate_compilation_database.py
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/copy_compilation_output.py
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/example.py
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/build_clang_tool.py
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/build_clang_tool.expected/basic.json
[modify] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipes/chromium_codesearch.expected/full_codesearch_gen_chromium_chromiumos_gen_compile_fail.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/build_clang_tool.expected/run_translation_unit_clang_tool_failed.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/__init__.py
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/filter_compilation.expected/basic.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/generate_compilation_database.expected/basic.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/checkout_generated_files_repo_and_sync.py
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/create_and_upload_kythe_index_pack.expected/basic.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/checkout_generated_files_repo_and_sync.expected/false_sync_generated_files.json
[add] https://crrev.com/5eb5de9c8c3da08fa9a1d546955e237ac719f912/scripts/slave/recipe_modules/codesearch/tests/configs.py

Comment 6 by shenghua...@chromium.org, May 20 2017

Blocking: 724721
Project Member

Comment 7 by bugdroid1@chromium.org, May 24 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/60365931c1bd3e27b9163c308e7482b9d7c700b8

commit 60365931c1bd3e27b9163c308e7482b9d7c700b8
Author: Shenghua Zhang <shenghuazhang@google.com>
Date: Wed May 24 00:16:40 2017

[codesearch] Add config options to codesearch/api.py

codesearch/api.py is shared by downstream clankium recipe, which has
different mb_config_path than the upstream. Add the config option in
this CL.

Bug:718658
Change-Id: I7658cbd1742626597b263abf260de4015602be4b
Reviewed-on: https://chromium-review.googlesource.com/509939
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Emma Söderberg <emso@chromium.org>
Commit-Queue: Shenghua Zhang <shenghuazhang@google.com>

[modify] https://crrev.com/60365931c1bd3e27b9163c308e7482b9d7c700b8/scripts/slave/recipe_modules/codesearch/api.py
Project Member

Comment 8 by bugdroid1@chromium.org, May 24 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/60365931c1bd3e27b9163c308e7482b9d7c700b8

commit 60365931c1bd3e27b9163c308e7482b9d7c700b8
Author: Shenghua Zhang <shenghuazhang@google.com>
Date: Wed May 24 00:16:40 2017

[codesearch] Add config options to codesearch/api.py

codesearch/api.py is shared by downstream clankium recipe, which has
different mb_config_path than the upstream. Add the config option in
this CL.

Bug:718658
Change-Id: I7658cbd1742626597b263abf260de4015602be4b
Reviewed-on: https://chromium-review.googlesource.com/509939
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Emma Söderberg <emso@chromium.org>
Commit-Queue: Shenghua Zhang <shenghuazhang@google.com>

[modify] https://crrev.com/60365931c1bd3e27b9163c308e7482b9d7c700b8/scripts/slave/recipe_modules/codesearch/api.py
Project Member

Comment 9 by bugdroid1@chromium.org, May 24 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/60365931c1bd3e27b9163c308e7482b9d7c700b8

commit 60365931c1bd3e27b9163c308e7482b9d7c700b8
Author: Shenghua Zhang <shenghuazhang@google.com>
Date: Wed May 24 00:16:40 2017

[codesearch] Add config options to codesearch/api.py

codesearch/api.py is shared by downstream clankium recipe, which has
different mb_config_path than the upstream. Add the config option in
this CL.

Bug:718658
Change-Id: I7658cbd1742626597b263abf260de4015602be4b
Reviewed-on: https://chromium-review.googlesource.com/509939
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Emma Söderberg <emso@chromium.org>
Commit-Queue: Shenghua Zhang <shenghuazhang@google.com>

[modify] https://crrev.com/60365931c1bd3e27b9163c308e7482b9d7c700b8/scripts/slave/recipe_modules/codesearch/api.py

Comment 10 by shenghua...@chromium.org, Jun 1 2017

Status: Fixed (was: Assigned)
►

Sign in to add a comment