New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 642914 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Sep 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 1
Type: Bug-Regression



Sign in to add a comment

ANGLE cannot land via CQ. Solution not being processed in bot_update

Project Member Reported by cwallez@chromium.org, Aug 31 2016

Issue description

Starting from around 12:00 MTV time, the ANGLE CQ bots' "git diff to analyze" step started returning nothing, causing no compilation to occur and not tests to be run.

On this bot https://build.chromium.org/p/tryserver.chromium.angle/builders/linux_angle_rel_ng

The last "correct" build was https://build.chromium.org/p/tryserver.chromium.angle/builders/linux_angle_rel_ng/builds/2060/steps/git%20diff%20to%20analyze%20patch/logs/stdio and the first failing was https://build.chromium.org/p/tryserver.chromium.angle/builders/linux_angle_rel_ng/builds/2061/steps/git%20diff%20to%20analyze%20patch/logs/stdio

This is similar to  https://crbug.com/606150 

In the meantime, asked everyone to stop CQ+2ing patch.
 
Cc: andyb...@chromium.org

Comment 2 by kbr@chromium.org, Aug 31 2016

Cc: dpranke@chromium.org
Labels: OS-All

Comment 3 by kbr@chromium.org, Aug 31 2016

Components: Infra>CQ
Cc: cwallez@chromium.org
Cc: -kbr@chromium.org aga...@chromium.org tandrii@chromium.org
Cc: -cwallez@chromium.org -tandrii@chromium.org kbr@chromium.org
Description: Show this description
Cc: -aga...@chromium.org iannucci@chromium.org
The recent roll sets gerrit_rebase_patch_ref to True by default. This means that the block here https://chromium.googlesource.com/chromium/tools/depot_tools/+/master/recipe_modules/bot_update/resources/bot_update.py#1090 is run before git('reset', '--soft', base_rev, cwd=root). This is mostly likely the problem. However...

What’s odd is that in build https://build.chromium.org/p/tryserver.chromium.angle/builders/linux_angle_rel_ng/builds/2063, the stdio for the bot_update step (https://build.chromium.org/p/tryserver.chromium.angle/builders/linux_angle_rel_ng/builds/2063/steps/bot_update/logs/stdio) doesn’t contain '===Rebasing===' OR '===Applying gerrit ref==='
Looks like the solution never gets processed according to https://chromium.googlesource.com/chromium/tools/depot_tools/+/master/recipe_modules/bot_update/resources/bot_update.py#1287

In this case,
patch_root = 'src/third_party/angle' and
package['name'] = 'src'

This means 
patch_root != package['name] and 
'src'.startswith('src/third_party/angle') == false so the solution never processed and the patch is never applied.
And by `package` I mean `solution`.
Cc: hinoka@chromium.org
It looks like the analysis in #9 is correct, based on how I read the code.

cc-ing hinoka@, who wrote this code. See also https://codereview.chromium.org/549223003

Comment 12 by kbr@chromium.org, Sep 1 2016

These bots have a special gclient configuration which pulls top-of-tree ANGLE. I don't know whether that's related or whether something changed recently. See:
https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_tests/chromium_gpu_fyi.py?q=angle_top_of_tree&l=27

Comment 13 by kbr@chromium.org, Sep 1 2016

Labels: -Type-Bug Type-Bug-Regression

Comment 14 by kbr@chromium.org, Sep 1 2016

Note: we should be looking for infrastructure changes within the past 24 hours that may have regressed this behavior. This was working earlier today Pacific time.

Status: Assigned (was: Available)
My change today revealed this behavior by making gerrit_rebase_patch_ref True by default. Assigning to tandrii as he will be up soon and can either:

1) Identify the right thing to do to ensure the solution is processed correctly

or

2) Disable gerrit_rebase_patch_ref to mask the broken behavior again so that we can investigate.
Summary: ANGLE cannot land via CQ. Solution not being processed in bot_update (was: ANGLE CQ: git diff returning no files)
Status: Started (was: Assigned)
I glimpsed over analysis, that looks about right.

Let's disable gerrit rebase for chromium trybot first. This will bring old behavior to Angle. Then we can think about how to bring rebase.
Project Member

Comment 18 by bugdroid1@chromium.org, Sep 1 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/39d64ecde5e09e2a7494fe957261e25d0d8df172

commit 39d64ecde5e09e2a7494fe957261e25d0d8df172
Author: tandrii <tandrii@chromium.org>
Date: Thu Sep 01 13:04:09 2016

Temporary fix for Angle trybots.

BUG= 642914 
TBR=andybons@chromium.org,sergiyb@chromium.org

Review-Url: https://codereview.chromium.org/2299993003

[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipe_modules/auto_bisect/example.expected/android_arm64_bisector.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipe_modules/auto_bisect/example.expected/android_bisector.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipe_modules/auto_bisect/example.expected/basic.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipe_modules/auto_bisect/example.expected/basic_bisect_script.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipe_modules/auto_bisect/example.expected/mac_bisector.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipe_modules/auto_bisect/example.expected/reversed_basic.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipe_modules/auto_bisect/example.expected/windows_bisector.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipe_modules/auto_bisect/example.expected/windows_x64_bisector.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipe_modules/chromium/example.expected/basic_out_dir.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipe_modules/chromium_checkout/api.py
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisect.expected/basic.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisect.expected/basic_return_code_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisect.expected/broken_bad_revision_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisect.expected/broken_good_revision_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisection/desktop_bisect.expected/basic.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisection/desktop_bisect.expected/basic_linux_bisect_tester_recipe.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisection/desktop_bisect.expected/basic_linux_bisect_tester_recipe_ret_code.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisection/desktop_bisect.expected/basic_perf_tryjob.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisection/desktop_bisect.expected/basic_perf_tryjob_with_metric.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisection/desktop_bisect.expected/basic_perf_tryjob_with_revisions.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisection/desktop_bisect.expected/perf_cq_no_benchmark_to_run.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisection/desktop_bisect.expected/perf_cq_no_changes.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisection/desktop_bisect.expected/perf_cq_run_benchmark.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisection/desktop_bisect.expected/perf_tryjob_config_error.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/bisection/desktop_bisect.expected/perf_tryjob_failed_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/build_dynamic_isolated_script_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/build_dynamic_isolated_script_test_compile_target_overriden.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/build_dynamic_swarmed_isolated_script_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/build_dynamic_swarmed_isolated_script_test_compile_target_overidden.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/buildnumber_zero.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/chromium_webkit_crash.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/chromium_webkit_parent_revision_chromium.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/chromium_webkit_parent_revision_webkit.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/chromium_webkit_revision_chromium.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/chromium_webkit_revision_webkit.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/chromium_webkit_warnings.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_gtest.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_gtest_android.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_gtest_asan_android.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_gtest_memory_asan_no_lsan.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_gtest_memory_builder.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_gtest_memory_mac64.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_gtest_on_builder.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_gtest_win.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_instrumentation_nodefault_build.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_instrumentation_nodefault_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_instrumentation_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_instrumentation_test_with_timeout_scale.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_isolated_script_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_isolated_script_test_harness_failure_zero_retcode.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_junit_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_script_test_failure.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_script_test_with_args.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_failed_isolated_script_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_gn_instrumentation_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_gtest.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_gtest_mac_gpu.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_gtest_override_compile_targets.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_instrumentation_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_isolated_script_test_linux_gpu.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_isolated_script_test_mac_gpu.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_isolated_script_test_win_gpu.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_isolated_script_test_win_non_gpu.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_passed_isolated_script_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_passed_isolated_script_test_with_swarming_failure.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_passed_with_bad_retcode_isolated_script_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_serialized_gtests.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_failed_isolated_script_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_isolated_script_test_harness_failure.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_isolated_script_test_missing_shard.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_passed_isolated_script_test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/ensure_goma_fail.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_bot_update_always_on_coverage_clobber.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_Android.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_Linux_x64.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_Mac.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_Win.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_Win_x64.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Android_MIPS_Builder__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Android_N5X_Swarm_Builder.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Android_Swarm_Builder.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Android_WebView_CTS_L_MR1__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Android_Webview_L__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Android_Webview_M__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Android_arm64_Builder__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Android_arm_Builder__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Android_x64_Builder__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Android_x86_Builder__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Jelly_Bean_Tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_KitKat_Tablet_Tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Lollipop_Consumer_Tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Lollipop_Low_end_Tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Lollipop_Phone_Tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Lollipop_Tablet_Tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Marshmallow_64_bit_Tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_android_Marshmallow_Tablet_Tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chrome_Google_Chrome_ChromeOS.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chrome_Google_Chrome_Linux_x64.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chrome_Google_Chrome_Mac.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chrome_Google_Chrome_Win.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chromiumos_ChromiumOS_amd64_generic_Compile.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chromiumos_ChromiumOS_daisy_Compile.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chromiumos_ChromiumOS_x86_generic_Compile.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chromiumos_Linux_ChromiumOS_Builder.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chromiumos_Linux_ChromiumOS_Builder__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chromiumos_Linux_ChromiumOS_Full.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chromiumos_Linux_ChromiumOS_GN.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chromiumos_Linux_ChromiumOS_GN__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chromiumos_Linux_ChromiumOS_Ozone_Builder.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chromiumos_Linux_ChromiumOS_Ozone_Tests__1_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chromiumos_Linux_ChromiumOS_Tests__1_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_chromiumos_Linux_ChromiumOS_Tests__dbg__1_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Android_Asan_Builder_Tests__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Android_Builder__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Android_Cloud_Tests.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Android_Coverage__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Android_Find_Annotated_Test.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Android_Remoting_Tests.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Android_Tests__trial__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Browser_Side_Navigation_Linux.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_CFI_Linux.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_CFI_Linux_CF.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_CFI_Linux_Full.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_CFI_Linux_ToT.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Chromium_Linux32_Goma_Canary__clobber_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Chromium_Linux_Goma_Canary.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Chromium_Linux_Goma_Canary__clobber_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Chromium_Linux_Precise_Goma_LinkTest.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Chromium_Mac_10_10_MacViews.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Chromium_Mac_10_11.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Chromium_Mac_10_11_Force_Mac_Toolchain.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Chromium_Mac_10_9_Goma_Canary.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Chromium_Mac_10_9_Goma_Canary__clobber_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Chromium_Mac_10_9_Goma_Canary__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Chromium_Mac_10_9_Goma_Canary__dbg__clobber_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTAndroidASan.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTLinux.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTLinuxASan.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTLinuxASan_tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTLinuxLLD.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTLinuxLLD_tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTLinuxUBSanVptr.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTLinuxUBSanVptr_tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTLinux__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTLinux_tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTMac.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTMacASan.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTMacASan_tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTMac__dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTMac_tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTWin.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTWin64.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTWin64_dbg_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTWin64_dbg__tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTWin64_dll_.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTWin64_dll__tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromium_fyi_ClangToTWin64_tester.json
[modify] https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172/scripts/slave/recipes/chromium.expected/full_chromi
So, let's test Angle CQ again: https://chromium-review.googlesource.com/c/379036/
Project Member

Comment 20 by bugdroid1@chromium.org, Sep 1 2016

Project Member

Comment 21 by bugdroid1@chromium.org, Sep 1 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/150aa20aeb0c5a74893d5941faff40f32948de12

commit 150aa20aeb0c5a74893d5941faff40f32948de12
Author: recipe-roller <recipe-roller@chromium.org>
Date: Thu Sep 01 13:24:25 2016

Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

build:
  https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172 Temporary fix for Angle trybots. (tandrii@chromium.org)

TBR=martiniss@chromium.org,phajdan.jr@chromium.org
BUG= 642914 

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Review-Url: https://codereview.chromium.org/2303723002

[modify] https://crrev.com/150aa20aeb0c5a74893d5941faff40f32948de12/infra/config/recipes.cfg

Status: Fixed (was: Started)
I've verified that Angle is fixed now. I'm very sorry for the same breakage again, Angle.


Below is IMO on on how bot_update should handle patches:

 1. checkout HEAD (or fixed rev) of main repo (e.g., chromium/src/)
 2. if patch is for main repo, apply it now  (as it may change the DEPS).
    For Rietveld, apply just DEPS file if any. For gerrit apply the whole thing.
 3. gclient sync 
 4. if patch for main repo:
      if rietveld: apply it except DEPS file applied at step 2.
      if Gerrit, do nothing, already applied.    
 5. if patch for not main repo:
    apply the patch, regardless of source.


The reason this breakage of Angle happened, is because Angle is not main repo (chromium/src is) and step 5 was skipped. As a result, patch was never applied.

The reason for prior breakage of Angle in ~May was because we added step #6 gclient sync which was supposed to fix stale dependencies *after* patch is applied. Gclient sync then wiped out applied patch at step 5, resulting in the same brekage.

Thanks everyone for the fast fix! That reduced the impact of the breakage to two external Europe-based contributors and one CL from the Montreal team.
Project Member

Comment 24 by bugdroid1@chromium.org, Sep 1 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8be150286de87333d2a467823ff52f7111eb1700

commit 8be150286de87333d2a467823ff52f7111eb1700
Author: recipe-roller <recipe-roller@chromium.org>
Date: Thu Sep 01 13:46:13 2016

Roll recipe dependencies (nontrivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

Please review the expectation changes, and LGTM as normal. The recipe roller
will *NOT* CQ the change itself, so you must commit the change manually.

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

build:
  https://crrev.com/39d64ecde5e09e2a7494fe957261e25d0d8df172 Temporary fix for Angle trybots. (tandrii@chromium.org)

R=tandrii@chromium.org
BUG= 642914 

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Review-Url: https://codereview.chromium.org/2307503003
Cr-Commit-Position: refs/heads/master@{#415943}

[modify] https://crrev.com/8be150286de87333d2a467823ff52f7111eb1700/infra/config/recipes.cfg
[modify] https://crrev.com/8be150286de87333d2a467823ff52f7111eb1700/infra/recipes/chromium.expected/builder.json
[modify] https://crrev.com/8be150286de87333d2a467823ff52f7111eb1700/infra/recipes/chromium.expected/tester.json
[modify] https://crrev.com/8be150286de87333d2a467823ff52f7111eb1700/infra/recipes/chromium_trybot.expected/basic.json

Andrii, in response to comment #22, how do you handle the case when ANGLE (Gerrit patch, not in main repo) has a DEPS change that gets pulled in via recursedeps?
AFAIU ATM, it is not handled at all, regardless of patch source because there is no DEPS sync (step 6).

That's what we tried to add in May, though for a wrong reason. If there is a way to tell gclient to not reset the patch, I'd add this step 6. But I don't know such way yet.
Should I file a feature request for that? It's important enough that I've been holding back 6 patches for several months.
Actually, please do file it. TBH, I didn't fully comprehend what you've been asking before by email, because I had too little understanding of how bot_update works.

Potential (as in I never tried it before) workaround for 6 patches you have is this:
1. Upload your Angle's DEPS patch to Gerrit, get it's ref and hash.
2. Create a new chromium patch which modifies chromium DEPS for Angle to use ref from #1.
3. Send chromium patch to whatever try builders you want manually (git cl try ...).
4. Maybe you get lucky.
Thanks Andrii. I had considered that. It won't be possible for future work, so I hadn't tried it yet. Filed  issue 643346 .
Project Member

Comment 30 by bugdroid1@chromium.org, Sep 6 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/6fe7737bed37cafbbb191e82e436b19724c66919

commit 6fe7737bed37cafbbb191e82e436b19724c66919
Author: recipe-roller <recipe-roller@chromium.org>
Date: Tue Sep 06 22:19:31 2016

Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

build:
  https://crrev.com/11d4488d8f3be4eca83d96d4ca931a557869b649 Revert of Temporary fix for Angle trybots. (tandrii@chromium.org)
  https://crrev.com/6821d6b5eb52ee5a1342493984dcef70c5bdeddf Make libyuv expectations more realistic. (iannucci@chromium.org)
  https://crrev.com/b5d7ed0c25c9bbc455522b4f8dd7ae5b58e8e310 Add the apk name to resource_sizes test output (agrieve@chromium.org)
  https://crrev.com/20f5588c5142a40bed4a404457c261f20f62c07c Make ios/try recipe expectations more accurate (iannucci@chromium.org)

TBR=martiniss@chromium.org,phajdan.jr@chromium.org
BUG=643885, 642914 ,609365

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Review-Url: https://codereview.chromium.org/2314273002

[modify] https://crrev.com/6fe7737bed37cafbbb191e82e436b19724c66919/infra/config/recipes.cfg

Project Member

Comment 31 by bugdroid1@chromium.org, Sep 7 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6ea486f948454b99d1f9b8671bcced49cdb79017

commit 6ea486f948454b99d1f9b8671bcced49cdb79017
Author: recipe-roller <recipe-roller@chromium.org>
Date: Wed Sep 07 00:17:16 2016

Roll recipe dependencies (nontrivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

Please review the expectation changes, and LGTM as normal. The recipe roller
will *NOT* CQ the change itself, so you must commit the change manually.

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

build:
  https://crrev.com/11d4488d8f3be4eca83d96d4ca931a557869b649 Revert of Temporary fix for Angle trybots. (tandrii@chromium.org)

R=tandrii@chromium.org
BUG= 642914 

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Review-Url: https://codereview.chromium.org/2317463004
Cr-Commit-Position: refs/heads/master@{#416790}

[modify] https://crrev.com/6ea486f948454b99d1f9b8671bcced49cdb79017/infra/config/recipes.cfg
[modify] https://crrev.com/6ea486f948454b99d1f9b8671bcced49cdb79017/infra/recipes/chromium.expected/builder.json
[modify] https://crrev.com/6ea486f948454b99d1f9b8671bcced49cdb79017/infra/recipes/chromium.expected/tester.json
[modify] https://crrev.com/6ea486f948454b99d1f9b8671bcced49cdb79017/infra/recipes/chromium_trybot.expected/basic.json

Project Member

Comment 32 by bugdroid1@chromium.org, Sep 8 2016

Sign in to add a comment