New issue
Advanced search Search tips

Issue 788104 link

Starred by 1 user
Status: Verified
Owner:
machenb...@chromium.org
Closed: Nov 2017
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Feature

Blocking:
issue 788113



Sign in to add a comment

V8: Need a way for developers to specify tests source-side

Project Member Reported by machenb...@chromium.org, Nov 23 2017 
A number of developers change features behind runtime flags. We can test those as specific test variants on the bots, but currently it's hard to set up new steps in the infrastructure.

We should introduce a v8-side specification file, similar to chromium's /testing/buildbot configurations, which will be dynamically read by specific bots, which in turn decide which tests to run.

The v8-side specification should define:
Mapping from master name to builder name. For each required builder, listing the tests, shards and variant name.

The bots will read the specification and on a match pass it from builder to tester as properties.

Comment 1 by machenb...@chromium.org, Nov 23 2017

Blocking: 788113
Project Member

Comment 2 by bugdroid1@chromium.org, Nov 26 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/072292157232334e1f530df3be8f7d26fa5b8c4a

commit 072292157232334e1f530df3be8f7d26fa5b8c4a
Author: Michael Achenbach <machenbach@chromium.org>
Date: Sun Nov 26 13:57:35 2017

V8: Refactoring - extract some interating code

Prepares for code reuse in https://crrev.com/c/789073

This doesn't change behavior.

Bug:  788104 
Change-Id: I77455d512352ec34a60b94845722a816f741cb51
Reviewed-on: https://chromium-review.googlesource.com/789570
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>

[modify] https://crrev.com/072292157232334e1f530df3be8f7d26fa5b8c4a/scripts/slave/README.recipes.md
[modify] https://crrev.com/072292157232334e1f530df3be8f7d26fa5b8c4a/scripts/slave/recipe_modules/v8/builders.py
[modify] https://crrev.com/072292157232334e1f530df3be8f7d26fa5b8c4a/scripts/slave/recipe_modules/v8/api.py
Project Member

Comment 3 by bugdroid1@chromium.org, Nov 27 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/6ccd362b1f4a2871c228d2c9dd3defa8120c3727

commit 6ccd362b1f4a2871c228d2c9dd3defa8120c3727
Author: Michael Achenbach <machenbach@chromium.org>
Date: Mon Nov 27 09:21:06 2017

V8: Deprecate exhaustive-variants flag

Passing --exhaustive-testing and --variants=more,dev is equivalent.
This uses the latter to make the api more uniform and eventually
deprecate the redundant flag.

Prepares for refactoring in https://crrev.com/c/789870

Bug:  788104 
Change-Id: I978b3d71b69caa4bda0a33e1e703d0311fdf0dcf
Reviewed-on: https://chromium-review.googlesource.com/789831
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_Linux___beta_branch___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___internal_snapshot_test_failures_wrong_results.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64_TSAN___concurrent_marking.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_nodcheck_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_ports_V8_Linux___arm___sim___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_ppc___sim___beta_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_rel_ng_triggered_triggered_by_cq.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___custom_snapshot___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___internal_snapshot_empty_json.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_win64_dbg.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_mipsel___sim___stable_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_win_asan_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Mac___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux64_asan_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux64_ubsan_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Win64___msvc.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___shared.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Mac_GC_Stress.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_mips64el___sim___beta_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_ports_V8_Linux___arm64___sim___gc_stress.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_s390___sim___beta_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___predictable_bisect_one_change.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_mac_gc_stress_dbg.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___internal_snapshot_test_failures_flakes.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_ports_V8_Linux___ppc64___sim.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux64_tsan_rel.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64_GC_Stress___custom_snapshot.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Mac64_ASAN.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64_ASAN.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_arm_armv8a_rel_positional_extra_flags.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_s390x___sim___stable_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___verify_csa.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_arm___sim___beta_branch___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Win64___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux64_sanitizer_coverage_rel.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Mac.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_mac64_dbg.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_ppc64___sim___stable_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_Linux___stable_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_rel_ng_triggered_failures.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_dbg_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___custom_snapshot___debug_slim_bisect_tester_swarming.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___predictable_bisect_recurring_failure.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_win_dbg.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_arm_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64_TSAN.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_ppc64___sim___beta_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Win32___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_gc_stress_dbg.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Win64.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_ports_V8_Linux___arm64___sim.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___gyp.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Win32.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Mac64___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Win32_ASAN.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_arm_armv8a_dbg.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_mac64_asan_rel.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___verify_csa.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipe_modules/v8/testing.py
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_Linux64___stable_branch___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_ppc___sim___stable_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_rel_ng_triggered_slow_tests.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_rel_ng_triggered_triggered_by_ts.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_s390x___sim___beta_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_win_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_verify_csa_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_ports_V8_Arm.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_arm___sim___beta_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_ports_V8_Linux___ppc___sim.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_Linux___beta_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_mipsel___sim___beta_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_rel_ng_triggered_flakes.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_dynamorio_linux_v8_dr.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_ports_V8_Linux___s390___sim.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___internal_snapshot_one_failure.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___interpreted_regexp.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___shared_bisect_swarming.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_s390___sim___stable_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___gcov_coverage.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_win64_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___predictable_bisect_tests_too_long.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_ports_V8_Linux___arm64___sim___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___predictable.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___gc_stress.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_arm64_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_mac_dbg.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_mac_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_ports_V8_Linux___s390x___sim.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_rel_ng_triggered_test_filter.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_gcc_rel.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux64_gyp_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___predictable_bisect.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_arm_armv8a_rel.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_arm___sim___stable_branch___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___predictable_bisect_override_changes.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_arm64_dbg.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux64_tsan_concurrent_marking_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_mips64el___sim___stable_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___cfi.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_mac64_rel.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux64_verify_csa_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___arm64___sim___MSAN.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_Linux___stable_branch___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_Linux64___stable_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_arm___sim___stable_branch.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___internal_snapshot.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_ports_V8_Linux___arm___sim.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux_gcc_4_8.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64_UBSanVptr.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_Linux64___beta_branch___debug.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64_one_failure_build_env_not_supported.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Mac64.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux_swarming_collect_failure.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux64_msan_rel.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux64_sanitizer_coverage_rel_gerrit.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_arm_dbg.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_arm64_gc_stress_dbg.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___internal_snapshot_test_failures.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64_bisect_tester_swarming.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux64_rel_ng_triggered.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_ports_V8_Linux___mipsel___sim.json
[modify] https://crrev.com/6ccd362b1f4a2871c228d2c9dd3defa8120c3727/scripts/slave/recipes/v8.expected/full_client_v8_branches_V8_Linux64___beta_branch.json
Project Member

Comment 4 by bugdroid1@chromium.org, Nov 27 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/030dd8cb61f202cb0366e698070475ca58d5b1af

commit 030dd8cb61f202cb0366e698070475ca58d5b1af
Author: Michael Achenbach <machenbach@chromium.org>
Date: Mon Nov 27 12:22:26 2017

V8: Simplify handling testing variants

This was overdesigned and allowed for much more flexibility than
needed. The simplification prepares for https://crrev.com/c/789073
and will make it easier to serialize variants and reason about
step names of steps running specific variants.

Bug:  788104 
Change-Id: I8bed72e16ba286beff4f339250c5bdd778bd9bf2
Reviewed-on: https://chromium-review.googlesource.com/789870
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>

[modify] https://crrev.com/030dd8cb61f202cb0366e698070475ca58d5b1af/scripts/slave/README.recipes.md
[modify] https://crrev.com/030dd8cb61f202cb0366e698070475ca58d5b1af/scripts/slave/recipe_modules/v8/builders.py
[modify] https://crrev.com/030dd8cb61f202cb0366e698070475ca58d5b1af/scripts/slave/recipe_modules/v8/api.py
[modify] https://crrev.com/030dd8cb61f202cb0366e698070475ca58d5b1af/scripts/slave/recipe_modules/v8/testing.py
Project Member

Comment 5 by bugdroid1@chromium.org, Nov 27 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/a2aff67f69b8b76eba33c8347dcff9c85ad9d754

commit a2aff67f69b8b76eba33c8347dcff9c85ad9d754
Author: Michael Achenbach <machenbach@chromium.org>
Date: Mon Nov 27 13:10:26 2017

[test] Simplify flags for configuring testing variants.

This deprecates --exhaustive-variants and --no-variants flags to
simplify code configuring variants.

Simplification after:
https://crrev.com/c/789831

Bug:  chromium:788104 
Change-Id: Ie77d48eca083e7721e02d34bc1e9aa1b1f0d0202
Reviewed-on: https://chromium-review.googlesource.com/789836
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49633}
[modify] https://crrev.com/a2aff67f69b8b76eba33c8347dcff9c85ad9d754/tools/gcov.sh
[modify] https://crrev.com/a2aff67f69b8b76eba33c8347dcff9c85ad9d754/tools/testrunner/standard_runner.py
Project Member

Comment 6 by bugdroid1@chromium.org, Nov 28 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/10133e606b58a6008fb7ad96f39b883b00f6f29a

commit 10133e606b58a6008fb7ad96f39b883b00f6f29a
Author: Michael Achenbach <machenbach@chromium.org>
Date: Tue Nov 28 13:51:36 2017

V8: Read v8-side test spec

This enables reading a V8-side pyl specification located at:
v8/infra/testing/<mastername>.pyl.

The expected format is a mapping from buildername to list of tests.
Each test is expected to be a dict with name, shards and variant.

This will allow to specify extra test for certain bots source side,
resulting in faster set-up with no infra involvement. An additional
benefit is that config changes will go through V8 CQ.

The test spec is also passed as property from builder to triggered
testers. We use the key "parent_test_spec" as it's ignored by CQ
by default.

Bug:  788104 
Change-Id: I113f113a64f152793811a2104fe75af1b8392726
Reviewed-on: https://chromium-review.googlesource.com/789073
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>

[modify] https://crrev.com/10133e606b58a6008fb7ad96f39b883b00f6f29a/scripts/slave/recipe_modules/v8/api.py
[add] https://crrev.com/10133e606b58a6008fb7ad96f39b883b00f6f29a/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___nosnap_builder_with_test_spec.json
[modify] https://crrev.com/10133e606b58a6008fb7ad96f39b883b00f6f29a/scripts/slave/recipe_modules/v8/test_api.py
[modify] https://crrev.com/10133e606b58a6008fb7ad96f39b883b00f6f29a/scripts/slave/README.recipes.md
[modify] https://crrev.com/10133e606b58a6008fb7ad96f39b883b00f6f29a/scripts/slave/recipe_modules/v8/builders.py
[modify] https://crrev.com/10133e606b58a6008fb7ad96f39b883b00f6f29a/scripts/slave/recipe_modules/v8/testing.py
[modify] https://crrev.com/10133e606b58a6008fb7ad96f39b883b00f6f29a/scripts/slave/recipes/v8.py
[add] https://crrev.com/10133e606b58a6008fb7ad96f39b883b00f6f29a/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___nosnap_with_test_spec.json
[add] https://crrev.com/10133e606b58a6008fb7ad96f39b883b00f6f29a/scripts/slave/recipes/v8.expected/full_client_v8_V8_Mac64_with_test_spec.json
Project Member

Comment 7 by bugdroid1@chromium.org, Nov 28 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/3747356b44b7b27abeee1a21a756595f3cdc9357

commit 3747356b44b7b27abeee1a21a756595f3cdc9357
Author: Michael Achenbach <machenbach@chromium.org>
Date: Tue Nov 28 17:21:36 2017

[test] Add v8-side demo test configuration

Bug:  chromium:788104 
Change-Id: I1d778292548642661c81ec62bc1f23a0ea4c0762
Reviewed-on: https://chromium-review.googlesource.com/793530
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49684}
[add] https://crrev.com/3747356b44b7b27abeee1a21a756595f3cdc9357/infra/testing/OWNERS
[add] https://crrev.com/3747356b44b7b27abeee1a21a756595f3cdc9357/infra/testing/README.md
[add] https://crrev.com/3747356b44b7b27abeee1a21a756595f3cdc9357/infra/testing/client.v8.pyl
[add] https://crrev.com/3747356b44b7b27abeee1a21a756595f3cdc9357/infra/testing/tryserver.v8.pyl
Project Member

Comment 8 by bugdroid1@chromium.org, Nov 29 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/23ccfa1d9487317dd68500a8ed81f399e72ac49c

commit 23ccfa1d9487317dd68500a8ed81f399e72ac49c
Author: Michael Achenbach <machenbach@chromium.org>
Date: Wed Nov 29 13:18:22 2017

[test] Turn exemplary configurations into reference comments

This also updates the README with guidelines.

Bug:  chromium:788104 
Change-Id: I0ca0ea78c5990204b0242be9c7fe6368439a5dd1
Reviewed-on: https://chromium-review.googlesource.com/796311
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49713}
[modify] https://crrev.com/23ccfa1d9487317dd68500a8ed81f399e72ac49c/infra/testing/README.md
[modify] https://crrev.com/23ccfa1d9487317dd68500a8ed81f399e72ac49c/infra/testing/client.v8.pyl
[modify] https://crrev.com/23ccfa1d9487317dd68500a8ed81f399e72ac49c/infra/testing/tryserver.v8.pyl

Comment 9 by machenb...@chromium.org, Nov 29 2017

Status: Verified (was: Assigned)
Project Member

Comment 10 by bugdroid1@chromium.org, Nov 30 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/df659b17edecebf6f5b5b96edf28655e5ee612b5

commit df659b17edecebf6f5b5b96edf28655e5ee612b5
Author: Michael Achenbach <machenbach@chromium.org>
Date: Thu Nov 30 10:41:57 2017

[test] Refactor variant flags

This derives two global variables to ease adding new variants.

Bug:  chromium:788104 
Change-Id: Ibe933dc0999f6dfa0ae8bf2afa0a5de8e44aefca
Reviewed-on: https://chromium-review.googlesource.com/799790
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49741}
[modify] https://crrev.com/df659b17edecebf6f5b5b96edf28655e5ee612b5/tools/testrunner/local/variants.py
Project Member

Comment 11 by bugdroid1@chromium.org, Dec 5 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/d533fe84b8e8439793ae490b7cf2c2b140eb131e

commit d533fe84b8e8439793ae490b7cf2c2b140eb131e
Author: Michael Achenbach <machenbach@chromium.org>
Date: Tue Dec 05 12:47:23 2017

V8: Add generic linux64 FYI bots for src-side test specifications

Developers will be able to dynamically add FYI tests to linux64
release/debug bots and corresponding optional trybot.

Bug:  788104 
Change-Id: Iaf65628202e09b6d6dbc50f1dfd595f93b772c12
Reviewed-on: https://chromium-review.googlesource.com/807989
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>

[modify] https://crrev.com/d533fe84b8e8439793ae490b7cf2c2b140eb131e/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___debug_builder.json
[modify] https://crrev.com/d533fe84b8e8439793ae490b7cf2c2b140eb131e/masters/master.client.v8/slaves.cfg
[add] https://crrev.com/d533fe84b8e8439793ae490b7cf2c2b140eb131e/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux64_fyi_rel_ng.json
[modify] https://crrev.com/d533fe84b8e8439793ae490b7cf2c2b140eb131e/scripts/slave/v8/gatekeeper_v8_tree_closers.json
[add] https://crrev.com/d533fe84b8e8439793ae490b7cf2c2b140eb131e/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___fyi.json
[modify] https://crrev.com/d533fe84b8e8439793ae490b7cf2c2b140eb131e/masters/master.client.v8/master.cfg
[modify] https://crrev.com/d533fe84b8e8439793ae490b7cf2c2b140eb131e/scripts/slave/recipe_modules/v8/builders.py
[modify] https://crrev.com/d533fe84b8e8439793ae490b7cf2c2b140eb131e/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___builder.json
[add] https://crrev.com/d533fe84b8e8439793ae490b7cf2c2b140eb131e/scripts/slave/recipes/v8.expected/full_tryserver_v8_v8_linux64_fyi_rel_ng_triggered.json
[modify] https://crrev.com/d533fe84b8e8439793ae490b7cf2c2b140eb131e/masters/master.tryserver.v8/master.cfg
[add] https://crrev.com/d533fe84b8e8439793ae490b7cf2c2b140eb131e/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___debug___fyi.json
[modify] https://crrev.com/d533fe84b8e8439793ae490b7cf2c2b140eb131e/masters/master.tryserver.v8/slaves.cfg
Project Member

Comment 12 by bugdroid1@chromium.org, Dec 5 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/19d040eaa77cb68cc56cb7ea1022e45fcaf68e40

commit 19d040eaa77cb68cc56cb7ea1022e45fcaf68e40
Author: Michael Achenbach <machenbach@chromium.org>
Date: Tue Dec 05 13:41:33 2017

[test] Add more documentation about test-specifications

NOTRY=true
NOPRESUBMIT=true
NOTREECHECKS=true

Bug:  chromium:788104 
Change-Id: I1918d4ba69b2c00151ac51ee9426d980e24ab871
Reviewed-on: https://chromium-review.googlesource.com/808389
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49863}
[modify] https://crrev.com/19d040eaa77cb68cc56cb7ea1022e45fcaf68e40/infra/testing/README.md
Project Member

Comment 13 by bugdroid1@chromium.org, Dec 21 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/77f96a5d2e177631f0bc5e8a78147cffd25b5791

commit 77f96a5d2e177631f0bc5e8a78147cffd25b5791
Author: Michael Achenbach <machenbach@chromium.org>
Date: Thu Dec 21 14:48:48 2017

[build] Fix missing fyi bot config

NOTRY=true
TBR=sergiyb@chromium.org

Bug:  chromium:788104 
Change-Id: I3f47332a0f0df7a5a016afcb7d0904142176bea3
Reviewed-on: https://chromium-review.googlesource.com/839775
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50271}
[modify] https://crrev.com/77f96a5d2e177631f0bc5e8a78147cffd25b5791/infra/mb/mb_config.pyl
►

Sign in to add a comment