Two examples: https://findit-for-me.appspot.com/waterfall/check-flake?master_name=chromium.win&builder_name=Win7%20(32)%20Tests&build_number=13257&step_name=browser_tests%20on%20Windows-7-SP1&test_name=CrExtensionsBrowserTest.ExtensionCodeSectionLayoutTest https://findit-for-me.appspot.com/waterfall/check-flake?master_name=chromium.mac&builder_name=Mac10.10%20Tests&build_number=9919&step_name=browser_tests%20on%20Mac-10.10&test_name=CrExtensionsBrowserTestWithIdQueryParam.ExtensionManagerNavigationToDetailsTest The culprit is r434935 (https://bugs.chromium.org/p/chromium/issues/detail?id=669435), but the rerun at the builds including the culprit had almost 100% or 99% pass rate.
We need to add feature to automatically decide iterations to run, rather than keep using 100 times.
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/619a8a9c3f1d1fd4b9f13dea1096e353a8304b4e commit 619a8a9c3f1d1fd4b9f13dea1096e353a8304b4e Author: Chan <chanli@chromium.org> Date: Wed Feb 15 05:19:32 2017 [Findit] Flake Analyzer: add config for automatically determining iterations. In preparation of automatically determining iterations for a flake analysis, add max_iterations_to_rerun. BUG= 669612 Change-Id: I6795168f4fb137a2b53dc338a4bbdab26cc8dd06 Reviewed-on: https://chromium-review.googlesource.com/442671 Reviewed-by: Jeffrey Li <lijeffrey@chromium.org> Commit-Queue: Jeffrey Li <lijeffrey@chromium.org> [modify] https://crrev.com/619a8a9c3f1d1fd4b9f13dea1096e353a8304b4e/appengine/findit/waterfall/test/waterfall_config_test.py [modify] https://crrev.com/619a8a9c3f1d1fd4b9f13dea1096e353a8304b4e/appengine/findit/waterfall/test/wf_testcase.py [modify] https://crrev.com/619a8a9c3f1d1fd4b9f13dea1096e353a8304b4e/appengine/findit/handlers/test/config_test.py [modify] https://crrev.com/619a8a9c3f1d1fd4b9f13dea1096e353a8304b4e/appengine/findit/model/wf_config.py [modify] https://crrev.com/619a8a9c3f1d1fd4b9f13dea1096e353a8304b4e/appengine/findit/handlers/config.py
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/5d06952ff2933024581c1b7a14ec0aa56cf5975b commit 5d06952ff2933024581c1b7a14ec0aa56cf5975b Author: Chan <chanli@chromium.org> Date: Tue Feb 21 18:35:20 2017 [Findit] Flake Analyzer: Automatically determine iterations to run for a flake analysis. For the first build, if the swarming rerun returns stable result, rerun the same build using a higher number of iterations. If all swarming reruns(with different number of iterations) for the first build all stable, Bail out. BUG= 669612 , 681714 Change-Id: I3fe1abeae43778c9bcab4da8a86ffeb6f36fd87d Reviewed-on: https://chromium-review.googlesource.com/444823 Commit-Queue: Chan Li <chanli@chromium.org> Reviewed-by: Roberto Carrillo <robertocn@chromium.org> Reviewed-by: Jeffrey Li <lijeffrey@chromium.org> [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/waterfall/test/waterfall_config_test.py [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/waterfall/trigger_base_swarming_task_pipeline.py [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/waterfall/flake/test/lookback_algorithm_test.py [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/waterfall/flake/recursive_flake_pipeline.py [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/waterfall/flake/lookback_algorithm.py [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/model/flake/test/flake_swarming_task_test.py [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/waterfall/flake/test/recursive_flake_pipeline_test.py [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/model/flake/master_flake_analysis.py [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/waterfall/flake/recursive_flake_try_job_pipeline.py [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/model/flake/flake_swarming_task.py [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/waterfall/flake/schedule_flake_try_job_pipeline.py [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/waterfall/test/trigger_flake_swarming_task_pipeline_test.py [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/waterfall/flake/test/schedule_flake_try_job_pipeline_test.py [modify] https://crrev.com/5d06952ff2933024581c1b7a14ec0aa56cf5975b/appengine/findit/waterfall/flake/test/recursive_flake_try_job_pipeline_test.py
Comment 1 by chanli@chromium.org
, Jan 17 2017Owner: chanli@chromium.org
Status: Assigned (was: Available)