In order to make DetermineApproximatePassRatePipeline a standalone pipeline independent of flake analysis, subpipelines/services must be decoupled from MasterFlakeAnalysis as well first.
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/b376ae07aa30542f6b77d8251f0b8c51dd33efe5 commit b376ae07aa30542f6b77d8251f0b8c51dd33efe5 Author: Jeffrey Li <lijeffrey@chromium.org> Date: Mon Aug 13 18:27:22 2018 [Findit] Flake Analyzer - Decoupling RunFlakeSwarmingTaskPipeline from MasterFlakeAnalysis This change is purely refactoring only. DetermineApproximatePassRatePipeline is to become a standalone pipeline independent of flake analysis. In order to achieve this, RunFlakeSwarmingTaskPipeline, as a sub-pipeline of DAPR, must also be decoupled in preparation to remove analysis_urlsafe_key from DAPR. No functional change otherwise. Bug: 873381 Change-Id: I1a8fd5606f6b2020bcd47d089c783c4473166345 Reviewed-on: https://chromium-review.googlesource.com/1171828 Commit-Queue: Jeffrey Li <lijeffrey@chromium.org> Reviewed-by: Yuke Liao <liaoyuke@chromium.org> [modify] https://crrev.com/b376ae07aa30542f6b77d8251f0b8c51dd33efe5/appengine/findit/pipelines/flake_failure/run_flake_swarming_task_pipeline.py [modify] https://crrev.com/b376ae07aa30542f6b77d8251f0b8c51dd33efe5/appengine/findit/pipelines/flake_failure/test/determine_approximate_pass_rate_pipeline_test.py [modify] https://crrev.com/b376ae07aa30542f6b77d8251f0b8c51dd33efe5/appengine/findit/services/flake_failure/test/flake_swarming_test.py [modify] https://crrev.com/b376ae07aa30542f6b77d8251f0b8c51dd33efe5/appengine/findit/pipelines/flake_failure/determine_approximate_pass_rate_pipeline.py [modify] https://crrev.com/b376ae07aa30542f6b77d8251f0b8c51dd33efe5/appengine/findit/pipelines/flake_failure/test/run_flake_swarming_task_pipeline_test.py [modify] https://crrev.com/b376ae07aa30542f6b77d8251f0b8c51dd33efe5/appengine/findit/services/flake_failure/flake_swarming.py
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/818769f7f0038af290f792c4cc53693ed2a5580a commit 818769f7f0038af290f792c4cc53693ed2a5580a Author: Jeffrey Li <lijeffrey@chromium.org> Date: Wed Aug 22 22:19:28 2018 [Findit] Flake Analyzer - Making DetermineApproximatePassRatePipeline standalone 1. Introduce Flakiness as a DTO: an in-memory structure for aggregating swarming task data used to estimate flakiness at a commit position. This structure is easily converted to a DataPoint object. 2. Decouple DetermineApproximatePassRatePipeline from flake analysis entirely by accepting the configuration and commit position of a flaky test to run and yields a Flakiness object for the calling pipeline to handle. 3. UpdateFlakeAnalysisDataPoints now simply in charge of converting Flakiness --> DataPoint and appending it to a MasterFlakeAnalysis 4. Move functions out of data_point_util.py and introduce flakiness_util.py instead for operating on Flakiness objects Unfortunately this CL could not be broken down further as many pipelines and services would have broken by making the switch to Flakiness instead of DataPoint and decoupling DetermineApproximatePassRatePipeline from MasterFlakeAnalysis. Bug: 873381 Change-Id: Ifb0f6cfa617fc04fdcf51e1dd0aaeb9e58e85611 Reviewed-on: https://chromium-review.googlesource.com/1177192 Reviewed-by: Shuotao Gao <stgao@chromium.org> Reviewed-by: Yuke Liao <liaoyuke@chromium.org> Commit-Queue: Jeffrey Li <lijeffrey@chromium.org> [add] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/services/flake_failure/test/flakiness_util_test.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/services/flake_failure/test/run_swarming_util_test.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/pipelines/flake_failure/update_flake_analysis_data_points_pipeline.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/services/flake_failure/pass_rate_util.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/services/flake_failure/run_swarming_util.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/pipelines/flake_failure/test/update_flake_analysis_data_points_pipeline_test.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/pipelines/flake_failure/test/create_bug_for_flake_pipeline_test.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/services/flake_failure/test/data_point_util_test.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/pipelines/flake_failure/analyze_flake_pipeline.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/pipelines/flake_failure/determine_approximate_pass_rate_pipeline.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/pipelines/flake_failure/test/analyze_flake_pipeline_test.py [add] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/dto/flakiness.py [add] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/services/flake_failure/flakiness_util.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/services/flake_failure/flake_constants.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/pipelines/flake_failure/create_bug_for_flake_pipeline.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/services/flake_failure/data_point_util.py [modify] https://crrev.com/818769f7f0038af290f792c4cc53693ed2a5580a/appengine/findit/pipelines/flake_failure/test/determine_approximate_pass_rate_pipeline_test.py
Comment 1 by bugdroid1@chromium.org
, Aug 13