[Findit] Flake Analyzer - user-specified iterations not being passed correctly |
||
Issue descriptionrecursive_flake_try_job_pipline isn't passing the user's specified iterations correctly to the next call of RecursiveFlakeTryJobPipeline in NextBuildNumberPipeline, causing the expected args to be incorrect
,
Jun 8 2017
After further investigation it seems the above description is an over-simplification: In the unit test itself MonitorTryJobPipeline is being called, however it relies on callbacks which may cause issues in unit tests. Thus the pipelines following MonitorTryJobPipeline never get called, allowing the test to pass incorrectly afterward since the pipelines don't even get run
,
Jun 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/8fd45cbf2b2c4bea1301941669acc232a353dada commit 8fd45cbf2b2c4bea1301941669acc232a353dada Author: Jeffrey Li <lijeffrey@chromium.org> Date: Thu Jun 08 20:50:32 2017 [Findit] Fixing async pipeline unit tests Also fixes issues revealed by fixing unit tests using async pipelines for tests passing for the wrong reasons. Bug: 730900 Change-Id: I08e528151ada505a4403c284384bef6049907e3b Reviewed-on: https://chromium-review.googlesource.com/527710 Commit-Queue: Jeffrey Li <lijeffrey@chromium.org> Reviewed-by: Chan Li <chanli@chromium.org> Reviewed-by: Roberto Carrillo <robertocn@chromium.org> [modify] https://crrev.com/8fd45cbf2b2c4bea1301941669acc232a353dada/appengine/findit/waterfall/flake/test/recursive_flake_pipeline_test.py [modify] https://crrev.com/8fd45cbf2b2c4bea1301941669acc232a353dada/appengine/findit/waterfall/flake/recursive_flake_try_job_pipeline.py [modify] https://crrev.com/8fd45cbf2b2c4bea1301941669acc232a353dada/appengine/findit/gae_libs/testcase.py [modify] https://crrev.com/8fd45cbf2b2c4bea1301941669acc232a353dada/appengine/findit/waterfall/flake/test/recursive_flake_try_job_pipeline_test.py
,
Jun 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/8fd45cbf2b2c4bea1301941669acc232a353dada commit 8fd45cbf2b2c4bea1301941669acc232a353dada Author: Jeffrey Li <lijeffrey@chromium.org> Date: Thu Jun 08 20:50:32 2017 [Findit] Fixing async pipeline unit tests Also fixes issues revealed by fixing unit tests using async pipelines for tests passing for the wrong reasons. Bug: 730900 Change-Id: I08e528151ada505a4403c284384bef6049907e3b Reviewed-on: https://chromium-review.googlesource.com/527710 Commit-Queue: Jeffrey Li <lijeffrey@chromium.org> Reviewed-by: Chan Li <chanli@chromium.org> Reviewed-by: Roberto Carrillo <robertocn@chromium.org> [modify] https://crrev.com/8fd45cbf2b2c4bea1301941669acc232a353dada/appengine/findit/waterfall/flake/test/recursive_flake_pipeline_test.py [modify] https://crrev.com/8fd45cbf2b2c4bea1301941669acc232a353dada/appengine/findit/waterfall/flake/recursive_flake_try_job_pipeline.py [modify] https://crrev.com/8fd45cbf2b2c4bea1301941669acc232a353dada/appengine/findit/gae_libs/testcase.py [modify] https://crrev.com/8fd45cbf2b2c4bea1301941669acc232a353dada/appengine/findit/waterfall/flake/test/recursive_flake_try_job_pipeline_test.py
,
Jun 8 2017
|
||
►
Sign in to add a comment |
||
Comment 1 by lijeffrey@chromium.org
, Jun 8 2017Why this was not caught by unit tests: Unit tests for functions using "with pipeline.InOrder()" seem to ignore the parameters passed to mocked pipelines for testing. For example, if in product code: class TestPipeline(BasePipeline): def run(self, arg0): with pipeline.InOrder(): arg1 = 'blabla' arg2 = 'blabla' yield PipelineA(arg1, arg2) yield PipelineB(arg1, arg2) and in test code: self.MockPipeline( PipelineA, 'thing PipelineA yields, expected_args=['you', 'can', 'put', 'anything', 'you', 'want']) TestPipeline('blabla').run() The test will always pass even with invalid args in the mocked pipeline. Mocking pipeline.InOrder() at the beginning of the test seems to solve this.