New issue
Advanced search Search tips

Issue 645293 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner: ----
Closed: May 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Recipe modules need a stronger testing story

Project Member Reported by iannucci@chromium.org, Sep 8 2016

Issue description

Premise: Recipe modules are reusable modules of functionality. Currently their test coverage is a combination of example.py (which SHOULD be, you know, an example), plus whatever coverage they get from the recipes in the same repo that happen to use them.

Proposal: Modules should gain two features:
  * *_test.py
  * expectation_tests

*_test.py are normal python unittest-runnable tests. Technically these can exist right now, but the recipe engine should grow functionality to find/list/execute them.

expectation_tests is a recipes/-style root directory which contains one or more recipes. The intent of these recipes is to exercise the module code in order to ensure that it's well-defined and works correctly.

At some point later, I would also propose that we can add a boolean to some of the expectation_tests to have them become runnable "for real" on a representative set of platforms (e.g. mac, linux, windows, etc.). The expectation_test recipes could then contain internal asserts for testing conditions that they want to ensure.
 
Status: Fixed (was: Untriaged)
This is more effectively represented in other bugs (i.e.  issue 693058 ,  issue 459361 )

Sign in to add a comment