Premise: Recipe modules are reusable modules of functionality. Currently their test coverage is a combination of example.py (which SHOULD be, you know, an example), plus whatever coverage they get from the recipes in the same repo that happen to use them.
Proposal: Modules should gain two features:
* *_test.py
* expectation_tests
*_test.py are normal python unittest-runnable tests. Technically these can exist right now, but the recipe engine should grow functionality to find/list/execute them.
expectation_tests is a recipes/-style root directory which contains one or more recipes. The intent of these recipes is to exercise the module code in order to ensure that it's well-defined and works correctly.
At some point later, I would also propose that we can add a boolean to some of the expectation_tests to have them become runnable "for real" on a representative set of platforms (e.g. mac, linux, windows, etc.). The expectation_test recipes could then contain internal asserts for testing conditions that they want to ensure.
Comment 1 by iannucci@chromium.org
, May 15 2017