factory: Refactor generic and board test lists. |
|||||
Issue description
Currently the generic_* test list in public overlay is useless.
And the test list in board overlay is almost taking nothing from public overlay - we are seeing code duplication and copied between overlays.
I wonder if we can minimize what's available in board overlay, especially test list.
The ultimate goal is to switch to JSON test list, but there's some common things we can do so far, for example:
1. Change "test create functions" into JSON version. Currently we have lots of "helper functions" in main.py, which converts the construction of individual tests into a simpler function call, and easier to be shared between different stages (SMT, FATP). With the help of JSON, I wonder if we can do something like this:
In a common.test_list.json, define most "well-known test items", for example
"definitions": {
"FlushTestLog": {
"pytest_name": "flush_testlog",
"label": "i18n!Flush Testlog"
},
"Start": {
"pytest_name": "start",
"label": "i18n!Start",
"never_fails": true,
"args": {
"press_to_continue": true,
"require_external_power": false,
"check_factory_install_complete": false
}
}
}
And in common.py, create helper function
def CreateTest(name, **kargs):
test_args = ExtractTestArgs(kargs)
CreateJsonTest(name, **test_args, dargs=**kargs)
And in python test lists, we can remove the helper functions and simply do
common.CreateTest('KeyboardProbe', never_fails=True)
common.CreateTest('LidSwitch', some_dargs='test')
common.CreateTest('Keyboard')
common.CreateTest('HWbutton')
Meanwhile, we can remove all the 'id' at the same time.
This may be a huge change but I think we should take the chance of poppy development to finish the first migration. It will also help JSON test list since in the future JSON test lists can also use the common.test_list.json.
Pihsun, can you think about this? And you can get help from others if needed since the migration may be huge.
,
Jul 6 2017
After some more testing, it seems somewhat hard to use definitions in JSON test list for normal test list. The main issue is that "eval! " won't work since "eval! " arguments are not evaluated on LegacyTestList. Stimim, do you have idea on how to solve this?
,
Jul 6 2017
Goofy will resolve pytest args by test list manager when CL:505860 is merged.
,
Jul 18 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/b3b797efecca009da4854cd8a43abdb4c3a0fe19 commit b3b797efecca009da4854cd8a43abdb4c3a0fe19 Author: Peter Shih <pihsun@chromium.org> Date: Tue Jul 18 16:47:25 2017 config_utils: Change inheritance to use C3 linearization Change from simple depth-first search based inheritance to C3 superclass linearization algorithm. The original algorithm is same as old-style class inheritance in Python, and C3 superclass linearization is same as new-style class inheritance in Python. This solves the problem that the following config has final value of b as 0, which is unintuitive and cause problem when used the config for JSON test list: -- base.json -- {"b": 0} -- a.json -- {"inherit": ["base"]} -- b.json -- {"inherit": ["base"], "b": 42} -- final.json -- {"inherit": ["a", "b"]} BUG= chromium:738969 TEST=make test Change-Id: I13619593cdebe45afb72be233c52f92de9c26a66 Reviewed-on: https://chromium-review.googlesource.com/575789 Commit-Ready: Pi-Hsun Shih <pihsun@chromium.org> Tested-by: Pi-Hsun Shih <pihsun@chromium.org> Reviewed-by: Wei-Han Chen <stimim@chromium.org> [add] https://crrev.com/b3b797efecca009da4854cd8a43abdb4c3a0fe19/py/utils/testdata/config_utils_unittest_c3_base.json [add] https://crrev.com/b3b797efecca009da4854cd8a43abdb4c3a0fe19/py/utils/testdata/config_utils_unittest_c3.json [modify] https://crrev.com/b3b797efecca009da4854cd8a43abdb4c3a0fe19/py/utils/config_utils_unittest.py [add] https://crrev.com/b3b797efecca009da4854cd8a43abdb4c3a0fe19/py/utils/testdata/config_utils_unittest_c3_a.json [modify] https://crrev.com/b3b797efecca009da4854cd8a43abdb4c3a0fe19/py/utils/config_utils.py
,
Jul 26 2017
,
Aug 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/eb544cba16513d30502f071972c34e96492b8f86 commit eb544cba16513d30502f071972c34e96492b8f86 Author: Peter Shih <pihsun@chromium.org> Date: Tue Aug 08 13:47:52 2017 test_lists: Refactor test list to use JSON. Refactor generic test list to use JSON test list format, and move most definitions from private overlays to generic test list, so we can share most definitions across projects. Also cleaned up some unused generic_* test lists and unused options in test lists. CQ-DEPEND=CL:597562 BUG= chromium:738969 , chromium:694901 TEST=make test TEST=manually on DUT Change-Id: I9b1bd5c2fa1fab90675c27b97f8e92456b31a4ad Reviewed-on: https://chromium-review.googlesource.com/559028 Commit-Ready: Pi-Hsun Shih <pihsun@chromium.org> Tested-by: Pi-Hsun Shih <pihsun@chromium.org> Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org> [add] https://crrev.com/eb544cba16513d30502f071972c34e96492b8f86/py/test/test_lists/generic_run_in.test_list.json [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/generic_control_run.py [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/firmware_stress.py [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/generic_fatp.py [modify] https://crrev.com/eb544cba16513d30502f071972c34e96492b8f86/py/test/test_lists/manager.py [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/generic_diagnostic.py [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/generic_run_in.py [add] https://crrev.com/eb544cba16513d30502f071972c34e96492b8f86/py/test/test_lists/generic_grt.test_list.json [add] https://crrev.com/eb544cba16513d30502f071972c34e96492b8f86/py/test/test_lists/generic_common.test_list.json [add] https://crrev.com/eb544cba16513d30502f071972c34e96492b8f86/py/test/test_lists/disable_factory_server.test_list.json [add] https://crrev.com/eb544cba16513d30502f071972c34e96492b8f86/py/test/test_lists/generic_fat.test_list.json [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/generic.py [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/generic_experiment.py [add] https://crrev.com/eb544cba16513d30502f071972c34e96492b8f86/py/test/test_lists/generic_smt.test_list.json [add] https://crrev.com/eb544cba16513d30502f071972c34e96492b8f86/py/test/test_lists/generic_rrt.test_list.json [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/generic_tsab.py [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/generic_smt.py [add] https://crrev.com/eb544cba16513d30502f071972c34e96492b8f86/py/test/test_lists/generic_fft.test_list.json [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/e2e_test.py [add] https://crrev.com/eb544cba16513d30502f071972c34e96492b8f86/py/test/test_lists/generic_main.test_list.json [modify] https://crrev.com/eb544cba16513d30502f071972c34e96492b8f86/po/zh-CN.po [modify] https://crrev.com/eb544cba16513d30502f071972c34e96492b8f86/py/test/test_lists/station_based_test_list.py [add] https://crrev.com/eb544cba16513d30502f071972c34e96492b8f86/py/test/test_lists/station_based.test_list.json [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/generic_rrt.py [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/generic_vswr.py [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/firmware_stress_generic.py [delete] https://crrev.com/fa415e6867dbe65bb6b1ecde975f61bf428cceb3/py/test/test_lists/generic_grt.py
,
Aug 9 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/overlays/overlay-poppy-private/+/93d97474714a354b1a47cd4855866074a16ca895 commit 93d97474714a354b1a47cd4855866074a16ca895 Author: Peter Shih <pihsun@chromium.org> Date: Wed Aug 09 06:29:02 2017
,
Aug 9 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/01cc52bbff0993efb0cf8a446f1d70ed3a9d135e commit 01cc52bbff0993efb0cf8a446f1d70ed3a9d135e Author: Peter Shih <pihsun@chromium.org> Date: Wed Aug 09 09:23:18 2017 test_lists: Add gyroscope test items to JSON common. Add gyroscope related pytests to generic_common.test_list.json, so it can be reused by other test lists. BUG= chromium:738969 TEST=make test Change-Id: I6ddef59b16b33d1c77f39155e7ad74b755828afa Reviewed-on: https://chromium-review.googlesource.com/607771 Commit-Ready: Pi-Hsun Shih <pihsun@chromium.org> Tested-by: Pi-Hsun Shih <pihsun@chromium.org> Reviewed-by: Hung-Te Lin <hungte@chromium.org> [modify] https://crrev.com/01cc52bbff0993efb0cf8a446f1d70ed3a9d135e/py/test/test_lists/generic_common.test_list.json
,
Aug 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/0ced8738dd35b59782f0fefd9106d289a366e681 commit 0ced8738dd35b59782f0fefd9106d289a366e681 Author: Peter Shih <pihsun@chromium.org> Date: Thu Aug 10 08:15:00 2017 test_lists: Add stylus test item to JSON common. Add stylus pytests to generic_common.test_list.json, so it can be reused by other test lists. BUG= chromium:738969 TEST=make test Change-Id: Ib4e950fbd9b81a514dd8cd0fa9edcbc38cee4ec2 Reviewed-on: https://chromium-review.googlesource.com/607732 Commit-Ready: Pi-Hsun Shih <pihsun@chromium.org> Tested-by: Pi-Hsun Shih <pihsun@chromium.org> Reviewed-by: Wei-Han Chen <stimim@chromium.org> [modify] https://crrev.com/0ced8738dd35b59782f0fefd9106d289a366e681/py/test/test_lists/generic_common.test_list.json
,
Aug 14 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/overlays/overlay-coral-private/+/719223d1bfec3d03f541f8b73ee8201582491522 commit 719223d1bfec3d03f541f8b73ee8201582491522 Author: Peter Shih <pihsun@chromium.org> Date: Mon Aug 14 11:01:59 2017
,
Aug 17 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/overlays/overlay-coral-private/+/47faa19b5c11537949a856278867df3d79845929 commit 47faa19b5c11537949a856278867df3d79845929 Author: Hung-Te Lin <hungte@chromium.org> Date: Thu Aug 17 02:56:45 2017
,
Aug 17 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/overlays/overlay-poppy-private/+/0f47dd6fc0b6d05944c17111ae57c35b2b941bc4 commit 0f47dd6fc0b6d05944c17111ae57c35b2b941bc4 Author: Hung-Te Lin <hungte@chromium.org> Date: Thu Aug 17 06:37:54 2017
,
Aug 29 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/overlays/overlay-fizz-private/+/e29b76c8288f489a38ce83ff6bb3c19ea6c266c9 commit e29b76c8288f489a38ce83ff6bb3c19ea6c266c9 Author: Hung-Te Lin <hungte@chromium.org> Date: Tue Aug 29 04:15:36 2017
,
Sep 1 2017
,
Sep 1 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/4c5d40773a5a80f8cb17d4bfaa7b6edec1a534ab commit 4c5d40773a5a80f8cb17d4bfaa7b6edec1a534ab Author: Hung-Te Lin <hungte@chromium.org> Date: Fri Sep 01 19:39:20 2017 init: Add scripts from Coral to all projects. There are two scripts we implemented for Coral and found it to be useful for all projects: - select_default_test_list: A script to select default test list if not defined yet. Its current implementation is based on "model" and only works on JSON test lists. - device_id/reset_test_results: Sometimes partner may try to clone (dd, or reading eMMC directly) a disk image from one golden device to another. Also there are devices with internal storage can be removed (for example old SSD via SATA) that partner will run test by connecting newly built motherboard with a "golden disk". The factory should detect both cases and reset test states. BUG= chromium:738969 TEST=None Change-Id: I9baa524a0366d9d4add8d0e9aedce84c587709a2 Reviewed-on: https://chromium-review.googlesource.com/647065 Commit-Ready: Hung-Te Lin <hungte@chromium.org> Tested-by: Hung-Te Lin <hungte@chromium.org> Reviewed-by: Hung-Te Lin <hungte@chromium.org> [add] https://crrev.com/4c5d40773a5a80f8cb17d4bfaa7b6edec1a534ab/init/goofy.d/device/select_default_test_list.sh [add] https://crrev.com/4c5d40773a5a80f8cb17d4bfaa7b6edec1a534ab/init/goofy.d/device/device_id/reset_test_results.sh
,
Jan 22 2018
,
Jan 23 2018
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by hungte@chromium.org
, Jul 4 2017