factory: test list: revise generic test lists |
|||
Issue descriptionpihsun has recently migrated platform/factory/py/test/test_lists into generic JSON test list, and we'd like to revise this: - Merge with recent projects like poppy - Make shopfloor Start and End calls matched in same group (station). - Clean up GRT. For example, ClearTPMOwnerRequest should be moved to gooftool, and no reboot is needed. GRTFinish and Barrier may be merged. Decide what we want to do for GRTShopfloor, since we should do Start(GRT) and End(GRT) instead of only End. Otherwise, we should use NotifyEvent instead of NotifyStart/End.
,
Aug 14 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/1a6cb49ee94f8f1e1e9ed96f9865dd250b1176d3 commit 1a6cb49ee94f8f1e1e9ed96f9865dd250b1176d3 Author: Hung-Te Lin <hungte@chromium.org> Date: Mon Aug 14 11:01:58 2017 test_lists: Change 'disable_cros_shortcut_keys' to default True. The 'disable_cros_shortcut_keys' needs X and has been "does not work" since Chrome OS has changed to Freon for a long time. Maybe we don't really need this option. Before removing it, set the default value to what most test lists want (True) to eliminate the declaration in generic test lists. BUG= chromium:754647 TEST=make test Change-Id: I815c11d7515f58210e50187eb9cd50d21f90450f Reviewed-on: https://chromium-review.googlesource.com/611696 Commit-Ready: Hung-Te Lin <hungte@chromium.org> Tested-by: Hung-Te Lin <hungte@chromium.org> Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org> [modify] https://crrev.com/1a6cb49ee94f8f1e1e9ed96f9865dd250b1176d3/py/test/test_lists/generic_rrt.test_list.json [modify] https://crrev.com/1a6cb49ee94f8f1e1e9ed96f9865dd250b1176d3/py/test/test_lists/manager_unittest/override_args.test_list.json [modify] https://crrev.com/1a6cb49ee94f8f1e1e9ed96f9865dd250b1176d3/py/test/factory.py [modify] https://crrev.com/1a6cb49ee94f8f1e1e9ed96f9865dd250b1176d3/py/test/test_lists/manager_unittest.py [modify] https://crrev.com/1a6cb49ee94f8f1e1e9ed96f9865dd250b1176d3/py/test/test_lists/manager_unittest/a.test_list.json [modify] https://crrev.com/1a6cb49ee94f8f1e1e9ed96f9865dd250b1176d3/py/test/test_lists/generic_main.test_list.json
,
Aug 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/8787438621a0efc9b91eee8a7d34bb4a41cf804f commit 8787438621a0efc9b91eee8a7d34bb4a41cf804f Author: Hung-Te Lin <hungte@chromium.org> Date: Wed Aug 16 09:05:58 2017 test_lists: Deprecate Probe* tests by new Probe. The Probe* tests should be deprecated by the single Probe test, which takes a JSON config to define what must be probed and how. This also voids adding extra local or constants for specifying if we should probe some thing in particular stage (for example SMT). Since SMT (SA Test) usually has different set of components from FA tests, the generic test list is using a different config 'probe_smt.json' for SMT station while the default config is 'probe.json'. BUG= chromium:754647 TEST=make test Change-Id: I71fda7d89301bb5bb6670dd5e90fc11aa403d9db Reviewed-on: https://chromium-review.googlesource.com/611697 Commit-Ready: Hung-Te Lin <hungte@chromium.org> Tested-by: Hung-Te Lin <hungte@chromium.org> Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org> [add] https://crrev.com/8787438621a0efc9b91eee8a7d34bb4a41cf804f/py/test/pytests/probe/probe_smt.json [modify] https://crrev.com/8787438621a0efc9b91eee8a7d34bb4a41cf804f/py/test/test_lists/generic_common.test_list.json [add] https://crrev.com/8787438621a0efc9b91eee8a7d34bb4a41cf804f/py/test/pytests/probe/probe.json [modify] https://crrev.com/8787438621a0efc9b91eee8a7d34bb4a41cf804f/py/test/test_lists/generic_smt.test_list.json [modify] https://crrev.com/8787438621a0efc9b91eee8a7d34bb4a41cf804f/py/test/test_lists/generic_fft.test_list.json
,
Aug 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/47d5f6210a2ee4074fa42ac3de3317eac3e0a8df commit 47d5f6210a2ee4074fa42ac3de3317eac3e0a8df Author: Hung-Te Lin <hungte@chromium.org> Date: Wed Aug 16 11:03:22 2017 test_lists: Revise generic test lists. Changed all group tests to always be %{STATION}Start %{STATION}Items CheckPoint %{STATION}End So it's easier for projects to override how they want to construct the tests. Also changed all tests defined in particular station (for example, "Keyboard" in SMT) to have name prefixed by station name (i.e, "SMTKeyboard"). Legacy names "*Finish" are also reserved so test list in board overlay won't break immediately (but they may have duplicated entries after this merged and should be fixed). BUG= chromium:754647 TEST=make test Change-Id: I7fff7c9101fcab9018fe6f639b19f6789811f8c6 Reviewed-on: https://chromium-review.googlesource.com/611698 Commit-Ready: Hung-Te Lin <hungte@chromium.org> Tested-by: Hung-Te Lin <hungte@chromium.org> Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org> [modify] https://crrev.com/47d5f6210a2ee4074fa42ac3de3317eac3e0a8df/py/test/test_lists/generic_common.test_list.json [modify] https://crrev.com/47d5f6210a2ee4074fa42ac3de3317eac3e0a8df/py/test/test_lists/generic_run_in.test_list.json [modify] https://crrev.com/47d5f6210a2ee4074fa42ac3de3317eac3e0a8df/po/zh-CN.po [modify] https://crrev.com/47d5f6210a2ee4074fa42ac3de3317eac3e0a8df/py/test/pytests/finalize/finalize.py [modify] https://crrev.com/47d5f6210a2ee4074fa42ac3de3317eac3e0a8df/py/test/test_lists/generic_fat.test_list.json [modify] https://crrev.com/47d5f6210a2ee4074fa42ac3de3317eac3e0a8df/py/test/test_lists/generic_fft.test_list.json [modify] https://crrev.com/47d5f6210a2ee4074fa42ac3de3317eac3e0a8df/py/test/test_lists/generic_smt.test_list.json [modify] https://crrev.com/47d5f6210a2ee4074fa42ac3de3317eac3e0a8df/py/test/test_lists/generic_main.test_list.json [modify] https://crrev.com/47d5f6210a2ee4074fa42ac3de3317eac3e0a8df/py/test/test_lists/generic_grt.test_list.json
,
Aug 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/a4d08c901a7e99e3739dbdab6a857c2666b47456 commit a4d08c901a7e99e3739dbdab6a857c2666b47456 Author: Hung-Te Lin <hungte@chromium.org> Date: Wed Aug 16 11:03:23 2017 pytests: Add 'include_parents' to 'summary' test. The new 'include_parents' allow test 'summary' to check "all previous tests", not just inside same group", to have been passed. This is similar and more flexible than the prefligh check in 'finalize' test. A new 'AllCheckPoint' test has been defined that will run summary with include_parents=True. BUG= chromium:754647 , chromium:754190 TEST=make test Change-Id: I23e2b1097c25b1ef10b6ce7f7473c13924f7b9b4 Reviewed-on: https://chromium-review.googlesource.com/615065 Commit-Ready: Hung-Te Lin <hungte@chromium.org> Tested-by: Hung-Te Lin <hungte@chromium.org> Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org> [modify] https://crrev.com/a4d08c901a7e99e3739dbdab6a857c2666b47456/po/zh-CN.po [modify] https://crrev.com/a4d08c901a7e99e3739dbdab6a857c2666b47456/py/test/test_lists/generic_common.test_list.json [modify] https://crrev.com/a4d08c901a7e99e3739dbdab6a857c2666b47456/py/test/pytests/summary/summary.py [modify] https://crrev.com/a4d08c901a7e99e3739dbdab6a857c2666b47456/py/test/test_lists/generic_grt.test_list.json
,
Aug 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/91dfa9e35ebefec92ec14e900bf2e87417a1f8cc commit 91dfa9e35ebefec92ec14e900bf2e87417a1f8cc Author: Hung-Te Lin <hungte@chromium.org> Date: Wed Aug 16 11:03:23 2017 pytests: Add 'prompt_message' argument to test 'summary'. The summary can be made as "Barrier", and even better if we can change the prompt message so it can replace the "Barrier(summary) + Message" test group as a single "Check Point (summary)" test item, by setting `pass_without_prompt` to False and `prompt_message` to the message that we want to display. Also revised test doc string to show how summary can be used. BUG= chromium:754647 TEST=make test Change-Id: Ic20d25983c4f6ebfdd3ef336786f3996fac52c82 Reviewed-on: https://chromium-review.googlesource.com/615226 Commit-Ready: Hung-Te Lin <hungte@chromium.org> Tested-by: Hung-Te Lin <hungte@chromium.org> Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org> Reviewed-by: Wei-Han Chen <stimim@chromium.org> [modify] https://crrev.com/91dfa9e35ebefec92ec14e900bf2e87417a1f8cc/py/test/pytests/summary/summary.py
,
Aug 17 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/5384f0e55df962de711aa477e0d8cf281f98e7ad commit 5384f0e55df962de711aa477e0d8cf281f98e7ad Author: Wei-Han Chen <stimim@google.com> Date: Thu Aug 17 06:37:53 2017 pytests: summary: make status table scrollable If the pytest is showing too many test statuses, the status table will be too long, and cannot fit into the screen. Make the table scrollable in this CL. BUG= chromium:754647 , chromium:754190 TEST=manual Change-Id: I578990e464d4d5b09abe2fa122e4ff4dd67e21b6 Reviewed-on: https://chromium-review.googlesource.com/616407 Commit-Ready: Wei-Han Chen <stimim@chromium.org> Tested-by: Wei-Han Chen <stimim@chromium.org> Reviewed-by: Hung-Te Lin <hungte@chromium.org> [modify] https://crrev.com/5384f0e55df962de711aa477e0d8cf281f98e7ad/py/test/pytests/summary/summary.py
,
Aug 23 2017
,
Aug 23 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/da8a3fa4e66f0a86d1d01d38fdaefeea95f6c932 commit da8a3fa4e66f0a86d1d01d38fdaefeea95f6c932 Author: Hung-Te Lin <hungte@chromium.org> Date: Wed Aug 23 13:23:29 2017 pytests: sync_factory_server: Allow setting new server URL In order to support "different factory server URL per station", we want to allow changing the default server URL tracked in shopfloor.py module; and sync_factory_server.py should be the right place. This change also revised sync_factory_server so it has better user interface, structure and layout. BUG= chromium:754647 TEST=make test; manually tested. Change-Id: Icfe4ae595ae88610256800c06c1af69b907164f3 Reviewed-on: https://chromium-review.googlesource.com/616587 Commit-Ready: Hung-Te Lin <hungte@chromium.org> Tested-by: Hung-Te Lin <hungte@chromium.org> Reviewed-by: Hung-Te Lin <hungte@chromium.org> [modify] https://crrev.com/da8a3fa4e66f0a86d1d01d38fdaefeea95f6c932/po/zh-CN.po [modify] https://crrev.com/da8a3fa4e66f0a86d1d01d38fdaefeea95f6c932/py/test/pytests/sync_factory_server.py
,
Aug 24 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/overlays/overlay-coral-private/+/4c9fc912f851ded47ddad179139de96678f8ca5c commit 4c9fc912f851ded47ddad179139de96678f8ca5c Author: Hung-Te Lin <hungte@chromium.org> Date: Thu Aug 24 02:04:07 2017
,
Aug 24 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/6bf63d9eb382c04f35e200e9eb3b1e46d29d1be4 commit 6bf63d9eb382c04f35e200e9eb3b1e46d29d1be4 Author: Hung-Te Lin <hungte@chromium.org> Date: Thu Aug 24 02:04:07 2017 test_lists: Add a 'FATP' flatten group for better hierarchy. FAT, RunIn, FFT and GRT are all sub items of FATP. We find that there may be common settings that will be shared by all FATP stations (for example, projects using different shopfloor server IP for SMT and FATP) so it seems reasonable to create a FATP group for overriding local variables for it. BUG= chromium:754647 TEST=make test Change-Id: I0ff05c778ea96d9f66d0ab491f1d78e0123eb7f6 Reviewed-on: https://chromium-review.googlesource.com/627947 Commit-Ready: Hung-Te Lin <hungte@chromium.org> Tested-by: Hung-Te Lin <hungte@chromium.org> Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org> [modify] https://crrev.com/6bf63d9eb382c04f35e200e9eb3b1e46d29d1be4/py/test/test_lists/generic_main.test_list.json
,
Aug 24 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/134403cc38dfe21ab4b379eeab4cff9a11f47978 commit 134403cc38dfe21ab4b379eeab4cff9a11f47978 Author: Hung-Te Lin <hungte@chromium.org> Date: Thu Aug 24 02:04:15 2017 test_lists: Remove deprecated arguments. There are few arguments that have no implementation in public repo (which implies it won't work for JSON test lists), or has been deprecated for a while. - ui_lang: Deprecated by ui_locale. - shopfloor_server_url_for_stage: Should be implemented by overriding shopfloor_server_url in locals. - check_if_mlb_changed: Should be implemented by init/goofy.d/device/device_id hooks. BUG= chromium:754647 TEST=make test Change-Id: I86438d8c415417a8f5899b8de717464eddc68dfb Reviewed-on: https://chromium-review.googlesource.com/628055 Commit-Ready: Hung-Te Lin <hungte@chromium.org> Tested-by: Hung-Te Lin <hungte@chromium.org> Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org> [modify] https://crrev.com/134403cc38dfe21ab4b379eeab4cff9a11f47978/py/test/factory.py [modify] https://crrev.com/134403cc38dfe21ab4b379eeab4cff9a11f47978/py/goofy/goofy.py
,
Aug 24 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/overlays/overlay-coral-private/+/6ce02f6380bb76026707fa106b765689748f8e88 commit 6ce02f6380bb76026707fa106b765689748f8e88 Author: Hung-Te Lin <hungte@chromium.org> Date: Thu Aug 24 08:25:52 2017
,
Aug 25 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/1c2e803bba30ce3fc93e1d6d1264b63cb19fd0af commit 1c2e803bba30ce3fc93e1d6d1264b63cb19fd0af Author: Hung-Te Lin <hungte@chromium.org> Date: Fri Aug 25 14:49:19 2017 test_lists: Easier to re-test GRT tests. It is very often that projects may fail to finish Finalize test and need to re-run several times, and current GRT is a huge AutomatedSequence group so it takes a long time to retry, also reproducing lots of useless logs. In order to simplify the flow, we should change GRT from AutomatedSequence to TestGroup. However, Finalize should still be protected by CheckPoint, and we should set CheckPoint to action_on_failure=STOP so even if the operator clicked "Mark Failed" in engineering mode, Goofy won't start Finalization automatically. BUG= chromium:754647 , chromium:754190 TEST=manually run GRT group. Change-Id: I8da67e47fea3c3d9cb1666b98aafc2e78efc7073 Reviewed-on: https://chromium-review.googlesource.com/634790 Commit-Ready: Hung-Te Lin <hungte@chromium.org> Tested-by: Hung-Te Lin <hungte@chromium.org> Reviewed-by: Hung-Te Lin <hungte@chromium.org> [modify] https://crrev.com/1c2e803bba30ce3fc93e1d6d1264b63cb19fd0af/py/test/test_lists/generic_common.test_list.json [modify] https://crrev.com/1c2e803bba30ce3fc93e1d6d1264b63cb19fd0af/py/test/test_lists/generic_grt.test_list.json
,
Aug 28 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/631861d6810b54189cb68096797e3cf15dff6fa2 commit 631861d6810b54189cb68096797e3cf15dff6fa2 Author: Hung-Te Lin <hungte@chromium.org> Date: Mon Aug 28 11:27:09 2017 pytests: Merge flush_testlog to sync_factory_server. The flush_testlog is actually doing same process as sync_factory_server tasks so we may merge the two tests. This also helps us to first ping server when diagnosing flush_testlog errors. A new test list definition "FlushTestlog" is also added to explicitly flush test logs without syncing other resources like toolkit or event log (nor uploading reports). BUG= chromium:754647 TEST=make test; manually set flush_testlog=True and run sync_factory_server. Change-Id: Iaad5b84891bee78be50c707bef170bb2930d5d9e Reviewed-on: https://chromium-review.googlesource.com/634764 Commit-Ready: Hung-Te Lin <hungte@chromium.org> Tested-by: Hung-Te Lin <hungte@chromium.org> Reviewed-by: Hung-Te Lin <hungte@chromium.org> [modify] https://crrev.com/631861d6810b54189cb68096797e3cf15dff6fa2/po/zh-CN.po [modify] https://crrev.com/631861d6810b54189cb68096797e3cf15dff6fa2/py/test/pytests/sync_factory_server.py [modify] https://crrev.com/631861d6810b54189cb68096797e3cf15dff6fa2/py/test/test_lists/generic_common.test_list.json [delete] https://crrev.com/305ba42475af2c20b76c617270e16a6604812d30/py/test/pytests/flush_testlog.py
,
Aug 29 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/overlays/overlay-poppy-private/+/2b89af6202f12e3e42ac97fa237c38acf5029dae commit 2b89af6202f12e3e42ac97fa237c38acf5029dae Author: Yong Hong <yhong@google.com> Date: Tue Aug 29 14:13:37 2017
,
Aug 29 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/overlays/overlay-poppy-private/+/d57e0e765ef1f8ce8d66c4adb847185d1a220164 commit d57e0e765ef1f8ce8d66c4adb847185d1a220164 Author: Yong Hong <yhong@google.com> Date: Tue Aug 29 14:13:37 2017
,
Sep 21 2017
I think we're in good state. |
|||
►
Sign in to add a comment |
|||
Comment 1 by hungte@chromium.org
, Aug 11 2017