Currently, the HWID database is manually created and modified during the build. The procedure is annoying and error-prone. we need a tool to automatically generate and update the HWID database by the probed result. The design doc is here: https://docs.google.com/document/d/1OYevIFwQGj8WnxAg8XIZpZ5djXcuLoDHJsfKz0pOEmM/edit#
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/0074ad67f748e03e0966f8e1b3d5ff974aba3866 commit 0074ad67f748e03e0966f8e1b3d5ff974aba3866 Author: Chih-Yu Huang <akahuang@google.com> Date: Thu Sep 22 09:55:37 2016 yaml_utils: Parse mapping to OrderedDict. While we load a yaml file to a object, modify the object, and dump to a yaml file, we hope to keep the order of the mapping instance. Therefore, we should parse the mapping to the Python OrderedDict object, and dump the OrderedDict instance to yaml like a dict object. BUG= chromium:649246 TEST=py/utils/yaml_utils_unittest.py Change-Id: Id5da27765bdc9a9b804b1171e268c2983c7170b4 Reviewed-on: https://chromium-review.googlesource.com/388250 Commit-Ready: Chih-Yu Huang <akahuang@chromium.org> Tested-by: Chih-Yu Huang <akahuang@chromium.org> Reviewed-by: Hung-Te Lin <hungte@chromium.org> [modify] https://crrev.com/0074ad67f748e03e0966f8e1b3d5ff974aba3866/py/utils/yaml_utils_unittest.py [modify] https://crrev.com/0074ad67f748e03e0966f8e1b3d5ff974aba3866/py/utils/yaml_utils.py
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/0c44fd9b300796853be17dd718675f88e88f196b commit 0c44fd9b300796853be17dd718675f88e88f196b Author: Chih-Yu Huang <akahuang@google.com> Date: Wed Sep 21 10:02:31 2016 Implement the yaml representer of the region field and component. In this CL, we can dump the RegionField and RegionComponent to yaml. Note that we don't support the legacy style of RegionField. Because of the restriction of PyYaml, there is a dummy string after the RegionComponent. But it does not affect the result of yaml loading. That is, when we load a RegionComponent, dump it back to yaml string, and then load it again. The Python instance is the same. BUG= chromium:649246 TEST=py/hwid/v3/yaml_tags_unittest.py Change-Id: I127dbba632b33f012453e6fd028c37fe83e5ce8e Reviewed-on: https://chromium-review.googlesource.com/388371 Commit-Ready: Chih-Yu Huang <akahuang@chromium.org> Tested-by: Chih-Yu Huang <akahuang@chromium.org> Reviewed-by: Hung-Te Lin <hungte@chromium.org> [modify] https://crrev.com/0c44fd9b300796853be17dd718675f88e88f196b/py/hwid/v3/yaml_tags.py [modify] https://crrev.com/0c44fd9b300796853be17dd718675f88e88f196b/py/hwid/v3/yaml_tags_unittest.py
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/bf8c4c6ebdaf1c0176b5832f42fac010832a72cb commit bf8c4c6ebdaf1c0176b5832f42fac010832a72cb Author: Chih-Yu Huang <akahuang@google.com> Date: Fri Sep 30 08:09:34 2016 probe.py: add firmware_key field in probed result. Originally, we put ro_main_firmware, ro_ec_firmware, ro_pd_firmware, key_root, key_recovery, and hash_gbb in the 'firmware' field at HWID database. Now we decide to split key_root and key_recovery out and store them to a new component 'firmware_key'. We still keep the original probed result for compatibility. BUG= chromium:649246 TEST=database_unittest.py Change-Id: I80bca7a123c000834b5751d71474d67439ddf98b Reviewed-on: https://chromium-review.googlesource.com/393506 Commit-Ready: Chih-Yu Huang <akahuang@chromium.org> Tested-by: Chih-Yu Huang <akahuang@chromium.org> Reviewed-by: Chih-Yu Huang <akahuang@chromium.org> [modify] https://crrev.com/bf8c4c6ebdaf1c0176b5832f42fac010832a72cb/py/hwid/v3/testdata/test_probe_result.yaml [modify] https://crrev.com/bf8c4c6ebdaf1c0176b5832f42fac010832a72cb/py/gooftool/probe.py [modify] https://crrev.com/bf8c4c6ebdaf1c0176b5832f42fac010832a72cb/py/hwid/v3/testdata/test_probe_result_hwid_utils.yaml [modify] https://crrev.com/bf8c4c6ebdaf1c0176b5832f42fac010832a72cb/py/hwid/v3/database_unittest.py [add] https://crrev.com/bf8c4c6ebdaf1c0176b5832f42fac010832a72cb/py/hwid/v3/testdata/test_new_db.yaml
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/09c3befcb61b303626398b601158dc1e4c8b7875 commit 09c3befcb61b303626398b601158dc1e4c8b7875 Author: Chih-Yu Huang <akahuang@google.com> Date: Wed Nov 02 03:51:08 2016 hwid: add helper methods of RegionField Yaml tags. The RegionField has legacy style and new list style. Originally we can only dump the new list style. This CL implements the functionality of dumping the legacy style of RegionField. We also implemented AddRegion and GetRegions method. BUG= chromium:649246 TEST=py/hwid/v3/yaml_tags_unittest.py Change-Id: Id92b146c14efc8e639dc5eb560c4b0db72e6c195 Reviewed-on: https://chromium-review.googlesource.com/406813 Commit-Ready: Chih-Yu Huang <akahuang@chromium.org> Tested-by: Chih-Yu Huang <akahuang@chromium.org> Reviewed-by: Wei-Han Chen <stimim@chromium.org> [modify] https://crrev.com/09c3befcb61b303626398b601158dc1e4c8b7875/py/hwid/v3/yaml_tags.py [modify] https://crrev.com/09c3befcb61b303626398b601158dc1e4c8b7875/py/hwid/v3/yaml_tags_unittest.py
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/3a957d58d7ed03da2db12865c956e831df7ce4c8 commit 3a957d58d7ed03da2db12865c956e831df7ce4c8 Author: Chih-Yu Huang <akahuang@google.com> Date: Wed Nov 02 08:28:35 2016 hwid: implement HWID database builder. Currently, the HWID database is manually created and modified during the build. The procedure is tedious and error-prone. We implement a tool to build and update the HWID database by the probed result. BUG= chromium:649246 TEST=py/hwid/v3/builder_unittest.py Change-Id: I0ef78b1cb8d40aaf23dec5ec51a0b1f2ef074f34 Reviewed-on: https://chromium-review.googlesource.com/406814 Commit-Ready: Chih-Yu Huang <akahuang@chromium.org> Tested-by: Chih-Yu Huang <akahuang@chromium.org> Reviewed-by: Wei-Han Chen <stimim@chromium.org> [add] https://crrev.com/3a957d58d7ed03da2db12865c956e831df7ce4c8/py/hwid/v3/builder_unittest.py [add] https://crrev.com/3a957d58d7ed03da2db12865c956e831df7ce4c8/py/hwid/v3/testdata/test_db_builder.yaml [add] https://crrev.com/3a957d58d7ed03da2db12865c956e831df7ce4c8/py/hwid/v3/builder.py
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/bd6270ef62297e0d2fbbf7c9f3707e8a2216aa79 commit bd6270ef62297e0d2fbbf7c9f3707e8a2216aa79 Author: Chih-Yu Huang <akahuang@google.com> Date: Wed Nov 02 08:33:00 2016 hwid: Add the builder to the HWID cmdline. BUG= chromium:649246 TEST=hwid_utils_unittest.py Change-Id: I7eb4e4ebf8b08918e6f7cadf7617b86020af4440 Reviewed-on: https://chromium-review.googlesource.com/406815 Commit-Ready: Chih-Yu Huang <akahuang@chromium.org> Tested-by: Chih-Yu Huang <akahuang@chromium.org> Reviewed-by: Wei-Han Chen <stimim@chromium.org> [modify] https://crrev.com/bd6270ef62297e0d2fbbf7c9f3707e8a2216aa79/py/hwid/v3/builder_unittest.py [modify] https://crrev.com/bd6270ef62297e0d2fbbf7c9f3707e8a2216aa79/py/utils/yaml_utils_unittest.py [modify] https://crrev.com/bd6270ef62297e0d2fbbf7c9f3707e8a2216aa79/py/hwid/v3/database.py [modify] https://crrev.com/bd6270ef62297e0d2fbbf7c9f3707e8a2216aa79/py/utils/yaml_utils.py [modify] https://crrev.com/bd6270ef62297e0d2fbbf7c9f3707e8a2216aa79/py/hwid/v3/hwid_utils.py [add] https://crrev.com/bd6270ef62297e0d2fbbf7c9f3707e8a2216aa79/py/hwid/v3/testdata/test_builder_probe_results.yaml [modify] https://crrev.com/bd6270ef62297e0d2fbbf7c9f3707e8a2216aa79/py/hwid/v3/hwid_cmdline.py [modify] https://crrev.com/bd6270ef62297e0d2fbbf7c9f3707e8a2216aa79/py/hwid/v3/hwid_utils_unittest.py
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/8e0bc06bbbb5fd2d7326608ec3d61a49e58f89d6 commit 8e0bc06bbbb5fd2d7326608ec3d61a49e58f89d6 Author: Chih-Yu Huang <akahuang@google.com> Date: Wed Nov 09 03:57:02 2016 hwid: Add README.md In README.md, we define the standard way to update HWID v3 database, and introduce the usage of the HWID database builder. BUG= chromium:649246 TEST=none Change-Id: I2dbe400d43d1a51a72b609b553e318eb43a5e552 Reviewed-on: https://chromium-review.googlesource.com/409890 Commit-Ready: Chih-Yu Huang <akahuang@chromium.org> Tested-by: Chih-Yu Huang <akahuang@chromium.org> Reviewed-by: Joel Kitching <kitching@chromium.org> [add] https://crrev.com/8e0bc06bbbb5fd2d7326608ec3d61a49e58f89d6/py/hwid/README.md
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/factory/+/ff19d2535602e61873c7aef5a875f8f5a39c213e commit ff19d2535602e61873c7aef5a875f8f5a39c213e Author: Chih-Yu Huang <akahuang@google.com> Date: Wed Feb 08 03:38:04 2017 CHERRY-PICK: hwid: Cherry-pick HWID database builder from ToT. In order to let SOC/ODM partner use the HWID database builder directly, we cherry-pick all change in py/hwid/v3 and README.md from ToT. The latest commit of ToT: "hwid: add use case in README.md." Hash: e5f32e3b2c9175285b61827dbeb8b48a4f68504f Also merged "probe.py: add firmware_key field in probed result." Hash: bf8c4c6ebdaf1c0176b5832f42fac010832a72cb BUG= chromium:649246 TEST=unittest in py/hwid/v3 Change-Id: Ic86ce830f6bd721540e7422a9f5a1a8254a7a11f Reviewed-on: https://chromium-review.googlesource.com/439104 Reviewed-by: Chih-Yu Huang <akahuang@chromium.org> Commit-Queue: Chih-Yu Huang <akahuang@chromium.org> Tested-by: Chih-Yu Huang <akahuang@chromium.org> [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/hwid_rule_functions.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/yaml_tags_unittest.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/yaml_tags.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/testdata/test_db.yaml [add] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/testdata/new_test_probe_result_hwid_utils.yaml [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/decoder.py [add] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/testdata/test_builder_probe_results.yaml [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/testdata/test_probe_result_hwid_utils.yaml [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/hwid_cmdline.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/hwid_utils_unittest.py [add] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/testdata/test_new_db.yaml [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/testdata/test_db_no_checksum.yaml [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/common_unittest.py [add] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/README.md [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/encoder_unittest.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/database.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/encoder.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/testdata/test_db_wrong_checksum_field.yaml [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/common.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/verify_db_pattern.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/rule.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/valid_hwid_db_unittest.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/testdata/TEST_BOARD [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/testdata/test_probe_result.yaml [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/database_unittest.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/hwid_utils.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/gooftool/probe.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/utils/yaml_utils.py [add] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/builder_unittest.py [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/decoder_unittest.py [add] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/testdata/NEW_TEST_BOARD [add] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/testdata/test_db_builder.yaml [modify] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/hwid_rule_functions_unittest.py [add] https://crrev.com/ff19d2535602e61873c7aef5a875f8f5a39c213e/py/hwid/v3/builder.py
The fundamental feature of the HWID database builder is finished, also added README.md in comment 9. Close this issue.
Comment 1 by bugdroid1@chromium.org
, Sep 22 2016