Scenario:
We remove a component in the latest image_id. When we decode a HWID with latest image_id, we will generate a BOM object which the index of the component is set as 0.

We propose to fix the bug by this doc:
https://docs.google.com/document/d/1uNcH1OGuyjGOGk2CkN_bk7VVh39oSwWFXe564M3qJl4/edit#

- HWID creates the binary string lazily
- BOM only keeps the known component

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/9a5b05b228ddac55c8c37f3473110e8209c33c2f

commit 9a5b05b228ddac55c8c37f3473110e8209c33c2f
Author: Chih-Yu Huang <akahuang@google.com>
Date: Thu Oct 06 03:28:06 2016

hwid: remove binary_string and encoded_string in HWID class.

Originally, HWID has the database, BOM, binary string, and encoded
string. But the binary string actually depends on the database and
the BOM only, and the encoded string is just another representation
of the binary string. So now we only keep the database and the BOM
object in HWID, and calculate the binary string while needed. We can
also ignore the verification between BOM and the binary string.

If the HWID is created from the encoded string, we still store the
original one, and return it if the BOM is not changed.

BUG= chromium:653369 
TEST=unittest in py/hwid/v3

Change-Id: Ic9f7d203c07ebb481c412303ce273fa5c4afbb07
Reviewed-on: https://chromium-review.googlesource.com/396638
Commit-Ready: Chih-Yu Huang <akahuang@chromium.org>
Tested-by: Chih-Yu Huang <akahuang@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/9a5b05b228ddac55c8c37f3473110e8209c33c2f/py/hwid/v3/decoder.py
[modify] https://crrev.com/9a5b05b228ddac55c8c37f3473110e8209c33c2f/py/hwid/v3/common_unittest.py
[modify] https://crrev.com/9a5b05b228ddac55c8c37f3473110e8209c33c2f/py/hwid/v3/hwid_rule_functions.py
[modify] https://crrev.com/9a5b05b228ddac55c8c37f3473110e8209c33c2f/py/hwid/v3/common.py
[modify] https://crrev.com/9a5b05b228ddac55c8c37f3473110e8209c33c2f/py/hwid/v3/encoder.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/04b30225b5fd50de3574a5fe766c09802a89a5d5

commit 04b30225b5fd50de3574a5fe766c09802a89a5d5
Author: Chih-Yu Huang <akahuang@google.com>
Date: Tue Oct 11 08:46:22 2016

hwid: add Pattern.GetFieldNames method.

This method returns the set of the encoded field names in the pattern.

BUG= chromium:653369 
TEST=py/hwid/v3/database_unittest.py

Change-Id: I01bfc30bf28f30d1c4e3820f9ffaf53bbc64f264
Reviewed-on: https://chromium-review.googlesource.com/397198
Commit-Ready: Chih-Yu Huang <akahuang@chromium.org>
Tested-by: Chih-Yu Huang <akahuang@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/04b30225b5fd50de3574a5fe766c09802a89a5d5/py/hwid/v3/database.py
[modify] https://crrev.com/04b30225b5fd50de3574a5fe766c09802a89a5d5/py/hwid/v3/database_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/fc6e1a725014ae06fdcc0b98921a7bef09bed97b

commit fc6e1a725014ae06fdcc0b98921a7bef09bed97b
Author: Chih-Yu Huang <akahuang@google.com>
Date: Wed Oct 12 08:06:00 2016

hwid: refactor the database.py

This CL only refactor the code, doesn't change the logic.

BUG= chromium:653369 
TEST=database_unittest.py

Change-Id: Iba34bca232b5f9f09dd5aab3d4592e9046b951fe
Reviewed-on: https://chromium-review.googlesource.com/397199
Commit-Ready: Chih-Yu Huang <akahuang@chromium.org>
Tested-by: Chih-Yu Huang <akahuang@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/fc6e1a725014ae06fdcc0b98921a7bef09bed97b/py/hwid/v3/database.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/7b85e14da0a21a5db7b93231c04794af1f75d672

commit 7b85e14da0a21a5db7b93231c04794af1f75d672
Author: Chih-Yu Huang <akahuang@google.com>
Date: Wed Oct 12 07:59:01 2016

hwid: Change the logic of transfer probe result to BOM.

For the unprobeable components class, usually we would set the value
while evaluating the rules. But if there is only one item in the
component class, there might be no rule to set the value because
originally the index of the encoded field is set to 0 as default.

Now we don't set the default index to 0. To handle this special
situation, if an unprobeable component class only has one item, we
pretend we found this item.

BUG= chromium:653369 
TEST=unittest in py/hwid/v3

Change-Id: I0bbc2e1787a44c163851797ed0872bbcc1d3f5c1
Reviewed-on: https://chromium-review.googlesource.com/397200
Commit-Ready: Chih-Yu Huang <akahuang@chromium.org>
Tested-by: Chih-Yu Huang <akahuang@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/7b85e14da0a21a5db7b93231c04794af1f75d672/py/hwid/v3/database.py
[modify] https://crrev.com/7b85e14da0a21a5db7b93231c04794af1f75d672/py/hwid/v3/database_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/e4b0934892ccefedac0e42cb92c973266cd2bc22

commit e4b0934892ccefedac0e42cb92c973266cd2bc22
Author: Chih-Yu Huang <akahuang@google.com>
Date: Wed Oct 12 07:53:27 2016

hwid: change logic of 'Database.VerifyBOM' method.

- Only verify the components in the corresponding image_id
Because the components for each image ID might be different, we only
verify the components in the target image ID, not all components
listed in the database.

- Add 'probeable_only' argument
When the BOM is decoded by HWID string, it would contain every
component in the database. But if the BOM object is created by the
probed result, it does not contain the unprobeable component before
evaluating the rule. We should verify the probeable components only.

BUG= chromium:653369 
TEST=database_unittest.py

Change-Id: Iceb60c2b72a64e7045d5013aeea5248235e31e3e
Reviewed-on: https://chromium-review.googlesource.com/397201
Commit-Ready: Chih-Yu Huang <akahuang@chromium.org>
Tested-by: Chih-Yu Huang <akahuang@chromium.org>
Reviewed-by: Wei-Han Chen <stimim@chromium.org>

[modify] https://crrev.com/e4b0934892ccefedac0e42cb92c973266cd2bc22/py/hwid/v3/database.py
[modify] https://crrev.com/e4b0934892ccefedac0e42cb92c973266cd2bc22/py/hwid/v3/database_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/faea302dcd7dae49048e49ac1d47e86989fa2a74

commit faea302dcd7dae49048e49ac1d47e86989fa2a74
Author: Chih-Yu Huang <akahuang@google.com>
Date: Wed Oct 12 07:59:37 2016

hwid: BOM only keeps the known components.

Orginally when we decode the HWID string, we set the index of missing
encoded fields to 0. This is because we needed to encode BOM before we
evaluate the unprobeable components.

Since now we can encode the BOM at the last minute, i.e. after it gets
the information of image_id and the unprobeable components, we dont
set the index of the unknown field as 0.

BUG= chromium:653369 
TEST=unittest in py/hwid/v3

Change-Id: I8e19c2917d7b3502b3bbe1c03dac62614ef337d6
Reviewed-on: https://chromium-review.googlesource.com/397202
Commit-Ready: Chih-Yu Huang <akahuang@chromium.org>
Tested-by: Chih-Yu Huang <akahuang@chromium.org>
Reviewed-by: Wei-Han Chen <stimim@chromium.org>

[modify] https://crrev.com/faea302dcd7dae49048e49ac1d47e86989fa2a74/py/hwid/v3/common_unittest.py
[add] https://crrev.com/faea302dcd7dae49048e49ac1d47e86989fa2a74/py/hwid/v3/testdata/new_test_probe_result_hwid_utils.yaml
[modify] https://crrev.com/faea302dcd7dae49048e49ac1d47e86989fa2a74/py/hwid/v3/valid_hwid_db_unittest.py
[modify] https://crrev.com/faea302dcd7dae49048e49ac1d47e86989fa2a74/py/hwid/v3/decoder_unittest.py
[modify] https://crrev.com/faea302dcd7dae49048e49ac1d47e86989fa2a74/py/hwid/v3/encoder_unittest.py
[add] https://crrev.com/faea302dcd7dae49048e49ac1d47e86989fa2a74/py/hwid/v3/testdata/NEW_TEST_BOARD
[modify] https://crrev.com/faea302dcd7dae49048e49ac1d47e86989fa2a74/py/hwid/v3/decoder.py
[modify] https://crrev.com/faea302dcd7dae49048e49ac1d47e86989fa2a74/py/hwid/v3/hwid_utils.py
[modify] https://crrev.com/faea302dcd7dae49048e49ac1d47e86989fa2a74/py/hwid/v3/encoder.py
[modify] https://crrev.com/faea302dcd7dae49048e49ac1d47e86989fa2a74/py/utils/type_utils_unittest.py
[modify] https://crrev.com/faea302dcd7dae49048e49ac1d47e86989fa2a74/py/hwid/v3/common.py
[modify] https://crrev.com/faea302dcd7dae49048e49ac1d47e86989fa2a74/py/hwid/v3/hwid_utils_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/factory/+/134affe6135b3728efbe688e4b1069be160cf5aa

commit 134affe6135b3728efbe688e4b1069be160cf5aa
Author: Chih-Yu Huang <akahuang@google.com>
Date: Wed Oct 12 10:17:48 2016

hwid: fix lint error.

BUG= chromium:653369 
TEST=make lint LINT_WHITELIST='py/hwid/v3/*.py'

Change-Id: Ifbf0ca98fc07924b1fd69d985b1c58c129f68cb2
Reviewed-on: https://chromium-review.googlesource.com/397203
Commit-Ready: Chih-Yu Huang <akahuang@chromium.org>
Tested-by: Chih-Yu Huang <akahuang@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>

[modify] https://crrev.com/134affe6135b3728efbe688e4b1069be160cf5aa/py/hwid/v3/verify_db_pattern.py
[modify] https://crrev.com/134affe6135b3728efbe688e4b1069be160cf5aa/py/hwid/v3/hwid_rule_functions_unittest.py
[modify] https://crrev.com/134affe6135b3728efbe688e4b1069be160cf5aa/py/hwid/v3/rule.py
[modify] https://crrev.com/134affe6135b3728efbe688e4b1069be160cf5aa/py/hwid/v3/common.py
[modify] https://crrev.com/134affe6135b3728efbe688e4b1069be160cf5aa/py/hwid/v3/hwid_utils_unittest.py