The first stage of this work will be getting the devserver RPC to be able to work with payloads by filename/URI instead of just build number. 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/8e976f60c5c1c8cda793ac8ff04cc401c1137a22

commit 8e976f60c5c1c8cda793ac8ff04cc401c1137a22
Author: David Haddock <dhaddock@chromium.org>
Date: Thu May 04 02:40:27 2017

3/3 Allow auto_update by specifying payload filename.

This adds support for a payload_filename parameter to the devserver
auto_update function. This should be the file we staged by Google
Storage URI in the autoupdate_EndToEndTest. It will be passed to cros_au.

BUG= chromium:709710 
TEST=autoupdate_EndToEndTest

Change-Id: Iee618f280129971ca9abb76af922438e69d9206f
Reviewed-on: https://chromium-review.googlesource.com/471883
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: David Haddock <dhaddock@chromium.org>

[modify] https://crrev.com/8e976f60c5c1c8cda793ac8ff04cc401c1137a22/client/common_lib/cros/dev_server.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/dev-util/+/90e4944949ccb39262e8391693dc50055ac468bb

commit 90e4944949ccb39262e8391693dc50055ac468bb
Author: David Haddock <dhaddock@chromium.org>
Date: Thu May 04 13:07:02 2017

2/3 Allow devserver RPC to use arbitrary payload file for update.

This add support for the payload_filename to the cros_au RPC and transports it
to chromite/lib/auto_updater.py.

BUG= chromium:709710 
TEST=autoupdate_EndToEndTest

Change-Id: I22657038f57d84ba46c65388d5fe3c21759ab72a
Reviewed-on: https://chromium-review.googlesource.com/471882
Commit-Ready: Hsinyu Chao <hychao@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/90e4944949ccb39262e8391693dc50055ac468bb/cros_update.py
[modify] https://crrev.com/90e4944949ccb39262e8391693dc50055ac468bb/devserver.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/b88c40996a5c8c4306b1207561f16d5e160b1635

commit b88c40996a5c8c4306b1207561f16d5e160b1635
Author: David Haddock <dhaddock@chromium.org>
Date: Thu May 04 13:07:03 2017

1/3 Allow devserver RPC to use arbitrary payload file for auto-update.

Today we cannot stage a payload by Google Storage URI and then have cros_au
use that payload file during update. We currently stage by artifact.

This change is one of 3 changes to allow us to autoupdate by specifying the
payload we want to use.

This change copies the payload_filename (that we previously staged) to the
DUT and renames it to update.gz. The devserver on the DUT now responds with
this payload to update pings.

With these changes we can replace the source image setup of
autoupdate_EndToEndTest with the new provision flow. Subsequent changes
will deal with getting the target update in the new provision flow.

BUG= chromium:709710 
TEST=Take all three CLs
TEST=cros flash ssh://<DUT IP> xbuddy://remote/samus/latest-canary
TEST=Existing provision flow. Stage by artifacts and then AU:
ds = dev_server.ImageServer('http://127.0.0.1:8080')
ds.stage_artifacts('samus-release/R60-9495.0.0', ['full_payload',
'stateful', 'autotest_packages'])
ds.auto_update(host, 'samus-release/R60-9495.0.0', force_update=True,
full_update=True)
TEST=The new flow. Stage by google storage URI (this is what the
autoupdate_EndToEndTest does), stage stateful and then AU:
ds = dev_server.ImageServer('http://127.0.0.1:8080')
ds.stage_artifacts('samus-release/R59-9458.0.0', artifacts=None,
files=['chromeos_R59-9458.0.0_samus_full_dev.bin]',
archive_url='gs://chromeos-image-archive/samus-release/R59-9458.0.0')
ds.stage_artifacts('samus-release/R59-9458.0.0', artifacts=None,
files=['stateful.tgz]',
archive_url='gs://chromeos-image-archive/samus-release/R59-9458.0.0'

dev_server.auto_update(host_name=hostname,
build_name='samus-release/R59-9458.0.0',force_update=True,full_update=True,
log_dir=self.job.resultdir,
payload_filename='chromeos_R59-9458.0.0_samus_full_dev.bin')

Change-Id: I2a63614833ff2cf3799e67441c50ad42caf28706
Reviewed-on: https://chromium-review.googlesource.com/471836
Commit-Ready: Hsinyu Chao <hychao@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/b88c40996a5c8c4306b1207561f16d5e160b1635/lib/auto_updater.py

Adding folks who have expressed interest in this test so they can follow along with progress

Progress is currently halted while I work with Verily team on why the test recently began failing for them. See issue 710716. 

Verily bug has a solution now so I'm back to this. 

Several changes need to happen to get the AU test working with the chromite autoupdater (new provision flow). Including:

The au test is essentially a verifier for two sets of event logs (host logs) returned during a single update. The logs come from the rootfs update and a post reboot update check. So we need to enable the host log on the devserver it kicks up on DUT and copy them back to the devserver handling cros_au RPC. 

Then I'll update collect_cros_au_log RPC to return everything to the test. 

Handling updates from gs://chromeos-releases where the payloads and stateful are not in the same root directory when staged on devserver. 

 Issue 717133  has been merged into this issue.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/3b803ead531b92c76b5fb251706443a842eaba64

commit 3b803ead531b92c76b5fb251706443a842eaba64
Author: David Haddock <dhaddock@chromium.org>
Date: Fri May 19 20:57:47 2017

chromite/autoupdater: Prepare for use with autoupdate_EndToEndTest.

This CL does two thing:

1. The autoupdate_EndToEndTest is essentially a verifier for two sets of
host logs returned from a devserver instance during an update. One
during rootfs update and the other from an update check when the update
is complete. In order to switch from remote devserver to devserver on the
DUT we need to enable the host log on the devserver during the provision flow.
We will also copy these logs back from the DUT so the test can analyse them.

2. The au test stages payloads by google storage uri from
gs://chromeos-releases. This means the stateful.tgz and payload are not
put in the same root directory when they are staged like they would be
when they are staged by artifact. So before we copy the
files from the devserver to the DUT we need to account for this.

BUG= chromium:709710 
TEST=cros flash ssh://<DUT IP> xbuddy://remote/samus/latest-canary
TEST=dev_server.auto_update (exisitng provision flow isnt broken)
TEST=dev_server.auto_update(payload_filename=file)

Change-Id: Ib9135049bf12d646c403aa72fe1eb839add1e1d6
Reviewed-on: https://chromium-review.googlesource.com/503615
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/3b803ead531b92c76b5fb251706443a842eaba64/lib/dev_server_wrapper.py
[modify] https://crrev.com/3b803ead531b92c76b5fb251706443a842eaba64/lib/auto_updater.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/a65125c70f5459e35c18bbe552a51d0613005d96

commit a65125c70f5459e35c18bbe552a51d0613005d96
Author: David Haddock <dhaddock@chromium.org>
Date: Fri May 19 23:19:40 2017

autotest: Write all logs returned from collect_cros_au to disk.

Since the RPC call returns multiple logs, the caller should write all of
the files returned to disk.

I have also changed the order it is called within auto_update so the logs are
read and saved first before handler_cleanup is called to delete them.
This required me to update the unittest too.

BUG= chromium:709710 
TEST=collect_cros_au RPC call
TEST=Tested with and without CL:503616
TEST=dev_server.auto_update()
TEST=dev_server.auto_update(payload_filename=...)

Change-Id: Ie6a5e9de73a1b10e8b0efd7409c88383d3dfc0e9
Reviewed-on: https://chromium-review.googlesource.com/503537
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: David Haddock <dhaddock@chromium.org>

[modify] https://crrev.com/a65125c70f5459e35c18bbe552a51d0613005d96/client/common_lib/cros/dev_server.py
[modify] https://crrev.com/a65125c70f5459e35c18bbe552a51d0613005d96/client/common_lib/cros/dev_server_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/dev-util/+/9f459636391cbce4df355c289261728fd1059904

commit 9f459636391cbce4df355c289261728fd1059904
Author: David Haddock <dhaddock@chromium.org>
Date: Sun May 21 02:04:23 2017

devserver: Update collect_cros_au_log RPC to return the host logs.

The collect_cros_au_log RPC currently returns just the executing log.
We will need it to return the host logs saved during the update so that
the autoupdate_EndToEndTest test can analyse them.

BUG= chromium:709710 
TEST=dev_server.auto_update() and verify that the host logs are saved
into job.resultdir when the test is finished.

Change-Id: I7cf14a81da5481ba8474de9cf1d115280eb93dda
Reviewed-on: https://chromium-review.googlesource.com/503616
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/9f459636391cbce4df355c289261728fd1059904/cros_update_progress.py
[modify] https://crrev.com/9f459636391cbce4df355c289261728fd1059904/devserver.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/31bf12f34e4a32309ca1e0b72d97e47f61e38b60

commit 31bf12f34e4a32309ca1e0b72d97e47f61e38b60
Author: David Haddock <dhaddock@chromium.org>
Date: Thu May 25 20:22:46 2017

autotest: Write all logs returned from collect_cros_au to disk.

Since the RPC call returns multiple logs, the caller should write all of
the files returned to disk.

I have also changed the order it is called within auto_update so the logs are
read and saved first before handler_cleanup is called to delete them.
This required me to update the unittest too.

BUG= chromium:709710 
TEST=collect_cros_au RPC call
TEST=Tested with and without CL:503616
TEST=dev_server.auto_update()
TEST=dev_server.auto_update(payload_filename=...)

Change-Id: Ie6a5e9de73a1b10e8b0efd7409c88383d3dfc0e9
Reviewed-on: https://chromium-review.googlesource.com/503537
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: David Haddock <dhaddock@chromium.org>
(cherry picked from commit a65125c70f5459e35c18bbe552a51d0613005d96)
Reviewed-on: https://chromium-review.googlesource.com/516262

[modify] https://crrev.com/31bf12f34e4a32309ca1e0b72d97e47f61e38b60/client/common_lib/cros/dev_server.py
[modify] https://crrev.com/31bf12f34e4a32309ca1e0b72d97e47f61e38b60/client/common_lib/cros/dev_server_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/beadce4ff351c5f2bee137de255735979ef7c660

commit beadce4ff351c5f2bee137de255735979ef7c660
Author: David Haddock <dhaddock@chromium.org>
Date: Wed Jun 14 06:30:09 2017

[autotest] Remove try catch around collect_cros_au_logs.

The devserver has been changed to return json and it is working
successfully so we can remove this check since there are no longer two
cases to account for.

Updating dev_server_unittest too to mock out the json calls.

BUG= chromium:709710 
TEST=test_that <IP> provision_AutoUpdate

Change-Id: I8eec58eb957c47f4051ef377069c8a8fb2ec31ef
Reviewed-on: https://chromium-review.googlesource.com/516511
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/beadce4ff351c5f2bee137de255735979ef7c660/client/common_lib/cros/dev_server.py
[modify] https://crrev.com/beadce4ff351c5f2bee137de255735979ef7c660/client/common_lib/cros/dev_server_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/ff1d8babb1961ff7324de380f8f592005431c9e4

commit ff1d8babb1961ff7324de380f8f592005431c9e4
Author: David Haddock <dhaddock@chromium.org>
Date: Sat Jun 17 08:12:44 2017

[autotest] Remove Brillo v1 update code.

The BrilloUpdater/BrilloTestPlatform implementation was for version one
of Brillo. This is no longer in use. Confirmed by Brillo Test Engineer.

As part of autoupdate_EndToEndTest rewrite, I will remove this code.

BUG= chromium:709710 
TEST=test_that chromeos2-row3-rack10-host13.cros autoupdate_EndToEndTest

Change-Id: I7a5500ea14c33fde7d4cc998d1f536130868e371
Reviewed-on: https://chromium-review.googlesource.com/538915
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Justin Giorgi <jgiorgi@google.com>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/ff1d8babb1961ff7324de380f8f592005431c9e4/server/site_tests/autoupdate_EndToEndTest/autoupdate_EndToEndTest.py
[modify] https://crrev.com/ff1d8babb1961ff7324de380f8f592005431c9e4/client/common_lib/cros/autoupdater.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/1192c41145b533f4138434bfa7e6ba2af9f75330

commit 1192c41145b533f4138434bfa7e6ba2af9f75330
Author: David Haddock <dhaddock@chromium.org>
Date: Wed Jun 21 00:24:18 2017

[autotest] Remove check for old build from autoupdate_EndToEndTest.

Since we have setup a stepping stone in the lab at build R53-8530.96.0
this test will never run against a build older than that. So we can
remove unnecessary checks from the test.

BUG= chromium:709710 
TEST=test_that <ip> autoupdate_EndToEndTest

Change-Id: I457489105fb22e8dd6ec4077f671519760037bc6
Reviewed-on: https://chromium-review.googlesource.com/541736
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/1192c41145b533f4138434bfa7e6ba2af9f75330/server/site_tests/autoupdate_EndToEndTest/autoupdate_EndToEndTest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/dbebc909b6823afcd971bfe98f1a3a1edee6882a

commit dbebc909b6823afcd971bfe98f1a3a1edee6882a
Author: Ilja H. Friedel <ihf@chromium.org>
Date: Thu Jun 22 04:20:34 2017

autoupdate_EndToEndTest: partial revert of commit 0c88356d.

We want tests to be narrow in scope and autoupdate only to check updates.
There is no point in starting ARC container. All that adds at this stage
is flakes in update test. ARC is checked by other tests later.

BUG= chromium:709710 
TEST=None.

Change-Id: Ie928fadb49241d5f0cc8276262c388ffb394682b
Reviewed-on: https://chromium-review.googlesource.com/407581
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>

[modify] https://crrev.com/dbebc909b6823afcd971bfe98f1a3a1edee6882a/server/site_tests/autoupdate_EndToEndTest/autoupdate_EndToEndTest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/3ce538e93371a56c6ff2d1f7c1b9bd4e7c6873fa

commit 3ce538e93371a56c6ff2d1f7c1b9bd4e7c6873fa
Author: David Haddock <dhaddock@chromium.org>
Date: Sat Jun 24 05:56:44 2017

[autotest] Return the PID of the successful autoupdate run.

dev_server.auto_update should return the pid of the succesful update run
so that autoupdate_EndToEndTest can use it to parse the correct logs
about the update.

The dev_server.auto_update() can rerun a few times so we want to know
only the successful pid.

BUG= chromium:709710 
TEST=test_that <ip> provision_AutoUpdate

Change-Id: I49a035c20ac30ae3ad50a2e84310150938e76520
Reviewed-on: https://chromium-review.googlesource.com/544914
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/3ce538e93371a56c6ff2d1f7c1b9bd4e7c6873fa/server/hosts/cros_host.py
[modify] https://crrev.com/3ce538e93371a56c6ff2d1f7c1b9bd4e7c6873fa/client/common_lib/cros/dev_server.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/f337c590bcaf6d1e2829db23a2c4dc76ed62c41b

commit f337c590bcaf6d1e2829db23a2c4dc76ed62c41b
Author: David Haddock <dhaddock@chromium.org>
Date: Fri Jun 30 05:01:13 2017

ChromiumOSUpdater: Allow it to deal with clobber_stateful being False.

If clobber_stateful is false, PostCheckStatefulUpdate() will still check
for files being deleted and fail.

autoupdate_EndToEndTest needs clobber_stateful to be False when it does
the target update. Otherwise update_engine will not know the version
that it updated from.

To speed up the test, PostRebootUpdateCheck() should only be fired on the
target update of autoupdate_EndToEndTest so we check for
payload_filename and clobber_stateful status before running it.

BUG= chromium:709710 
TEST=autoupdate_EndToEndTest with this CL
TEST=cros flash still works

Change-Id: I562b8930371b59f1adeb5ccf3bda0d4ec9b93ab9
Reviewed-on: https://chromium-review.googlesource.com/554264
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/f337c590bcaf6d1e2829db23a2c4dc76ed62c41b/lib/auto_updater.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/fadbfdfc6473770ede87e76ed8530accbb341697

commit fadbfdfc6473770ede87e76ed8530accbb341697
Author: David Haddock <dhaddock@chromium.org>
Date: Fri Jun 30 08:18:24 2017

[autotest] Pass clobber_stateful to cros_au.

This will tell the updater if we want to clobber stateful.
This is currently set to True by default anyway but a coming CL for
autoupdate_EndToEndTest will require it to be false in certain
conditions.

BUG= chromium:709710 
TEST=test_that <IP> provision_AutoUpdate

Change-Id: I5dae2724d166b243003a0e40ac1e5ffa4458bd14
Reviewed-on: https://chromium-review.googlesource.com/557299
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/fadbfdfc6473770ede87e76ed8530accbb341697/client/common_lib/cros/dev_server.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/dev-util/+/20559619622f7a0dfb95aba0202ab39351e65207

commit 20559619622f7a0dfb95aba0202ab39351e65207
Author: David Haddock <dhaddock@chromium.org>
Date: Fri Jun 30 23:02:54 2017

[devserver] Allow handling of clobber_stateful variable.

chromite/lib/autoupdater.py already has a clobber_stateful variable but
the devserver does not pass it to the constructor. We just leave it at the
default of True. autoupdate_EndToEndTest needs this to be False in certain
conditions so we will pass it in.

The if statement here allows this CL to be submitted now without waiting for
any other CLs to catch up and will not break provision_AutoUpdate.

BUG= chromium:709710 
TEST=autoupdate_EndToEnd with this CL

Change-Id: Iac5aa7ca1349f282dd4ed1e31128f5dddd5bb30e
Reviewed-on: https://chromium-review.googlesource.com/554167
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/20559619622f7a0dfb95aba0202ab39351e65207/cros_update.py
[modify] https://crrev.com/20559619622f7a0dfb95aba0202ab39351e65207/devserver.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/52aa4c3c47d82777f1515f9324b020d322cd032e

commit 52aa4c3c47d82777f1515f9324b020d322cd032e
Author: David Haddock <dhaddock@chromium.org>
Date: Thu Jul 06 04:54:51 2017

Show autoupdate_EndToEndTest perf data on Chrome Perf Dashboard.

Currently the test saves the peak memory usage of update_engine_client
with output_perf_value but it is not surfaced because it is not included
in this file.

BUG= chromium:709710 
TEST=None

Change-Id: Ifff50d35964022f37101675a2d8ffcfcfd1e53d4
Reviewed-on: https://chromium-review.googlesource.com/559981
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Rohit Makasana <rohitbm@chromium.org>

[modify] https://crrev.com/52aa4c3c47d82777f1515f9324b020d322cd032e/tko/perf_upload/perf_dashboard_config.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/d0249f17e2129ebf89f7f2f7caff7bc4ca641058

commit d0249f17e2129ebf89f7f2f7caff7bc4ca641058
Author: David Haddock <dhaddock@chromium.org>
Date: Thu Jul 06 19:12:06 2017

[autotest] Remove more dead AU code.

server/autoupdate_utils is not used by anything and it is lives
confusingly close to server/cros/autoupdate_utils that is used.

autoupdate_ParameterizedJob is dead code. I searched the code base and
found a function that looks up a db for that name and returns True if it
was found, else false. So I'm assuming that the function always returns
False and so it can be removed along with its single caller.

BUG= chromium:709710 
TEST=scheduler_models_unittest

Change-Id: I609914b5b8dc4be657a57cd60d4b3dcb1b3562f0
Reviewed-on: https://chromium-review.googlesource.com/550568
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[delete] https://crrev.com/2284834caadd6f3a5ea53d7d93f3ef54be4bf231/client/site_tests/autoupdate_ParameterizedJob/control
[modify] https://crrev.com/d0249f17e2129ebf89f7f2f7caff7bc4ca641058/scheduler/monitor_db.py
[modify] https://crrev.com/d0249f17e2129ebf89f7f2f7caff7bc4ca641058/scheduler/scheduler_models.py
[delete] https://crrev.com/2284834caadd6f3a5ea53d7d93f3ef54be4bf231/server/autoupdate_utils.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/a0dee8465654b79d97cbc1cfca82c42fc95bd8b5

commit a0dee8465654b79d97cbc1cfca82c42fc95bd8b5
Author: David Haddock <dhaddock@chromium.org>
Date: Fri Jul 07 04:13:56 2017

Rewrite autoupdate_EndToEndTest to use the Devserver-on-DUT provision flow.

The test will now follow the same flow as the lab provisioning and cros
flash. This is done with the devserver cros_au RPC. The rootfs
and stateful payloads will be copied to the DUT along
with the devserver code. It will then startup a devserver on the DUT and check
for an update at 127.0.0.1 eliminating network issues.

This test specifies the exact Google Storage URIs for the
payloads that we want to use (source and target). I have done the code for
having cros_au handle exact payload filenames in previous CLs.

The basic flow of this test will now be:
-Stage payloads on a devserver
-Install the source image on the DUT using cros_au RPC
(via dev_server.auto_update()).
-Call cros_au (again via dev_server.auto_update) for the target update and
parse the hostlog collected during the update AFTER the update is complete.
Before this change we made repeated requests to a hostlog URI during the update.

In this CL I have done these things:

1) Remove the OmahaDevserver code from the test. This was used to kick off
a devserver instance for the test on a remote machine. This is no longer
needed.

2) The test verifies an update is succesful by parsing a event "hostlog"
during an update. We will still be parsing this hostlog to check for all
of the expected steps BUT it will now be done with a log file after the
update instead of making many requests to a remote URL during the
update. (The setup for returning the hostlog was done in previous CLs)

3) The TestPlatform interface no longer made sense as is and since there is
now only one implementation (ChromiumOSTestPlatform) I have updated the
API where applicable so that it fits the new logic.

4) Extract the correct values from a Google Storage URI (build_name and
payload_filename) and pass these to cros_au (via dev_server.auto_update).
cros_au is now used to update to the source and target payloads.

5) Stateful update is controlled by cros_au too. The code for selecting
the stateful_update_bin is done by chromite/lib/auto_updater.py

6) Update the control file to have up to steps for running locally.

BUG= chromium:709710 
BUG=chromium:689105
TEST=test_that <ip> autoupdate_EndToEndTest locally
TEST=Used using both a lab devserver and a local devserver
TEST=With Full and Delta payloads
TEST=Tried with payloads from gs://chromeos-image-archive and
gs://chromeos-releases

Change-Id: I0e0064b3301cecb99dc57f7c9eea7ff2bc1d057a
Reviewed-on: https://chromium-review.googlesource.com/544536
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Tested-by: Gwendal Grignou <gwendal@chromium.org>
Reviewed-by: Gwendal Grignou <gwendal@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/a0dee8465654b79d97cbc1cfca82c42fc95bd8b5/server/site_tests/autoupdate_EndToEndTest/autoupdate_EndToEndTest.py
[modify] https://crrev.com/a0dee8465654b79d97cbc1cfca82c42fc95bd8b5/server/site_tests/autoupdate_EndToEndTest/control

FYI 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/0c13495f06d389adeabbb95f858c9b0c836d2492

commit 0c13495f06d389adeabbb95f858c9b0c836d2492
Author: Gwendal Grignou <gwendal@chromium.org>
Date: Fri Jul 07 22:04:15 2017

autoupdate_EndToEndTest: call .auto_update in a single place

Refactor _install[_source]_version so that it can be called when update
to source (prep_device_for_update) or to target (trigger_update).

BUG= chromium:709710 
TEST=Run test:
test_that 100.115.199.98 autoupdate_EndToEndTest \
--args="name=paygen_au_canary
update_type=full
source_release=8172.62.0
target_release=9675.0.0
target_payload_uri=gs://chromeos-releases/canary-channel/sentry/9675.0.0/...
SUITE=paygen_au_canary
source_payload_uri=gs://chromeos-releases/stable-channel/sentry/8172.62.0/...
source_archive_uri=gs://chromeos-releases/stable-channel/sentry/8172.62.0"

Change-Id: I381d1f1fe899a73aa6228d731ee0c7fd227481b5
Signed-off-by: Gwendal Grignou <gwendal@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/563584
Reviewed-by: David Haddock <dhaddock@chromium.org>

[modify] https://crrev.com/0c13495f06d389adeabbb95f858c9b0c836d2492/server/site_tests/autoupdate_EndToEndTest/autoupdate_EndToEndTest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/16af14c4448aad2da7c643c5196b1666e65b1ae4

commit 16af14c4448aad2da7c643c5196b1666e65b1ae4
Author: David Haddock <dhaddock@chromium.org>
Date: Fri Jul 14 02:46:07 2017

Make au_EndToEnd faster when the DUT powerwashes when we downgrade.

autoupdate_EndToEndTest runs multiple times on every build (via
paygen_au_canary). One of the tests it runs everyday in that suite updates from
the stepping stone build (8530.96.0) to the current build.

If the DUT is running a recent build with an encrypted ext4 file system before
the test starts, we run into a problem when we apply the source update to
8530.96.0 which does not have support for ext4 cyrpto: The stateful partition
is no longer acessible.

Luckily when the test runs today it passes. This is because the
dev_server.auto_update tries the update again. It notices that stateful
partition is messed up and restores it before trying the update again. Then it
succeeds.

This is really slow though as we do the entire update to 8530.96.0
again!! (copying payloads, updating rootfs etc)

This CL catches the exception when we try to interact with stateful for
the first time in this condition, restores stateful and continues the update.

To do this I have removed the memoized call to HasRsync() for remote_device.
I argue that the second savings this provides is a lot less than every DUT
spending ~5 extra minutes on this test 10+ times a day (canary, dev,
beta, stable paygen suites). Also the assumption that the remote_device won't
lose rsync once connected is no longer true so it makes sense to remove it for
that reason too.

BUG= chromium:709710 
BUG=chromium:689105
TEST=test_that <DUT IP> autoupdate_EndToEndTest --args="name=paygen_au_canary update_type=full source_release=8530.96.0 target_release=9676.0.0 target_payload_uri=gs://chromeos-releases/canary-channel/samus/9676.0.0/payloads/chromeos_9676.0.0_samus_canary-channel_full_test.bin-7d999bfa297b1a067b81f0eaad90dbe8 SUITE=paygen_au_canary source_payload_uri=gs://chromeos-releases/stable-channel/samus/8530.96.0/payloads/chromeos_8530.96.0_samus_stable-channel_full_test.bin-484fca418d97b16a0e1cd8cfea999fa7"
TEST=cros flash still works

Change-Id: Ib418c0043f253e27cb21b8cbc28c9edd6676bf41
Reviewed-on: https://chromium-review.googlesource.com/567428
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/16af14c4448aad2da7c643c5196b1666e65b1ae4/lib/remote_access.py
[modify] https://crrev.com/16af14c4448aad2da7c643c5196b1666e65b1ae4/lib/auto_updater.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/3f9543109fc26571607da554e285eaf069ab2d51

commit 3f9543109fc26571607da554e285eaf069ab2d51
Author: David Haddock <dhaddock@chromium.org>
Date: Fri Jul 14 02:46:19 2017

Use event log timestamps relative to each other to calculate timeouts.

We read from an event hostlog file AFTER the target update to figure out how
long each event (download_started, update_complete) took.

The old way read from a hostlog URL DURING the update.

Without this change everything is reported as taking 0 seconds
since it is read instanly from the file.

With this change we work out how long each event took by comparing the
timestamps of new each event relative to the previous one.

BUG= chromium:709710 
TEST=autoupdate_EndToEndTest

Change-Id: Id7f573ee311eda2d7b99a628c484a4238f104875
Reviewed-on: https://chromium-review.googlesource.com/564206
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/3f9543109fc26571607da554e285eaf069ab2d51/server/site_tests/autoupdate_EndToEndTest/autoupdate_EndToEndTest.py

Any chance the work here is causing the P0 at https://bugs.chromium.org/p/chromium/issues/detail?id=744212?

Yup probably. There are still a few more changes that need to land on prod that could be causing it

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/9aa70d8b7ed93d6e6e2cb40987b24fe33b7c9d26

commit 9aa70d8b7ed93d6e6e2cb40987b24fe33b7c9d26
Author: David Haddock <dhaddock@chromium.org>
Date: Tue Jul 18 07:32:40 2017

Add performance monitoring script during update.

The autoupdate_EndToEndTest has a script that it uses to measure the
peak memory used during an update. The test now uses the
devserver-on-dut provision flow and we temporarily removed
the performance monitoring flow. This adds it back

The test will copy the the script to the DUT during its setup.
chromite/lib/auto_updater will then kick off the script during rootfs
update and save the results to a file.

This can be submitted before or after the autotest CL.

BUG= chromium:709710 
TEST=cros flash still works
TEST=autoupdate_EndToEnd with this change

Change-Id: I0cccede985d0bdce2d9525939b37ffc61c690c30
Reviewed-on: https://chromium-review.googlesource.com/564324
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/9aa70d8b7ed93d6e6e2cb40987b24fe33b7c9d26/lib/auto_updater.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/aa97f6e69d7a58bb4e0b20fbd41bcadb42575b9a

commit aa97f6e69d7a58bb4e0b20fbd41bcadb42575b9a
Author: David Haddock <dhaddock@chromium.org>
Date: Tue Jul 18 07:32:49 2017

Add performance monitoring back to the AU test.

In CL:544536 the performance monitoring was removed because the device
was restarted before it could retrieve the running test results.

This CL adds the performance monitoring back to the test.
It copies the script to the DUT. During the update the chromite
auto_updater will kick it off and save the results when done.

This CL then pulls the file back from the DUT and records the metrics.
This can be submitted before or after the chromite CL.

BUG= chromium:709710 
TEST=autoupdate_EndToEnd passes

Change-Id: I4cb59e30ac68c1e97ca74136f51bac3f76be050f
Reviewed-on: https://chromium-review.googlesource.com/564340
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/aa97f6e69d7a58bb4e0b20fbd41bcadb42575b9a/server/site_tests/autoupdate_EndToEndTest/autoupdate_EndToEndTest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/43daff2f18ee7ab92e3e7bf0bb57d2bc5864ad5f

commit 43daff2f18ee7ab92e3e7bf0bb57d2bc5864ad5f
Author: David Haddock <dhaddock@chromium.org>
Date: Wed Jul 26 09:07:31 2017

Add paygen variations to the perf dashboard.

These are the names that perf dashboard expects.

BUG= chromium:709710 
TEST=None

Change-Id: Ib8072c7cdefe4df263c5c1aec3b60d442b9df790
Reviewed-on: https://chromium-review.googlesource.com/583740
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Chung-yih Wang <cywang@google.com>

[modify] https://crrev.com/43daff2f18ee7ab92e3e7bf0bb57d2bc5864ad5f/tko/perf_upload/perf_dashboard_config.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/ad748312d1e651e032a37f89d84224fe4023301b

commit ad748312d1e651e032a37f89d84224fe4023301b
Author: David Haddock <dhaddock@chromium.org>
Date: Tue Aug 01 09:39:57 2017

Each step in an update chain can only be one event.

During the AU test we look for a set chain of events (e.g download started,
download finished etc). Once upon a time when update_engine was updated we
needed to account for one of two different event codes that it MAY have output.
This was done with CL:311414.

Since we are never running this test on a build earlier that M53 we can
get rid of this code.

The real reason for this change is it will make rewriting the error messaging
simple. The error messaging from this test is often useless. I'll fix
that in an upcoming CL.

BUG= chromium:709710 
TEST=autoupdate_endtoendtest works

Change-Id: I6bf655ae1ed90eb0a4a61f9b6e23a6a9473d0091
Reviewed-on: https://chromium-review.googlesource.com/583946
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/ad748312d1e651e032a37f89d84224fe4023301b/server/site_tests/autoupdate_EndToEndTest/autoupdate_EndToEndTest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/ea9361e3cef1dadf4ffc73d3c8de909222b51907

commit ea9361e3cef1dadf4ffc73d3c8de909222b51907
Author: David Haddock <dhaddock@chromium.org>
Date: Sun Aug 06 05:07:58 2017

Improve the error messaging for autoupdate_EndToEndTest.

The test verifies that the update went through a set of expected steps.
The update event steps are received from the update_engine hostlog.
We compare the hostlog events against a set of expected events.
This is done AFTER the update is complete.

The old way was to compare the events while the update was in progress
and the current error messages still read like this is the case.

I've updated the error messages for the most common failures to explain
what actually happened:
1. initial update check
2. timeout
3. post reboot update check.

The omaha log is no longer included in the test so I've removed any
mention of that.

I will be refactoring this test a bit soon and will look to improve
the error messaging code then.

BUG= chromium:709710 
TEST=autoupdate_endtoendtest with all error conditions

Change-Id: Ie08e4fe7a5b8459611ba073f0778500bb08b0b64
Reviewed-on: https://chromium-review.googlesource.com/585928
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/ea9361e3cef1dadf4ffc73d3c8de909222b51907/server/site_tests/autoupdate_EndToEndTest/autoupdate_EndToEndTest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/ae0db25e9a83d3ff734a077f7c4671c130a4270f

commit ae0db25e9a83d3ff734a077f7c4671c130a4270f
Author: David Haddock <dhaddock@chromium.org>
Date: Fri Aug 25 00:20:43 2017

[autotest] Refactor TestPlatform in AU test.

autoupdate_EndToEndTest is badly in need of a refactor. This is the
first of a few planned CLs to do that.

The test file contains two classes (300+ lines) that I will be refactoring
in this CL: TestPlatform and ChromiumOSTestPlatform.

These two are used to represent an "updateable os". It is an implementation
left over from a time when the test supported Brillo and ChromeOS.

I am going to remove the TestPlatform interface class and move
ChromiumOSTestPlatform to a separate file/module.

The separate module will represent the CROS DUT in the test.
It will have functions for things the DUT can do: get logs, reboot, update to
payload X etc

The logic for staging payloads currently lives in ChromiumOSTestPlatform.
I will be moving this code to the test class instead. The "DUT"
shouldn't know about staging paylaods.

So the test class will stage payloads, control the flow of the test, and
check the hostlog of events returned from the update at the end.
ChromiumOSTestPlatform will be used to start the update, reboot, login,
get logs.

BUG= chromium:709710 
TEST=autoupdate_EndToEndTest

Change-Id: I9a770bdf9b2c38a83d2801c986184b94f1e0ff01
Reviewed-on: https://chromium-review.googlesource.com/627860
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[add] https://crrev.com/ae0db25e9a83d3ff734a077f7c4671c130a4270f/server/cros/update_engine/chromiumos_test_platform.py
[add] https://crrev.com/ae0db25e9a83d3ff734a077f7c4671c130a4270f/server/cros/update_engine/__init__.py
[modify] https://crrev.com/ae0db25e9a83d3ff734a077f7c4671c130a4270f/server/site_tests/autoupdate_EndToEndTest/autoupdate_EndToEndTest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/f2525bff3aa14b0a14c74734b393fbd0f69aef5c

commit f2525bff3aa14b0a14c74734b393fbd0f69aef5c
Author: David Haddock <dhaddock@chromium.org>
Date: Fri Sep 15 03:14:15 2017

[autotest] Refactor autoupdate_EndToEndTest.

This CL introduces a base test class for update engine related tests:
UpdateEngineTest. autoupdate_EndToEndToTest will inherit from this
class.

During an update, update_engine fires several events. We need to ensure
that the update completes AND that all of these events are what we
expected. These events are returned in the *hostlog* file.

The base class will take care of ensuring the hostlog is correct.
This CL removes classes UpdateEventLogVerifier and ExpectedEventUpdateChain
since the base class will effectivelly be doing the same things.

ExpectedUpdateEvent will be gutted and the parts needed will be moved to
UpdateEngineEvent.

So the flow is now:
1. autoupdate_endtoendtest will stage the payloads and control the flow of
the test.
2. chromiumos_test_platform will start the update with cros_au.
3. update_engine_test will verify that update steps were correct.

CL also includes rewriting several functions and fixing error messages for a
few failure cases.

Some more details:
https://docs.google.com/document/d/1fWulzN095dfiKOQl3XITtbzdPBAFuDxH9mGqHm31NYM/edit#

BUG= chromium:709710 
TEST=test with source and target payload, tested with no source payload
supplied, tested stepping stone build, tested with both Google Storage
buckets, tested all error conditions and timeouts
Change-Id: I3107a61b74c552c4610a1fecab8338514388e1b3
Reviewed-on: https://chromium-review.googlesource.com/654064
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/f2525bff3aa14b0a14c74734b393fbd0f69aef5c/server/site_tests/autoupdate_EndToEndTest/autoupdate_EndToEndTest.py
[add] https://crrev.com/f2525bff3aa14b0a14c74734b393fbd0f69aef5c/server/cros/update_engine/update_engine_test.py
[add] https://crrev.com/f2525bff3aa14b0a14c74734b393fbd0f69aef5c/server/cros/update_engine/update_engine_event.py

paygen_au_canary and paygen_au_dev look great after a weekend of runs with the latest refactor CL.

I'm calling it a win. 

Sweet!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/400d294f9b73ff018eba5edf0b5aeb0d400261fd

commit 400d294f9b73ff018eba5edf0b5aeb0d400261fd
Author: David Haddock <dhaddock@chromium.org>
Date: Wed Sep 27 01:24:10 2017

[autotest] Save metrics for cros_au triggered from autoupdate_EndToEndTest.

autoupdate_EndToEndTest uses dev_server.auto_update() to trigger cros_au
and start an update. When it does this, the devserver tries to parse a
build_name in the format samus-release/RXX-0000.0.0.

But the test is run with two google storage uris from the
gs://chromeos-releases bucket that don't have a build_name in this
format. So it outputs a message that metrics cannot be parsed and it
will continue anyway.

This test runs a lot so we should save metrics from it. The metrics want
the board, release type, and milestone. The only thing I couldn't get
from the URI was the milestone so I'm getting this from the
instructions.json file stored with the bucket.

BUG= chromium:709710 
TEST=autoupdate_EndToEndTest and provision_AutoUpdate pass

Change-Id: I99f00d3ff0cf20487e9fa8ae786674c65ddd9409
Reviewed-on: https://chromium-review.googlesource.com/671218
Commit-Ready: David Haddock <dhaddock@chromium.org>
Tested-by: David Haddock <dhaddock@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/400d294f9b73ff018eba5edf0b5aeb0d400261fd/client/common_lib/cros/dev_server.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/088441fce033187fada981e59ee0b8834c1e6147

commit 088441fce033187fada981e59ee0b8834c1e6147
Author: Aviv Keshet <akeshet@chromium.org>
Date: Thu Sep 28 20:10:23 2017

Revert "[autotest] Save metrics for cros_au triggered from autoupdate_EndToEndTest."

This reverts commit 400d294f9b73ff018eba5edf0b5aeb0d400261fd.

Reason for revert: risk of inconsistent field set

Original change's description:
> [autotest] Save metrics for cros_au triggered from autoupdate_EndToEndTest.
> 
> autoupdate_EndToEndTest uses dev_server.auto_update() to trigger cros_au
> and start an update. When it does this, the devserver tries to parse a
> build_name in the format samus-release/RXX-0000.0.0.
> 
> But the test is run with two google storage uris from the
> gs://chromeos-releases bucket that don't have a build_name in this
> format. So it outputs a message that metrics cannot be parsed and it
> will continue anyway.
> 
> This test runs a lot so we should save metrics from it. The metrics want
> the board, release type, and milestone. The only thing I couldn't get
> from the URI was the milestone so I'm getting this from the
> instructions.json file stored with the bucket.
> 
> BUG= chromium:709710 
> TEST=autoupdate_EndToEndTest and provision_AutoUpdate pass
> 
> Change-Id: I99f00d3ff0cf20487e9fa8ae786674c65ddd9409
> Reviewed-on: https://chromium-review.googlesource.com/671218
> Commit-Ready: David Haddock <dhaddock@chromium.org>
> Tested-by: David Haddock <dhaddock@chromium.org>
> Reviewed-by: Xixuan Wu <xixuan@chromium.org>

Bug:  chromium:709710 
Change-Id: I65e1c30d10493ced16e865b34f697c93b85735e3
Reviewed-on: https://chromium-review.googlesource.com/687955
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>

[modify] https://crrev.com/088441fce033187fada981e59ee0b8834c1e6147/client/common_lib/cros/dev_server.py