New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 593423 link

Starred by 4 users

Issue metadata

Status: Verified
Owner:
Last visit > 30 days ago
Closed: Oct 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug

Blocking:
issue 554867
issue 589094
issue 589102
issue 589751
issue 590449
issue 607633



Sign in to add a comment

HWTest failure: provision_AutoUpdate.double,None (ABORT: None)

Project Member Reported by steve...@chromium.org, Mar 9 2016

Issue description

~60 failures (~30 on pfq) from Feb 1 - Mar 9 with the following symptoms:

TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Looking at the difference between a successful run:
https://uberchromegw.corp.google.com/i/chromeos/builders/tricky-chrome-pfq/builds/1609

and a failed run:
https://uberchromegw.corp.google.com/i/chromeos/builders/tricky-chrome-pfq/builds/1608

I see:

* No "Update complete" message in the failed autoserv.INFO log.
* "Collecting crash information" in the failed autoserv.INFO log.
* No client.2 or client.3 logs in the failed /debug/ output dir
* The failed run has contents in crashinfo... with an update_engine.log that appears to be truncated at "Completed 493/620 operations (79%)"
** I checked another example and it was truncated at 62%

* The failed run I looked at was the long tail on the PFQ run, so my suspicion is that it may have timed out somewhere? 


 
Showing comments 12 - 111 of 111 Older
Labels: -autofiled-count-9 autofiled-count-10 Proj-wolf
[bvt-inline] provision_AutoUpdate.double Failure on wolf-release/R51-8126.0.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: wolf-release/R51-8126.0.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/wolf-release/R51-8126.0.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/58288263-chromeos-test/chromeos4-row1-rack4-host13/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/58288263-chromeos-test/chromeos4-row1-rack4-host13/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=58288263.

You may want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-10 autofiled-count-11
[bvt-inline] provision_AutoUpdate.double Failure on cyan-release/R50-7978.36.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: cyan-release/R50-7978.36.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/cyan-release/R50-7978.36.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/58402054-chromeos-test/chromeos4-row11-rack11-host5/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/58402054-chromeos-test/chromeos4-row11-rack11-host5/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=58402054.

You may want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-11 autofiled-count-12
[bvt-inline] provision_AutoUpdate.double Failure on edgar-release/R50-7978.43.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: edgar-release/R50-7978.43.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/edgar-release/R50-7978.43.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/59021447-chromeos-test/chromeos4-row12-rack8-host13/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/59021447-chromeos-test/chromeos4-row12-rack8-host13/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=59021447.

You may want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-12 autofiled-count-13 Proj-gnawty
[bvt-inline] provision_AutoUpdate.double Failure on gnawty-release/R51-8172.0.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: gnawty-release/R51-8172.0.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/gnawty-release/R51-8172.0.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/59519303-chromeos-test/chromeos4-row12-rack2-host5/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/59519303-chromeos-test/chromeos4-row12-rack2-host5/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=59519303.

You may want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-13 autofiled-count-14 Proj-big
[bvt-inline] provision_AutoUpdate.double Failure on nyan_big-release/R49-7834.70.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: nyan_big-release/R49-7834.70.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/nyan_big-release/R49-7834.70.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/59542131-chromeos-test/chromeos4-row5-rack8-host11/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/59542131-chromeos-test/chromeos4-row5-rack8-host11/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=59542131.

You may want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-14 autofiled-count-15
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R52-8193.0.0-b411

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R52-8193.0.0-b411.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R52-8193.0.0-b411.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/60097247-chromeos-test/chromeos4-row2-rack3-host15/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/60097247-chromeos-test/chromeos4-row2-rack3-host15/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=60097247.

You may want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-15 autofiled-count-16
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R52-8215.0.0-b462

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R52-8215.0.0-b462.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R52-8215.0.0-b462.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/60603593-chromeos-test/chromeos4-row2-rack3-host18/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/60603593-chromeos-test/chromeos4-row2-rack3-host18/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=60603593.

You may want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-16 autofiled-count-17
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R52-8216.0.0-b465

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R52-8216.0.0-b465.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R52-8216.0.0-b465.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/60637488-chromeos-test/chromeos4-row2-rack3-host15/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/60637488-chromeos-test/chromeos4-row2-rack3-host15/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=60637488.

You may want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-17 autofiled-count-18
[bvt-inline] provision_AutoUpdate.double Failure on wolf-release/R52-8221.0.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: wolf-release/R52-8221.0.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/wolf-release/R52-8221.0.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/60737908-chromeos-test/chromeos4-row1-rack3-host11/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/60737908-chromeos-test/chromeos4-row1-rack3-host11/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=60737908.

You may want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-18 autofiled-count-19 Proj-clapper
[bvt-inline] provision_AutoUpdate.double Failure on clapper-release/R51-8172.14.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: clapper-release/R51-8172.14.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/clapper-release/R51-8172.14.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/61343018-chromeos-test/chromeos4-row4-rack4-host3/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/61343018-chromeos-test/chromeos4-row4-rack4-host3/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=61343018.

You may want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-19 autofiled-count-20
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R52-8252.0.0-b593

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R52-8252.0.0-b593.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R52-8252.0.0-b593.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/61523168-chromeos-test/chromeos4-row2-rack3-host6/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/61523168-chromeos-test/chromeos4-row2-rack3-host6/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=61523168.

You may want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-20 autofiled-count-21 Proj-speedy
[bvt-inline] provision_AutoUpdate.double Failure on veyron_speedy-release/R50-7978.68.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: veyron_speedy-release/R50-7978.68.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/veyron_speedy-release/R50-7978.68.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/61535589-chromeos-test/chromeos4-row4-rack10-host18/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/61535589-chromeos-test/chromeos4-row4-rack10-host18/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=61535589.

You may want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Cc: -dshi@chromium.org sbasi@chromium.org h...@chromium.org
Owner: fdeng@chromium.org
Reassigning to fdeng@ as he seems to have actually authored the control.double file, and he may be able to better direct the bug.

There have been many more failures in this test in the month and a half since this bug was filed filed.

https://bugs.chromium.org/p/chromium/issues/list?q=provision_AutoUpdate.double

In the three most recent failures, the update failed, apparently while downloading or unpacking.
Blocking: 607633
Labels: -autofiled-count-21 autofiled-count-22 Proj-panther
[bvt-inline] provision_AutoUpdate.double Failure on panther-release/R51-8172.23.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: panther-release/R51-8172.23.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/panther-release/R51-8172.23.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/62214648-chromeos-test/chromeos4-row2-rack8-host5/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/62214648-chromeos-test/chromeos4-row2-rack8-host5/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=62214648.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-22 autofiled-count-23 Proj-pit
[bvt-inline] provision_AutoUpdate.double Failure on peach_pit-tot-chrome-pfq-informational/R52-8288.0.0-b1307

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: peach_pit-tot-chrome-pfq-informational/R52-8288.0.0-b1307.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/peach_pit-tot-chrome-pfq-informational/R52-8288.0.0-b1307.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/62454872-chromeos-test/chromeos4-row8-rack12-host5/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/62454872-chromeos-test/chromeos4-row8-rack12-host5/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=62454872.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-23 autofiled-count-24 Proj-skate
[bvt-inline] provision_AutoUpdate.double Failure on daisy_skate-chrome-pfq/R52-8296.0.0-rc1

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: daisy_skate-chrome-pfq/R52-8296.0.0-rc1.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/daisy_skate-chrome-pfq/R52-8296.0.0-rc1.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/62649301-chromeos-test/chromeos4-row9-rack5-host7/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/62649301-chromeos-test/chromeos4-row9-rack5-host7/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=62649301.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-24 autofiled-count-25
[bvt-inline] provision_AutoUpdate.double Failure on veyron_speedy-release/R52-8306.0.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: veyron_speedy-release/R52-8306.0.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/veyron_speedy-release/R52-8306.0.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/62853580-chromeos-test/chromeos4-row4-rack11-host18/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/62853580-chromeos-test/chromeos4-row4-rack11-host18/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=62853580.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-25 autofiled-count-26
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R52-8311.0.0-b738

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R52-8311.0.0-b738.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R52-8311.0.0-b738.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63025473-chromeos-test/chromeos4-row2-rack3-host19/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63025473-chromeos-test/chromeos4-row2-rack3-host19/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=63025473.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-26 autofiled-count-27
[bvt-inline] provision_AutoUpdate.double Failure on tricky-chrome-pfq/R52-8311.0.0-rc2

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-chrome-pfq/R52-8311.0.0-rc2.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-chrome-pfq/R52-8311.0.0-rc2.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63026535-chromeos-test/chromeos4-row2-rack3-host10/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63026535-chromeos-test/chromeos4-row2-rack3-host10/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=63026535.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-27 autofiled-count-28 Proj-rikku
[bvt-inline] provision_AutoUpdate.double Failure on rikku-release/R50-7978.74.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: rikku-release/R50-7978.74.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/rikku-release/R50-7978.74.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63025002-chromeos-test/chromeos4-row3-rack5-host5/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63025002-chromeos-test/chromeos4-row3-rack5-host5/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=63025002.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-28 autofiled-count-29 Proj-x86
[bvt-inline] provision_AutoUpdate.double Failure on x86-zgb-release/R50-7978.76.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: x86-zgb-release/R50-7978.76.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/x86-zgb-release/R50-7978.76.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63162132-chromeos-test/chromeos4-row4-rack1-host13/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63162132-chromeos-test/chromeos4-row4-rack1-host13/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=63162132.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-29 autofiled-count-30
[bvt-inline] provision_AutoUpdate.double Failure on tricky-release/R51-8172.32.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-release/R51-8172.32.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-release/R51-8172.32.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63177526-chromeos-test/chromeos4-row2-rack4-host5/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63177526-chromeos-test/chromeos4-row2-rack4-host5/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=63177526.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-30 autofiled-count-31
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R52-8316.0.0-b748

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R52-8316.0.0-b748.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R52-8316.0.0-b748.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63181961-chromeos-test/chromeos4-row2-rack3-host3/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63181961-chromeos-test/chromeos4-row2-rack3-host3/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=63181961.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-31 autofiled-count-32
[bvt-inline] provision_AutoUpdate.double Failure on veyron_speedy-release/R52-8316.0.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: veyron_speedy-release/R52-8316.0.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/veyron_speedy-release/R52-8316.0.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63185952-chromeos-test/chromeos4-row4-rack11-host5/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63185952-chromeos-test/chromeos4-row4-rack11-host5/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=63185952.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-32 autofiled-count-33
[bvt-inline] provision_AutoUpdate.double Failure on tricky-release/R50-7978.76.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-release/R50-7978.76.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-release/R50-7978.76.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63183614-chromeos-test/chromeos4-row2-rack3-host8/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63183614-chromeos-test/chromeos4-row2-rack3-host8/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=63183614.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-33 autofiled-count-34 Proj-zako
[bvt-inline] provision_AutoUpdate.double Failure on zako-release/R52-8349.0.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: zako-release/R52-8349.0.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/zako-release/R52-8349.0.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63947325-chromeos-test/chromeos4-row2-rack6-host3/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/63947325-chromeos-test/chromeos4-row2-rack6-host3/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=63947325.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-34 autofiled-count-35 Proj-jaq
[bvt-inline] provision_AutoUpdate.double Failure on veyron_jaq-release/R51-8172.41.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: veyron_jaq-release/R51-8172.41.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/veyron_jaq-release/R51-8172.41.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64088212-chromeos-test/chromeos4-row4-rack6-host15/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64088212-chromeos-test/chromeos4-row4-rack6-host15/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=64088212.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-35 autofiled-count-36 Proj-peppy
[bvt-inline] provision_AutoUpdate.double Failure on peppy-release/R51-8172.42.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: peppy-release/R51-8172.42.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/peppy-release/R51-8172.42.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64193927-chromeos-test/chromeos4-row6-rack12-host1/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64193927-chromeos-test/chromeos4-row6-rack12-host1/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=64193927.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-36 autofiled-count-37 Proj-orco
[bvt-inline] provision_AutoUpdate.double Failure on orco-release/R53-8365.0.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: orco-release/R53-8365.0.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/orco-release/R53-8365.0.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64397744-chromeos-test/chromeos4-row6-rack4-host7/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64397744-chromeos-test/chromeos4-row6-rack4-host7/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=64397744.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-37 autofiled-count-38
[bvt-inline] provision_AutoUpdate.double Failure on tricky-chrome-pfq/R53-8365.0.0-rc2

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-chrome-pfq/R53-8365.0.0-rc2.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-chrome-pfq/R53-8365.0.0-rc2.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64400912-chromeos-test/chromeos4-row2-rack3-host7/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64400912-chromeos-test/chromeos4-row2-rack3-host7/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=64400912.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-38 autofiled-count-39 Proj-enguarde
[bvt-inline] provision_AutoUpdate.double Failure on enguarde-release/R51-8172.44.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: enguarde-release/R51-8172.44.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/enguarde-release/R51-8172.44.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64426189-chromeos-test/chromeos4-row10-rack2-host15/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64426189-chromeos-test/chromeos4-row10-rack2-host15/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=64426189.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-39 autofiled-count-40
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8373.0.0-b880

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8373.0.0-b880.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8373.0.0-b880.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64692972-chromeos-test/chromeos4-row2-rack3-host6/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64692972-chromeos-test/chromeos4-row2-rack3-host6/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=64692972.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-40 autofiled-count-41
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8384.0.0-b919

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8384.0.0-b919.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8384.0.0-b919.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64985313-chromeos-test/chromeos4-row2-rack3-host14/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64985313-chromeos-test/chromeos4-row2-rack3-host14/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=64985313.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-41 autofiled-count-42
[bvt-inline] provision_AutoUpdate.double Failure on wolf-release/R52-8350.11.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: wolf-release/R52-8350.11.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/wolf-release/R52-8350.11.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64999051-chromeos-test/chromeos4-row1-rack5-host5/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/64999051-chromeos-test/chromeos4-row1-rack5-host5/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=64999051.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-42 autofiled-count-43
[bvt-inline] provision_AutoUpdate.double Failure on zako-release/R52-8350.11.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: zako-release/R52-8350.11.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/zako-release/R52-8350.11.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65010544-chromeos-test/chromeos4-row2-rack6-host8/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65010544-chromeos-test/chromeos4-row2-rack6-host8/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65010544.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-43 autofiled-count-44
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8389.0.0-b932

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8389.0.0-b932.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8389.0.0-b932.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65126673-chromeos-test/chromeos4-row2-rack3-host3/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65126673-chromeos-test/chromeos4-row2-rack3-host3/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65126673.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-44 autofiled-count-45
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8392.0.0-b940

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8392.0.0-b940.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8392.0.0-b940.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65199358-chromeos-test/chromeos4-row2-rack3-host3/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65199358-chromeos-test/chromeos4-row2-rack3-host3/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65199358.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-45 autofiled-count-46
[bvt-inline] provision_AutoUpdate.double Failure on cyan-release/R52-8350.13.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: cyan-release/R52-8350.13.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/cyan-release/R52-8350.13.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65215012-chromeos-test/chromeos4-row11-rack11-host3/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65215012-chromeos-test/chromeos4-row11-rack11-host3/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65215012.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-46 autofiled-count-47
[bvt-inline] provision_AutoUpdate.double Failure on peach_pit-tot-chrome-pfq-informational/R53-8392.0.0-b1573

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: peach_pit-tot-chrome-pfq-informational/R53-8392.0.0-b1573.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/peach_pit-tot-chrome-pfq-informational/R53-8392.0.0-b1573.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65216817-chromeos-test/chromeos4-row8-rack13-host15/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65216817-chromeos-test/chromeos4-row8-rack13-host15/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65216817.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-47 autofiled-count-48
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8396.0.0-b950

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8396.0.0-b950.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8396.0.0-b950.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65294031-chromeos-test/chromeos4-row2-rack3-host17/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65294031-chromeos-test/chromeos4-row2-rack3-host17/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65294031.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-48 autofiled-count-49
[bvt-inline] provision_AutoUpdate.double Failure on peach_pit-tot-chrome-pfq-informational/R53-8396.0.0-b1582

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: peach_pit-tot-chrome-pfq-informational/R53-8396.0.0-b1582.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/peach_pit-tot-chrome-pfq-informational/R53-8396.0.0-b1582.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65301927-chromeos-test/chromeos4-row8-rack12-host9/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65301927-chromeos-test/chromeos4-row8-rack12-host9/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65301927.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-49 autofiled-count-50
[bvt-inline] provision_AutoUpdate.double Failure on daisy_skate-chrome-pfq/R53-8405.0.0-rc1

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: daisy_skate-chrome-pfq/R53-8405.0.0-rc1.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/daisy_skate-chrome-pfq/R53-8405.0.0-rc1.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65525070-chromeos-test/chromeos4-row9-rack5-host7/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65525070-chromeos-test/chromeos4-row9-rack5-host7/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65525070.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-50 autofiled-count-51
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8405.0.0-b1003

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8405.0.0-b1003.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8405.0.0-b1003.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65533412-chromeos-test/chromeos4-row2-rack3-host15/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65533412-chromeos-test/chromeos4-row2-rack3-host15/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65533412.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-51 autofiled-count-52
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8405.0.0-b1004

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8405.0.0-b1004.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8405.0.0-b1004.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65549426-chromeos-test/chromeos4-row2-rack3-host19/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65549426-chromeos-test/chromeos4-row2-rack3-host19/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65549426.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-52 autofiled-count-53 Proj-guado
[bvt-inline] provision_AutoUpdate.double Failure on guado-release/R53-8410.0.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: guado-release/R53-8410.0.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/guado-release/R53-8410.0.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65666443-chromeos-test/chromeos4-row3-rack8-host3/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65666443-chromeos-test/chromeos4-row3-rack8-host3/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65666443.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-53 autofiled-count-54
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8410.0.0-b1014

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8410.0.0-b1014.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8410.0.0-b1014.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65665932-chromeos-test/chromeos4-row2-rack3-host17/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65665932-chromeos-test/chromeos4-row2-rack3-host17/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65665932.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-54 autofiled-count-55
[bvt-inline] provision_AutoUpdate.double Failure on tricky-chrome-pfq/R53-8410.0.0-rc1

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-chrome-pfq/R53-8410.0.0-rc1.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-chrome-pfq/R53-8410.0.0-rc1.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65663698-chromeos-test/chromeos4-row2-rack4-host3/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65663698-chromeos-test/chromeos4-row2-rack4-host3/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65663698.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-55 autofiled-count-56
[bvt-inline] provision_AutoUpdate.double Failure on clapper-release/R53-8410.0.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: clapper-release/R53-8410.0.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/clapper-release/R53-8410.0.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65675378-chromeos-test/chromeos4-row4-rack3-host3/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65675378-chromeos-test/chromeos4-row4-rack3-host3/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65675378.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-56 autofiled-count-57
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8410.0.0-b1015

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8410.0.0-b1015.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8410.0.0-b1015.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65690606-chromeos-test/chromeos4-row2-rack3-host3/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65690606-chromeos-test/chromeos4-row2-rack3-host3/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65690606.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-57 autofiled-count-58
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8418.0.0-b1028

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8418.0.0-b1028.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8418.0.0-b1028.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65874126-chromeos-test/chromeos4-row2-rack3-host22/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65874126-chromeos-test/chromeos4-row2-rack3-host22/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65874126.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-58 autofiled-count-59
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8422.0.0-b1039

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8422.0.0-b1039.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8422.0.0-b1039.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65974741-chromeos-test/chromeos4-row2-rack3-host13/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65974741-chromeos-test/chromeos4-row2-rack3-host13/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65974741.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-59 autofiled-count-60
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8422.0.0-b1040

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8422.0.0-b1040.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8422.0.0-b1040.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65997261-chromeos-test/chromeos4-row2-rack3-host20/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/65997261-chromeos-test/chromeos4-row2-rack3-host20/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=65997261.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-60 autofiled-count-61
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8426.0.0-b1049

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8426.0.0-b1049.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8426.0.0-b1049.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66082649-chromeos-test/chromeos4-row2-rack3-host21/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66082649-chromeos-test/chromeos4-row2-rack3-host21/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=66082649.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-61 autofiled-count-62 Proj-jerry
[bvt-inline] provision_AutoUpdate.double Failure on veyron_jerry-release/R53-8426.0.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: veyron_jerry-release/R53-8426.0.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/veyron_jerry-release/R53-8426.0.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66087610-chromeos-test/chromeos4-row4-rack5-host8/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66087610-chromeos-test/chromeos4-row4-rack5-host8/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=66087610.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-62 autofiled-count-63
[bvt-inline] provision_AutoUpdate.double Failure on tricky-chrome-pfq/R53-8426.0.0-rc1

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-chrome-pfq/R53-8426.0.0-rc1.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-chrome-pfq/R53-8426.0.0-rc1.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66086191-chromeos-test/chromeos4-row2-rack4-host15/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66086191-chromeos-test/chromeos4-row2-rack4-host15/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=66086191.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-63 autofiled-count-64
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8426.0.0-b1050

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8426.0.0-b1050.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8426.0.0-b1050.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66095138-chromeos-test/chromeos4-row2-rack3-host6/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66095138-chromeos-test/chromeos4-row2-rack3-host6/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=66095138.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-64 autofiled-count-65
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8430.0.0-b1060

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8430.0.0-b1060.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8430.0.0-b1060.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66202286-chromeos-test/chromeos4-row2-rack3-host20/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66202286-chromeos-test/chromeos4-row2-rack3-host20/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=66202286.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-65 autofiled-count-66
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8431.0.0-b1063

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8431.0.0-b1063.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8431.0.0-b1063.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66232043-chromeos-test/chromeos4-row2-rack3-host14/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66232043-chromeos-test/chromeos4-row2-rack3-host14/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=66232043.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-66 autofiled-count-67
[bvt-inline] provision_AutoUpdate.double Failure on peach_pit-chrome-pfq/R53-8451.0.0-rc2

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: peach_pit-chrome-pfq/R53-8451.0.0-rc2.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/peach_pit-chrome-pfq/R53-8451.0.0-rc2.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66734053-chromeos-test/chromeos4-row8-rack12-host19/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66734053-chromeos-test/chromeos4-row8-rack12-host19/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=66734053.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-67 autofiled-count-68
[bvt-inline] provision_AutoUpdate.double Failure on tricky-chrome-pfq/R53-8455.0.0-rc2

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-chrome-pfq/R53-8455.0.0-rc2.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-chrome-pfq/R53-8455.0.0-rc2.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66817432-chromeos-test/chromeos4-row2-rack3-host4/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66817432-chromeos-test/chromeos4-row2-rack3-host4/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=66817432.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-68 autofiled-count-69
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8455.0.0-b1146

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8455.0.0-b1146.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8455.0.0-b1146.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66821996-chromeos-test/chromeos4-row2-rack3-host20/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66821996-chromeos-test/chromeos4-row2-rack3-host20/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=66821996.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-69 autofiled-count-70
[bvt-inline] provision_AutoUpdate.double Failure on daisy_skate-chrome-pfq/R53-8456.0.0-rc1

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: daisy_skate-chrome-pfq/R53-8456.0.0-rc1.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/daisy_skate-chrome-pfq/R53-8456.0.0-rc1.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66861457-chromeos-test/chromeos4-row9-rack6-host7/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66861457-chromeos-test/chromeos4-row9-rack6-host7/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=66861457.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-70 autofiled-count-71 Proj-quawks
[bvt-inline] provision_AutoUpdate.double Failure on quawks-release/R53-8459.0.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: quawks-release/R53-8459.0.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/quawks-release/R53-8459.0.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66913377-chromeos-test/chromeos4-row4-rack8-host6/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/66913377-chromeos-test/chromeos4-row4-rack8-host6/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=66913377.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-71 autofiled-count-72
[bvt-inline] provision_AutoUpdate.double Failure on peach_pit-tot-chrome-pfq-informational/R53-8464.0.0-b1818

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: peach_pit-tot-chrome-pfq-informational/R53-8464.0.0-b1818.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/peach_pit-tot-chrome-pfq-informational/R53-8464.0.0-b1818.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/67079144-chromeos-test/chromeos4-row8-rack12-host1/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/67079144-chromeos-test/chromeos4-row8-rack12-host1/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=67079144.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-72 autofiled-count-73
[bvt-inline] provision_AutoUpdate.double Failure on wolf-release/R53-8466.0.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: wolf-release/R53-8466.0.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/wolf-release/R53-8466.0.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/67107903-chromeos-test/chromeos4-row1-rack3-host13/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/67107903-chromeos-test/chromeos4-row1-rack3-host13/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=67107903.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-73 autofiled-count-74
[bvt-inline] provision_AutoUpdate.double Failure on peach_pit-tot-chrome-pfq-informational/R53-8471.0.0-b1830

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: peach_pit-tot-chrome-pfq-informational/R53-8471.0.0-b1830.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/peach_pit-tot-chrome-pfq-informational/R53-8471.0.0-b1830.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/67212697-chromeos-test/chromeos4-row7-rack13-host15/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/67212697-chromeos-test/chromeos4-row7-rack13-host15/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=67212697.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Cc: glevin@chromium.org
Components: -Build Infra>Client>ChromeOS
Labels: OS-Chrome
Recent failure with the same symptoms:
https://uberchromegw.corp.google.com/i/chromeos/builders/tricky-chrome-pfq/builds/2034

Suspicious lines:
06/27 06:46:35.619 DEBUG|          ssh_host:0180| Running (ssh) 'cgpt show -n -i 2 -P $(rootdev -s -d)'
06/27 07:01:36.267 ERROR|        base_utils:0278| [stderr] Write failed: Broken pipe

Labels: -autofiled-count-74 autofiled-count-75 Proj-falco
[bvt-inline] provision_AutoUpdate.double Failure on falco-chrome-pfq/R53-8510.0.0-rc1

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: falco-chrome-pfq/R53-8510.0.0-rc1.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/falco-chrome-pfq/R53-8510.0.0-rc1.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/67942340-chromeos-test/chromeos2-row24-rack4-host5/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/67942340-chromeos-test/chromeos2-row24-rack4-host5/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=67942340.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-75 autofiled-count-76
[bvt-inline] provision_AutoUpdate.double Failure on peach_pit-tot-chrome-pfq-informational/R53-8511.0.0-b2000

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: peach_pit-tot-chrome-pfq-informational/R53-8511.0.0-b2000.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/peach_pit-tot-chrome-pfq-informational/R53-8511.0.0-b2000.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/67947373-chromeos-test/chromeos4-row8-rack13-host15/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/67947373-chromeos-test/chromeos4-row8-rack13-host15/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=67947373.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-76 autofiled-count-77
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8511.0.0-b1274

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8511.0.0-b1274.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8511.0.0-b1274.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/67953853-chromeos-test/chromeos4-row2-rack3-host22/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/67953853-chromeos-test/chromeos4-row2-rack3-host22/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=67953853.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-77 autofiled-count-78
[bvt-inline] provision_AutoUpdate.double Failure on peach_pit-chrome-pfq/R53-8515.0.0-rc1

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: peach_pit-chrome-pfq/R53-8515.0.0-rc1.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/peach_pit-chrome-pfq/R53-8515.0.0-rc1.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/68028079-chromeos-test/chromeos4-row7-rack12-host11/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/68028079-chromeos-test/chromeos4-row7-rack12-host11/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=68028079.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-78 autofiled-count-79
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8517.0.0-b1286

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8517.0.0-b1286.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8517.0.0-b1286.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/68060207-chromeos-test/chromeos4-row2-rack3-host14/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/68060207-chromeos-test/chromeos4-row2-rack3-host14/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=68060207.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-79 autofiled-count-80
[bvt-inline] provision_AutoUpdate.double Failure on veyron_jaq-release/R53-8517.0.0

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: veyron_jaq-release/R53-8517.0.0.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/veyron_jaq-release/R53-8517.0.0.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/68065338-chromeos-test/chromeos4-row4-rack6-host14/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/68065338-chromeos-test/chromeos4-row4-rack6-host14/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=68065338.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Labels: -autofiled-count-80 autofiled-count-81
[bvt-inline] provision_AutoUpdate.double Failure on tricky-tot-chrome-pfq-informational/R53-8517.0.0-b1287

This report is automatically generated to track the following Failure:
Test: provision_AutoUpdate.double.
Suite: bvt-inline.
Chrome Version: None.
Build: tricky-tot-chrome-pfq-informational/R53-8517.0.0-b1287.

Reason:
None.
build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R53-8517.0.0-b1287.
results log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/68076569-chromeos-test/chromeos4-row2-rack3-host19/debug/.
status log: http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/68076569-chromeos-test/chromeos4-row2-rack3-host19/status.log.
buildbot stages: NA.
job link: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=68076569.

You may want to check the test history on wmatrix: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=provision_AutoUpdate.double
You may also want to check the test retry dashboard in case this is a flakey test: https://wmatrix.googleplex.com/retry_teststats/?days_back=30&tests=provision_AutoUpdate.double


ANCHOR  TestFailure{bvt-inline,provision_AutoUpdate.double,None}

Blocking: 590449
Blocking: 554867
Blocking: 589094
Blocking: 589102
Blocking: 589751
Labels: -Proj-Enguarde -Proj-Peppy -proj-pit -Proj-tricky -Proj-Gnawty -Proj-Panther -Proj-orco -proj-speedy -Proj-Clapper -Proj-skate -Proj-wolf -Proj-zako -Proj-falco Proj-Speedy Proj-Wolf proj-enguarde Proj-gnawty proj-Peppy proj-Falco Proj-Orco Proj-Tricky proj-Pit Proj-Zako Proj-panther Proj-clapper Proj-Skate
Another failed PFQ run: https://uberchromegw.corp.google.com/i/chromeos/builders/peach_pit-chrome-pfq/builds/2335
Cc: dshi@chromium.org xixuan@chromium.org
Status: Assigned
Failed informational PFQ run: 
https://uberchromegw.corp.google.com/i/chromeos.chrome/builders/peach_pit-tot-chrome-pfq-informational/builds/2540

Owner: lpique@chromium.org
Assigning it to myself to try and fix directly, as a gardener.
After spending a significant amount of time trying what was going on, it looks like the problem is simply that the provision_AutoUpdate.double test is hitting the control.bvt-inline suite configured "max_runtime_mins" of 20 minutes.

The only hint of it is a line like this in status.log:

INFO	----	----	Job aborted by autotest_system on 2016-08-15 15:05:28

(per the FAQ at https://sites.google.com/a/google.com/autotest/faq#Q_Autotest_system_aborted_my_job, it means your job hit either the timeout or max_runtime limits)

Its unfortunate there is no message for it in the debug/autoserve.* logs. From looking at the sources for autotest/files/scheduler, I saw that there was a "Aborting entry %s due to max runtime" message logged, but I'm not sure where. It is otherwise handled as a SIGTERM sent to the job, and so there is not a lot of detail possible anyway.

There does not seem to be a way to override max_runtime_mins on a test-by-test basis, and this test is included in two suites (bvt-inline, push_to_prod), both of which use max_runtime_mins=20.

The simplest fix would seem to be to increase max_runtime_mins for both suites. It is also possible we need to move it out of bvt-inline since it is meant for faster tests.
Project Member

Comment 96 by bugdroid1@chromium.org, Sep 7 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/9ae6011b0c4816d26402bdb9145d18a2b87bf99b

commit 9ae6011b0c4816d26402bdb9145d18a2b87bf99b
Author: Lloyd Pique <lpique@google.com>
Date: Wed Aug 24 22:57:34 2016

provision_AutoUpdate.double needs 30 minutes to run

The suite limit of 20 minutes was not quite enough time for the test to
run in consistently. The sheriffs/gardeners were seeing occasional red
builds because this test was aborted due to max_runtime being reached.

Typically the logs showed the test was in the process of wrapping up.
About half the time the test was being aborted during post-test cleanup
after the test itself had otherwise completed successfully.

There was no way to set the max_runtime for the test itself, but I left
a quick comment about what to look for, and what needs to be adjusted.

This patch extends the allowed time for all test in the bvt-inline and
push_to_prod suites by ten minutes (to 30 minutes), which is enough
time to get this test to pass with some room to spare.

BUG= chromium:593423 
TEST=Trybot

Change-Id: I616e5619e423325f6dfbd8431d4e5f663678e6ef
Reviewed-on: https://chromium-review.googlesource.com/375201
Tested-by: Lloyd Pique <lpique@google.com>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>

[modify] https://crrev.com/9ae6011b0c4816d26402bdb9145d18a2b87bf99b/test_suites/control.bvt-inline
[modify] https://crrev.com/9ae6011b0c4816d26402bdb9145d18a2b87bf99b/test_suites/control.push_to_prod
[modify] https://crrev.com/9ae6011b0c4816d26402bdb9145d18a2b87bf99b/server/site_tests/provision_AutoUpdate/control.double

I am not sure if this is exactly the same root cause, but it is pretty related.

elm-paladin/builds/530: 
provision_AutoUpdate.double               ABORT: Timed out, did not run.
https://uberchromegw.corp.google.com/i/chromeos/builders/elm-paladin/builds/530/steps/HWTest%20%5Bbvt-inline%5D/logs/stdio


veyron_mighty-paladin/builds/3073
provision_AutoUpdate.double               ABORT: Timed out, did not run.
https://uberchromegw.corp.google.com/i/chromeos/builders/veyron_mighty-paladin/builds/3073/steps/HWTest%20%5Bbvt-inline%5D/logs/stdio




master-paladin builds 12301, 12302 failed as mentioned in comments 97 & 98.

But build 12303 passed. Is it a flake?
I think the timeout in #97 and #98 is different than the original bug. Looking at the logs for the suite for the elm-paladin/builds/530 build:

http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=76181729

It started the suite at 16:55

    09/08 16:55:35.649 INFO |          autoserv:0703| Results placed in /usr/local/autotest/results/76181729-chromeos-test/hostless

The tests didn't really start running until 17:13:

    09/08 16:56:00.884 DEBUG|     dynamic_suite:0554| Waiting on suite.
    09/08 17:13:24.102 INFO |        server_job:0129| START	76181731-chromeos-test/chromeos4-row2-rack10-host17/platform_DMVerityCorruption	

It got as far as login_RetrieveActiveSessions, before something higher priority reduced the availability of the DUT

    09/08 17:20:36.286 DEBUG|             suite:1095| Adding status keyval for login_RetrieveActiveSessions=76181785-chromeos-test
    09/08 17:46:22.689 INFO |        server_job:0129| START	----	elm-paladin/R55-8783.0.0-rc3/bvt-inline/provision_AutoUpdate.double	timestamp=1473381982	localtime=Sep 08 17:46:22

I expect that it started to run the test, realized it was past the suit edeadline of 17:25 (16:55 + 30 minutes), and that is when it aborted before the test really even was started on the DUT (hence no logs there).


Status: Fixed (was: Assigned)
I kept the bug open for a month to see if there would be any new reports, but I didn't see any. The last burst of bugdroid reports was from the end of June, with nothing more recent before I made my fix. 

I'm going to go ahead and close this as fixed.
Labels: VerifyIn-55
Labels: -VerifyIn-55
Cc: josephsih@chromium.org jdiez@chromium.org pho...@chromium.org aga...@chromium.org
 Issue 643871  has been merged into this issue.

Comment 105 by dchan@google.com, Nov 19 2016

Labels: VerifyIn-56

Comment 106 by dchan@google.com, Jan 21 2017

Labels: VerifyIn-57

Comment 107 by dchan@google.com, Mar 4 2017

Labels: VerifyIn-58

Comment 108 by dchan@google.com, Apr 17 2017

Labels: VerifyIn-59

Comment 109 by dchan@google.com, May 30 2017

Labels: VerifyIn-60
Labels: VerifyIn-61
Status: Verified (was: Fixed)
Closing. Please reopen it if its not fixed. Thanks!
Showing comments 12 - 111 of 111 Older

Sign in to add a comment