New issue
Advanced search Search tips

Issue 743100 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jul 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocking:
issue 741829



Sign in to add a comment

dynamic_suite TypeError when waiting on jobs

Project Member Reported by akes...@chromium.org, Jul 14 2017

Issue description

This is blocking test_push.

07/14 10:15:10.557 DEBUG|        retry_util:0203| <type 'exceptions.TypeError'>(set([6064, 6065, 6066, 6067, 6068]) is not JSON serializable)
07/14 10:15:10.557 DEBUG|        retry_util:0065| Retrying in 10.000000 (10.000000 + jitter 0.000000) seconds ...
 
Blocking: 741829
Bigger log paste below

07/14 10:14:56.150 INFO |          autoserv:0687| Results placed in /usr/local/autotest/results/6062-chromeos-test/hostless
07/14 10:14:56.151 DEBUG|          autoserv:0695| autoserv is running in drone chromeos-drone2-staging.hot.corp.google.com.
07/14 10:14:56.151 DEBUG|          autoserv:0696| autoserv command was: /usr/local/autotest/server/autoserv -p -r /usr/local/autotest/results/6062-chromeos-test/hostless -u chromeos-test -l quawks-release/R58-9334.63.0-test_suites/control.dummy -s --lab True -P 6062-chromeos-test/hostless -n /usr/local/autotest/results/drone_tmp/attach.7
07/14 10:14:56.151 INFO |           pidfile:0016| Logged pid 31421 to /usr/local/autotest/results/6062-chromeos-test/hostless/.autoserv_execute
07/14 10:14:56.153 DEBUG|          autoserv:0406| faulthandler registered on SIGTERM.
07/14 10:14:56.155 DEBUG|          base_job:0357| Persistent state global_properties.test_retry now set to 0
07/14 10:14:56.155 DEBUG|          base_job:0357| Persistent state global_properties.tag now set to '6062-chromeos-test/hostless'
07/14 10:14:56.182 DEBUG|          base_job:0357| Persistent state global_properties.max_result_size_KB now set to 20000
07/14 10:14:56.317 DEBUG|             retry:0155| Converted retries value: 0 -> Retry(total=0, connect=None, read=None, redirect=0)
07/14 10:14:56.317 INFO |    connectionpool:0188| Starting new HTTP connection (1): metadata.google.internal
07/14 10:14:56.421 NOTIC|      cros_logging:0037| ts_mon was set up.
07/14 10:14:56.580 INFO |        server_job:0829| I am PID 31421
07/14 10:14:56.603 WARNI|        subcommand:0081| parallel_simple was called with an empty arglist, did you forget to pass in a list of machines?
07/14 10:14:56.604 WARNI|        server_job:0889| Not checking if job_repo_url contains autotest packages on []
07/14 10:14:56.604 INFO |        server_job:0912| Processing control file
07/14 10:14:56.705 WARNI|             suite:0923| /usr/local/autotest/server/cros/dynamic_suite/suite.py:923: UserWarning: Calling method "name_in_tag_predicate" from Suite is deprecated
  func.__name__)

07/14 10:14:56.706 WARNI|             suite:0923| /usr/local/autotest/server/cros/dynamic_suite/suite.py:923: UserWarning: Calling method "get_test_source_build" from Suite is deprecated
  func.__name__)

07/14 10:14:56.711 WARNI|     dynamic_suite:0346| /usr/local/autotest/server/cros/dynamic_suite/dynamic_suite.py:346: UserWarning: Ignored key 'version_prefix' was passed to suite with value 'cros-version'
  % (key, value))

07/14 10:14:56.711 WARNI|     dynamic_suite:0346| /usr/local/autotest/server/cros/dynamic_suite/dynamic_suite.py:346: UserWarning: Ignored key 'suite_args' was passed to suite with value None
  % (key, value))

07/14 10:14:56.712 WARNI|     dynamic_suite:0346| /usr/local/autotest/server/cros/dynamic_suite/dynamic_suite.py:346: UserWarning: Ignored key 'build' was passed to suite with value 'quawks-release/R58-9334.63.0'
  % (key, value))

07/14 10:14:56.712 WARNI|     dynamic_suite:0346| /usr/local/autotest/server/cros/dynamic_suite/dynamic_suite.py:346: UserWarning: Ignored key 'timeout' was passed to suite with value 24
  % (key, value))

07/14 10:14:56.713 DEBUG|     dynamic_suite:0502| Determined own job id: 6062
07/14 10:14:56.713 DEBUG|             suite:0857| Getting control file list for suite: dummy
07/14 10:14:56.714 DEBUG|             utils:0212| Running 'ssh 100.115.245.253 'curl "http://100.115.245.253:8082/list_suite_controls?suite_name=dummy&build=quawks-release/R58-9334.63.0"''
07/14 10:14:58.388 DEBUG|             suite:0863| Parsed 5 control files.
07/14 10:14:58.389 INFO |        dev_server:1114| Staging artifacts on devserver http://100.115.245.253:8082: build=quawks-release/R58-9334.63.0, artifacts=['control_files', 'test_suites'], files=, archive_url=gs://chromeos-image-archive/quawks-release/R58-9334.63.0
07/14 10:14:58.391 DEBUG|             utils:0212| Running 'ssh 100.115.245.253 'curl "http://100.115.245.253:8082/stage?artifacts=control_files,test_suites&files=&async=True&archive_url=gs://chromeos-image-archive/quawks-release/R58-9334.63.0"''
07/14 10:15:00.085 DEBUG|        dev_server:1056| response for RPC: 'Success'
07/14 10:15:00.085 DEBUG|             utils:0212| Running 'ssh 100.115.245.253 'curl "http://100.115.245.253:8082/is_staged?artifacts=control_files,test_suites&files=&archive_url=gs://chromeos-image-archive/quawks-release/R58-9334.63.0"''
07/14 10:15:01.789 DEBUG|        dev_server:1012| whether artifact is staged: 'True'
07/14 10:15:01.791 INFO |        dev_server:1132| Finished staging artifacts: build=quawks-release/R58-9334.63.0, artifacts=['control_files', 'test_suites'], files=, archive_url=gs://chromeos-image-archive/quawks-release/R58-9334.63.0
07/14 10:15:01.793 DEBUG|             suite:1141| Discovered 5 tests.
07/14 10:15:01.794 INFO |        server_job:0199| INFO	----	Start dummy	timestamp=1500052501	localtime=Jul 14 10:15:01	
07/14 10:15:01.795 DEBUG|             suite:1088| Scheduling dummy_Pass.experimental
07/14 10:15:03.240 DEBUG|             suite:1428| Adding job keyval for dummy_Pass.experimental=6064-chromeos-test
07/14 10:15:03.240 DEBUG|             suite:1088| Scheduling dummy_Pass.bluetooth
07/14 10:15:04.928 DEBUG|             suite:1428| Adding job keyval for dummy_Pass.bluetooth=6065-chromeos-test
07/14 10:15:04.928 DEBUG|             suite:1088| Scheduling dummy_Pass.actionable
07/14 10:15:06.579 DEBUG|             suite:1428| Adding job keyval for dummy_Pass.actionable=6066-chromeos-test
07/14 10:15:06.580 DEBUG|             suite:1088| Scheduling dummy_Pass
07/14 10:15:08.120 DEBUG|             suite:1428| Adding job keyval for dummy_Pass=6067-chromeos-test
07/14 10:15:08.120 DEBUG|             suite:1088| Scheduling dummy_Fail
07/14 10:15:09.251 DEBUG|             suite:1428| Adding job keyval for dummy_Fail=6068-chromeos-test
07/14 10:15:09.252 DEBUG|             suite:1161| Scheduled 5 tests, writing the total to keyval.
07/14 10:15:09.252 DEBUG|     dynamic_suite:0610| Waiting on suite.
07/14 10:15:10.224 DEBUG|         reporting:0038| Bug filing disabled. No module named tlslite.utils
07/14 10:15:10.557 DEBUG|        retry_util:0203| <type 'exceptions.TypeError'>(set([6064, 6065, 6066, 6067, 6068]) is not JSON serializable)
07/14 10:15:10.557 DEBUG|        retry_util:0065| Retrying in 10.000000 (10.000000 + jitter 0.000000) seconds ...
07/14 10:15:20.607 DEBUG|        retry_util:0203| <type 'exceptions.TypeError'>(set([6064, 6065, 6066, 6067, 6068]) is not JSON serializable)
07/14 10:15:20.607 DEBUG|        retry_util:0065| Retrying in 20.000000 (20.000000 + jitter 0.000000) seconds ...
07/14 10:15:40.689 DEBUG|        retry_util:0203| <type 'exceptions.TypeError'>(set([6064, 6065, 6066, 6067, 6068]) is not JSON serializable)
07/14 10:15:40.690 DEBUG|        retry_util:0065| Retrying in 40.000000 (40.000000 + jitter 0.000000) seconds ...
07/14 10:16:20.787 DEBUG|        retry_util:0203| <type 'exceptions.TypeError'>(set([6064, 6065, 6066, 6067, 6068]) is not JSON serializable)
07/14 10:16:20.787 DEBUG|        retry_util:0065| Retrying in 80.000000 (80.000000 + jitter 0.000000) seconds ...
07/14 10:17:40.887 DEBUG|        retry_util:0203| <type 'exceptions.TypeError'>(set([6064, 6065, 6066, 6067, 6068]) is not JSON serializable)
07/14 10:17:40.888 DEBUG|        retry_util:0065| Retrying in 160.000000 (160.000000 + jitter 0.000000) seconds ...
07/14 10:20:20.987 DEBUG|        retry_util:0203| <type 'exceptions.TypeError'>(set([6064, 6065, 6066, 6067, 6068]) is not JSON serializable)
07/14 10:20:20.987 DEBUG|        retry_util:0065| Retrying in 320.000000 (320.000000 + jitter 0.000000) seconds ...
07/14 10:25:41.087 DEBUG|        retry_util:0203| <type 'exceptions.TypeError'>(set([6064, 6065, 6066, 6067, 6068]) is not JSON serializable)
07/14 10:25:41.087 DEBUG|        retry_util:0065| Retrying in 640.000000 (640.000000 + jitter 0.000000) seconds ...
07/14 10:36:21.187 DEBUG|        retry_util:0203| <type 'exceptions.TypeError'>(set([6064, 6065, 6066, 6067, 6068]) is not JSON serializable)
07/14 10:36:21.187 DEBUG|        retry_util:0065| Retrying in 1280.000000 (1280.000000 + jitter 0.000000) seconds ...
Summary: dynamic_suite TypeError when waiting on jobs (was: dynamic_suite TypeError when terminating)
Project Member

Comment 6 by bugdroid1@chromium.org, Jul 14 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/1e00b222bc05a217ef1adaf1a49e8dba03add390

commit 1e00b222bc05a217ef1adaf1a49e8dba03add390
Author: Aviv Keshet <akeshet@chromium.org>
Date: Fri Jul 14 18:31:16 2017

Revert "[autotest] Factor out job result waiting"

This reverts commit 9ad9194e2af1bdbbac8269dcc691c749541a35aa.

BUG= chromium:743100 
TEST=None

Change-Id: I48d59fd91e4423686f0753e65403241075ebf1e1
Reviewed-on: https://chromium-review.googlesource.com/572102
Tested-by: Aviv Keshet <akeshet@chromium.org>
Trybot-Ready: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>

[modify] https://crrev.com/1e00b222bc05a217ef1adaf1a49e8dba03add390/server/cros/dynamic_suite/job_status.py
[modify] https://crrev.com/1e00b222bc05a217ef1adaf1a49e8dba03add390/server/cros/dynamic_suite/job_status_unittest.py

By the way, I think guado_moblab-paladin would have caught this if it hadn't been experimental. Yet another reason to re-enable it. https://uberchromegw.corp.google.com/i/chromeos/builders/guado_moblab-paladin
Cc: pprabhu@chromium.org haddowk@chromium.org
+moblab lovers :)
Project Member

Comment 9 by bugdroid1@chromium.org, Jul 18 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/434b6fc9fcc4c5ddab573fcd5939fecadf362c50

commit 434b6fc9fcc4c5ddab573fcd5939fecadf362c50
Author: Aviv Keshet <akeshet@chromium.org>
Date: Tue Jul 18 01:32:40 2017

chromeos_config: mark guado_moblab as important

BUG= chromium:714330 ,  chromium:743100 
TEST=None

Change-Id: Icb270cd21d19471cee266fbb40ba4725834ade3f
Reviewed-on: https://chromium-review.googlesource.com/572084
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: David Riley <davidriley@chromium.org>

[modify] https://crrev.com/434b6fc9fcc4c5ddab573fcd5939fecadf362c50/cbuildbot/config_dump.json
[modify] https://crrev.com/434b6fc9fcc4c5ddab573fcd5939fecadf362c50/cbuildbot/chromeos_config.py

Project Member

Comment 10 by bugdroid1@chromium.org, Jul 19 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/133dffdd0fb74d67aace196c490f30c05a7a9f04

commit 133dffdd0fb74d67aace196c490f30c05a7a9f04
Author: Allen Li <ayatane@chromium.org>
Date: Wed Jul 19 21:41:22 2017

[autotest] Factor out job result waiting (re-land)

Re-land due to RPC serialization of set bug

BUG= chromium:672348 
BUG= chromium:743100 
TEST=None

Change-Id: I32eb3782b2d6aa2f74af88e610fbe2865ddb29d6
Reviewed-on: https://chromium-review.googlesource.com/572085
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>

[modify] https://crrev.com/133dffdd0fb74d67aace196c490f30c05a7a9f04/server/cros/dynamic_suite/job_status.py
[modify] https://crrev.com/133dffdd0fb74d67aace196c490f30c05a7a9f04/server/cros/dynamic_suite/job_status_unittest.py

Status: Fixed (was: Untriaged)

Sign in to add a comment