Can you include the control.srv file from the results?

Attaching server control file.

Traceback from autoserv.DEBUG:

server_job:0803| Exception escaped control file, job aborting:
Traceback (most recent call last):
  File "/usr/local/autotest/server/server_job.py", line 795, in run
    self._execute_code(server_control_file, namespace)
  File "/usr/local/autotest/server/server_job.py", line 1295, in _execute_code
    execfile(code_file, namespace, namespace)
  File "/usr/local/autotest/results/3614-lgoodby/hostless/control.srv", line 72, in <module>
    arguments = test_that.parse_arguments(shlex.split(suite_args))
  File "/usr/lib64/python2.7/shlex.py", line 279, in split
    return list(lex)
  File "/usr/lib64/python2.7/shlex.py", line 269, in next
    token = self.get_token()
  File "/usr/lib64/python2.7/shlex.py", line 96, in get_token
    raw = self.read_token()
  File "/usr/lib64/python2.7/shlex.py", line 124, in read_token
    nextchar = self.instream.read(1)
AttributeError: 'dict' object has no attribute 'read'

Deleted: server_control_file 3.7 KB

server_control_file 3.7 KB View Download

I think what is going on is that the moblab version was built during the period of time where I broke test_that.

The suite_args line should look like this

suite_args=':lab: cheets_CTS_N.7.1_r7.arm.CtsTelephonyTestCases'

It should be a string instead of a dict.  I changed it to a dict to make the API more well-defined, but I changed to back to a string to preserve backward compatibility.

I'm assuming that 192.168.231.1 is a moblab.  That moblab running the AFE needs to be updated.

We have a bug open to track adding test_that to our moblab tests: crbug.com/760261

The suite_args in the original description above were not built by the moblab -- those messages are from the goobuntu chroot side, running latest code.

Yes, 192.168.231.1 is a moblab.

Please keep in mind that moblab users will typically be sticking to one of the recent "official" tested releases listed here:

  https://www.chromium.org/chromium-os/testing/moblab/releasenotes

The latest release is R60-9592.88.0.

I see, I did not account for backward compatibility from ToT chroot to stable moblab.

Is it safe to assume that the opposite direction does not need to be supported?

i.e., very old chroot running a test against a new moblab

Re: #7, using an old chroot to run tests against a newer moblab would be unusual.

It might conceivably happen if a developer is trying to launch tests from a branch checkout to verify a patch.

I think there may be a workaround in this case (e.g., use ToT site_utils) so perhaps it is not a concern.

After some thought, maintaining backward compatibility across the RPC is non-trivial, as we do not currently version our RPCs.

Is there a reason to run test_that in a ToT chroot against moblab?  Running site_utils/test_that.py from the same version of Autotest as the moblab should work.  Therefore, possible workarounds:

1. SSH into the moblab and run test_that.
2. Check out the same version of Autotest as the moblab and call test_that from there.

Do these make sense for our moblab users?

Re: #9:

 1) SSH to moblab will not work for a couple reasons:
    - makes the test flow very cumbersome since it requires staging developer builds to the moblab
    - we also run a lab server based on moblab. We don't want developers to have direct SSH access to this machine.

 2) Checking out a specific version of autotest is cumbersome:
    - having developers check out an old branch so they can run tests on ToT is excessive burden.
    - this also complicates staging custom developer builds.

Perhaps consider adding a backward compatibility flag to test_that and run_suite as a workaround.

#11 if that is a satisfactory solution I am happy to implement it.

I don't intend to support such a flag forever.  Does this limited support work?

(from haddowk in the other bug)

Moblab partners run on stable channel so M60 is about to be pushed.

They should never need to test anything before the current stable for a DUT

So when M62 moblab is pushed they should not need to test anything older than M62

Basically whenever M62 moblab is pushed to stable we say everyone just needs the new format.

That may be the best way to support running against older servers, short of adding some kind of API versioning (for example, probe for server version, if that fails fall back to using the old API).

Re: #13, supporting backward compatibility until there is an official stable M62 moblab release seems reasonable.

We will probably want to be able to test older builds on a DUT (for mesh interoperability testing and testing updating from old builds), but I don't think that blocks this deprecation plan.

Patch: https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/661218

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/02b46c55690dcbccaf2687ce3f0a6f6440b959db

commit 02b46c55690dcbccaf2687ce3f0a6f6440b959db
Author: Allen Li <ayatane@chromium.org>
Date: Tue Sep 12 08:12:03 2017

[autotest] Add old RPC support to test_that

BUG= chromium:763207 
TEST=trybot
TEST=test_that :lab: -b reks -i reks-release/R63-9927.0.0 -p suites dummy_Pass -w cautotest
TEST=Run test_that against old moblab (verify bug fix)

Change-Id: I18ee13e8b561b0595358aba4f5cb3e8cb148c140
Reviewed-on: https://chromium-review.googlesource.com/661218
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Laurence Goodby <lgoodby@chromium.org>

[modify] https://crrev.com/02b46c55690dcbccaf2687ce3f0a6f6440b959db/site_utils/run_suite.py
[modify] https://crrev.com/02b46c55690dcbccaf2687ce3f0a6f6440b959db/site_utils/test_that.py