A while back pprabhu that there is a way to pass free form arguments into a test suite. This could include json that describes which tests to run.

The challenge was getting that argument piped all the way through the various intermediaries.

Including the list of payloads to test into the actual suite request seems like the cleanest and most flexible approach, not least because it allows us to re-run the suite for a given build with a different set of tests.

+1 #1

Even if a solution such as #1 was implemented (which would be cool), we still need the deltas information to be stored somewhere accessible which is why I created this bug. 

Eg. I am writing a new test. This test will not know anything about deltas for a build because that information is currently only stored in the control files of autoupdate_EndToEndTest 

I think it's fully reasonable to require you to know what you are testing to be able schedule a test. This includes the related build artifacts like source versions.

I very specifically don't want any more code in the labs that knows anything about how to go find a build or the payloads for it (and would love to rip out what's there).

But what new test are you building? The control files aren't the only place that data lives, and maybe there is a good place for the code scheduling the test to look.

I agree that having the scheduling code know the source versions is a good approach! I don't know where else this data lives though other than in the control files of paygen_au_canary_control.tar.bz2.

The test I am currently writing is an autoupdate over cellular. (There will probably be others too as I am trying to remove most/all of our manual AU)

I have the test connecting via sim and doing the update.

I just need to put the device on a certain source version first. And I don't know what source version is correct for the current build.

Ah.

I wouldn't depend on the Paygen tests for that. They are intended only to sanity check the payloads we plan to ship to customers, and the payloads aren't even created until well after other HW Tests have completed.

I would instead use the payloads which are generated for AU testing. There are only two.

Two random examples:
  chromeos_R66-10362.0.0_monroe_full_dev.bin
  chromeos_R66-10362.0.0_R66-10362.0.0_monroe_delta_dev.bin

These are used for all hwtesting, and can be used for provisioning in the lab.

I would suggest that for cellular testing, you provision the DUT with the build you want to test, then update it to the same build, once with the full playload, and once with the delta.

The full payload is very large, and the delta is very, very small.

This is exactly how we run the standard AU tests.

PS: Even when you update from same build to same build, you can verify that it worked by checking that you switch OS installs from slot A to slot B. Our standard test provision paths should know to do this.

OK sounds good.