We're getting close to be able to run the layout tests under swarming (finally! see bug 524758 ).
I would like for these tests to work like any other isolated script test,
but that can't currently work because we're missing a bunch of functionality.
The most important thing is that the standard isolate command doesn't yet work with an isolate the size of the layout tests. There is a new "exparchive" command that is supposed to work (and that we're testing), but
that means we need to modify the isolate recipes to know whether to use that or not.
Second, we need to make sure that we can correctly merge the JSON test results from multiple shards. Currently the standard mechanism for merging results from isolated scripts doesn't quite work, because the "layout_test_dir" varies from shard to shard, and because run-webkit-tests records tests that are skipped via sharding differently than the merger code expects.
Third, we need to embed the build properties into the test results. In regular script tests, these get passed in a --properties argument; we should probably support that.
Fourth, the layout test results are large and complicated, and we normally upload them to a cloud storage bucket. So, we need to be able to merge the layout-test-results directories from each shard, and then build another complete dir tree and zip and upload them.
I think the way to the second through fourth things is to be able to configure a custom "collect" script that runs when trying to collect the results from swarming. I think this is probably a generically useful hook.
Comment 1 by dpranke@chromium.org
, Dec 21 2016