Interesting, though given the large amount of benchmarks to be processed & upload in parallel, I don't think at any time we have free cores sitting around not having work to do? Wouldn't that mean splitting up the upload & processing won't save much time?

Maybe I misread the script, but isn't each core basically consist of process + upload?

So let's say there's 2 cores and several tests with a 10s process, 5 second upload

The execution sequence on each core would be 10s processing, 5s block on upload, 10s processing, 5s block on upload

In this case, that's 30s of work.

It might be faster to have 1 pool that's dedicated to processing, and then a separate parallel upload whenever anything is free. Hope I'm explaining that correctly.

So that would be, 10s process, spawn an upload and immedaitely process next 10s, spawn an upload, etc.

So that'd be 20s of processing + the last upload time

Additional tweak: I think you could order them from heaviest to lightest with heaviest uploading first.