Example failure:
https://codereview.chromium.org/1864173002/

We'll need this to work correctly if we want to be able to do perf tryjobs that test out 2G networks, large pagesets, all the SH benchmarks at once, etc.

Anyone know if this is just a hard-coded limit somewhere that we can remove?