While investigating issues with the Offline Pages UI-Automator bot (issue 791175) I learned that the variations configurations are normally applied to test runs so that unreleased features are tested ahead of release (see issue 848098). It seems that this is the case for local runs but somehwo not for our test bots. This causes differences in test results and creates very confused developers. :)

We should address that so that these flags are consistently applied over both run environments, probably by making sure the configured variations are applied to our bots.