Someone from layout test Infra should probably take the first pass on this.

The only bots on the CQ that run layout tests currently are:
  linux_chromium_rel_ng (Linux Precise release)
  mac_chromium_rel_ng (Mac 10.9 release)
  win_chromium_rel_ng (Win 7 release)

The advantage of using release builds is that all of the layout test baselines are generated using release builds, and there are some mismatches between the results with release and debug builds.

In theory, we could also have try jobs that run layout tests with debug builds (and with more platforms and OS versions) since for cases where debug builds have results that don't match the baselines, this should be noted in LayoutTests/TestExpectations.

As far as I know, the main reasons why we wouldn't have try bots on the CQ that run layout tests on all platforms with debug builds is resource consumption and cost -- layout tests currently take a long time to run so we'd need more computing time -- which may be more acceptable if layout tests were all run on swarming ( bug 524758 ).

Dirk, does that sound correct?

We don't run any tests in debug configurations in the commit queue (it's not just that we don't run layout tests).

And, yes, the reasons are that debug is too slow, would take too many machines, and the overhead of managing all the different configurations for both infra and users would be too high.

Most test steps run release with DCHECKs enabled, which gives us pretty good debug coverage, but we can't do that for the layout tests because we've sometimes seen that that will produce slight differences in the PNGs, which causes trybots to fail on tests that will pass on the waterfall.

These sorts of failures are pretty rare, so this is viewed as the right tradeoff.

mpearson@, does that all make sense? If so, I'd WontFix this.

This makes sense.  You can WontFix this.  But if as I/we look down the list I made recently of things that passed the CQ and failed on the bots and we see many similar situations, we'll need to revisit it.