Apologies for my ignorant, but I don't know what the "reference build" is, nor how to disable them.

Probably, timewise, this started occurring after I landed https://codereview.chromium.org/2285053002

The CL speeds up the test by 10-20x faster, and thus I adjusted the test to be slower by 12x. If the "reference build" is a build a bit older than ToT, it should timeout.

When is the "reference build" going to be updated? If it's going to take days or more, I'm fine to disable the reference build but could you mind to point me how to do that?

This is still failing this week.  

The reference build refers to last known stable build of chrome as opposed to the other tests that are run at head.  

To disable tests on the reference build you add the disabled annotation to the test with the reference target, @decorators.Disabled('reference').  

it would be worth investigating why this is still failing, I am not sure how the reference build is calculated though.  I have cc'd Ned who might be able to give you more insight on the reference build.

Win 10 Perf(3) seems to be failing for the exact same reason.  

See logs here: 

https://uberchromegw.corp.google.com/i/chromium.perf/builders/Win%2010%20Perf%20%283%29/builds/3958/steps/blink_perf.layout.reference/logs/stdio

The failure build's version is in https://github.com/catapult-project/catapult/blob/master/common/py_utils/py_utils/chrome_binaries.json#L144

Usually the action item is just disable with a TODO to reenabled when reference build update. I don't think it's valuable to investigate the ref build failure if TOT is working fine.

is there a timeline on when or how frequent that update happens?

+Kari for questions related to updating the ref build

> To disable tests on the reference build you add the disabled annotation to the test with the reference target, @decorators.Disabled('reference')

Sorry for the troublesome but I still don't know what the disabled annotation is. Searched for tests in PerformanceTests directory without luck. Is there a pointer to help?

sorry, @benchmark.Disabled 

https://cs.chromium.org/chromium/src/tools/perf/benchmarks/v8.py?l=121

Ok, I guess I should add that to "BlinkPerfLayout" in blink_perf.py
https://cs.chromium.org/chromium/src/tools/perf/benchmarks/blink_perf.py?l=230

but that will disable reference for all blink layout perf tests, correct?

Maybe I should make the test amount smaller so that it can run on old builds? We will lose capability to compare with other tests (because the text amount varies then) but I can live with it if required.

Looks like this is troubling enough, and figuring out how to disable single layout perf test over pacific takes too long.

I'll make the test smaller by 12x, https://codereview.chromium.org/2323213002

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f91d8dafdbc7951f3ac8ce4d89c2535b9eeb5249

commit f91d8dafdbc7951f3ac8ce4d89c2535b9eeb5249
Author: kojii <kojii@chromium.org>
Date: Fri Sep 09 11:29:45 2016

Reduce the amount of text to perf test for long-line-nowrap-collapse

When a perf improvements for collapsing whitespace[1] landed,
following tests were adjusted to be 12x slower, to match to the test
without whitespace collapsing.

However, these tests then timeout on reference build.

This patch defers the adjust until the reference build is updated.

PerformanceTests/Layout/long-line-nowrap-collapse.html
PerformanceTests/Layout/long-line-nowrap-spans-collapse.html

[1] https://codereview.chromium.org/2285053002

BUG= 642884 

Review-Url: https://codereview.chromium.org/2323213002
Cr-Commit-Position: refs/heads/master@{#417557}

[modify] https://crrev.com/f91d8dafdbc7951f3ac8ce4d89c2535b9eeb5249/third_party/WebKit/PerformanceTests/Layout/long-line-nowrap-collapse.html
[modify] https://crrev.com/f91d8dafdbc7951f3ac8ce4d89c2535b9eeb5249/third_party/WebKit/PerformanceTests/Layout/long-line-nowrap-spans-collapse.html

By looking at links in  issue 645138 , it looks like the revert in comment #13 fixed many, but not all. Android bots are ok but win bots are not?

Do I need more actions, or is the update of the reference build (which should automatically fix this issue) close enough to wait for?

#14: I think the best fix in the short term is to specifically disable the reference build for Windows.

#15:
> disable the reference build for Windows

Could you tell me how I can do it, or could you do it for me? The advice on #8 disables reference build of all blink_perf.layout on all platforms, correct?

We still notice blink_perf.layout.reference failed on following windows bots:
Win 10 High-DPI Perf (3)
Win 10 Perf (3)
Win 7 Perf (3)
Win 7 x64 Perf (3)
Win 7 ATI GPU Perf (3)
Win 7 Intel GPU Perf (3)
Win 7 Nvidia GPU Perf (3)

I'm going to upload a CL to disable blink_perf.layout.reference benchmark.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c9c60a7a3737679854c05bf23da87d677da5f364

commit c9c60a7a3737679854c05bf23da87d677da5f364
Author: prasadv <prasadv@chromium.org>
Date: Tue Sep 27 01:56:16 2016

Disable blink_perf.layout.reference on windows perf bots.

BUG= 642884 
TBR=nednguyen,kojii

Review-Url: https://codereview.chromium.org/2373693002
Cr-Commit-Position: refs/heads/master@{#421079}

[modify] https://crrev.com/c9c60a7a3737679854c05bf23da87d677da5f364/tools/perf/benchmarks/blink_perf.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3263a553bc0f2e750972280cc70318bd09bdeb76

commit 3263a553bc0f2e750972280cc70318bd09bdeb76
Author: kojii <kojii@chromium.org>
Date: Fri Mar 03 13:30:17 2017

Re-enable blink_perf.layout.reference on windows perf bots

This was disabled in http://crrev.com/2373693002 to make bots happy.

BUG= 642884 

Review-Url: https://codereview.chromium.org/2726813005
Cr-Commit-Position: refs/heads/master@{#454570}

[modify] https://crrev.com/3263a553bc0f2e750972280cc70318bd09bdeb76/tools/perf/benchmarks/blink_perf.py