Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8999719824962914400

Kari, only the reference build failed. Is it even possible to bisect against reference build?

===== BISECT JOB RESULTS =====
Status: completed


===== TESTED REVISIONS =====
Revision         Exit Code  Std Dev  N  Good?
chromium@422630  0          N/A      5  good
chromium@422647  0          N/A      5  bad

Bisect job ran on: linux_perf_bisect
Bug ID: 652724

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.dom
Test Metric: long-sibling-list/long-sibling-list.html
Relative Change: 0.00%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6743
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8999719824962914400


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5804616736833536

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

The long-sibling-test was added in this CL:
https://chromium.googlesource.com/chromium/src/+/99c2073218960208c0f54de95b9c31ff9521ba46

And it is failing on the reference build since we do not have support of it there. I do not want to disable the whole reference test suite just for this test, is there a work around?

The best we currently have is to disable the reference build on just the platforms that are failing. Or remove the failing page entirely.

CL is out to disable the reference build for this test on linux and windows:

https://codereview.chromium.org/2397583003/

If this test just failing on the reference build because we don't have modern DOM methods like append on the reference build? I can rewrite the test to use appendChild, etc.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/659111dbb9bdd5a690500aa1d8cd6304dcf22354

commit 659111dbb9bdd5a690500aa1d8cd6304dcf22354
Author: eyaich <eyaich@chromium.org>
Date: Thu Oct 06 14:37:58 2016

Disabling BlinkPerfDom on reference build for win and linux

BUG= chromium:652724 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq

Review-Url: https://codereview.chromium.org/2397583003
Cr-Commit-Position: refs/heads/master@{#423527}

[modify] https://crrev.com/659111dbb9bdd5a690500aa1d8cd6304dcf22354/tools/perf/benchmarks/blink_perf.py

Assigning to Dominic per comment in the CL now that it is landed:

"If this test just failing on the reference build because we don't have modern
DOM methods like append on the reference build? I can rewrite the test to use
appendChild, etc."

Please update the tests and then re-enable.  

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/659111dbb9bdd5a690500aa1d8cd6304dcf22354

commit 659111dbb9bdd5a690500aa1d8cd6304dcf22354
Author: eyaich <eyaich@chromium.org>
Date: Thu Oct 06 14:37:58 2016

Disabling BlinkPerfDom on reference build for win and linux

BUG= chromium:652724 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq

Review-Url: https://codereview.chromium.org/2397583003
Cr-Commit-Position: refs/heads/master@{#423527}

[modify] https://crrev.com/659111dbb9bdd5a690500aa1d8cd6304dcf22354/tools/perf/benchmarks/blink_perf.py

[Automated comment] removing mislabelled merge-merged-2840

 Issue 660063  has been merged into this issue.

Ping?  Any update on a fix for this on the reference build?  

It looks like it is now failing for the reference build on mac as well.  See merged issue as well as Mac Pro 10.11 Perf: 

https://uberchromegw.corp.google.com/i/chromium.perf/builders/Mac%20Pro%2010.11%20Perf/builds/92

Link to log output: 
https://chromium-swarm.appspot.com/user/task/3256a857b7be6c10

Same stack trace: 
Traceback (most recent call last):
  RunBenchmark at /b/s/w/irfH8VBR/third_party/catapult/telemetry/telemetry/internal/story_runner.py:343
    benchmark.ShouldTearDownStateAfterEachStorySetRun())
  Run at /b/s/w/irfH8VBR/third_party/catapult/telemetry/telemetry/internal/story_runner.py:229
    _RunStoryAndProcessErrorIfNeeded(story, results, state, test)
  _RunStoryAndProcessErrorIfNeeded at /b/s/w/irfH8VBR/third_party/catapult/telemetry/telemetry/internal/story_runner.py:87
    state.RunStory(results)
  traced_function at /b/s/w/irfH8VBR/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  RunStory at /b/s/w/irfH8VBR/third_party/catapult/telemetry/telemetry/page/shared_page_state.py:301
    self._current_page, self._current_tab, results)
  traced_function at /b/s/w/irfH8VBR/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  ValidateAndMeasurePage at /b/s/w/irfH8VBR/tools/perf/benchmarks/blink_perf.py:115
    results.current_page, metric, units, values))
  __init__ at /b/s/w/irfH8VBR/third_party/catapult/telemetry/telemetry/value/list_of_scalar_values.py:83
    assert len(values) > 0
AssertionError

I am going to disable this for the reference build on mac as well.  Please investigate and re-enable.

Taking a look.

Patch up at https://codereview.chromium.org/2491203002

 Issue 665470  has been merged into this issue.

Re#15, I still see blink_perf.dom.reference failure on Mac perf bots ( issue 665470 ). It seems it is not disabled on Mac.

Emily, are you going to CQ https://codereview.chromium.org/2483873002/ ?

Chatted with Emily offline. Will wait for dominicc@'s CL to see if it will fix the problem.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3da78c45b2dd07d644decd03d5bcfc02945f4d5b

commit 3da78c45b2dd07d644decd03d5bcfc02945f4d5b
Author: dominicc <dominicc@chromium.org>
Date: Wed Nov 16 16:56:41 2016

Backport the long-sibling-list perf test to reference build DOM APIs.

This test uses .append, etc. which isn't available in the reference
build.

BUG= 652724 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq

Review-Url: https://codereview.chromium.org/2491203002
Cr-Commit-Position: refs/heads/master@{#432522}

[modify] https://crrev.com/3da78c45b2dd07d644decd03d5bcfc02945f4d5b/third_party/WebKit/PerformanceTests/DOM/long-sibling-list.html
[modify] https://crrev.com/3da78c45b2dd07d644decd03d5bcfc02945f4d5b/tools/perf/benchmarks/blink_perf.py

Reassigning back to Zhen to re-enable the test now that Dominic's CL has landed.

I thought I reenabled these? There was a method that disabled the test which I deleted...

Yep, these are getting data again, example chart: https://chromeperf.appspot.com/report?sid=c0153b00141d9ceeb48c2ff3fcfd756982787f473b2eaa82d9fb99b9bb2509c5&rev=434640