New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 697903 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Xoogler
Closed: Aug 2017
Cc:
Components:
EstimatedDays: ----
NextAction: 2017-06-01
OS: All
Pri: 3
Type: Feature



Sign in to add a comment

Add ref build testing for tracing_perftests

Project Member Reported by fmea...@chromium.org, Mar 2 2017

Issue description

This test is very sensitive to any changes related to the device, and it would be useful to run it on the ref build as well.

An issue where this might have helped is:  crbug.com/695522 


 
Not sure we technically can. This is an independent test unit (tracing_perftests) doesn't run in chrome. I don't know if there is any notion of "ref" version of a unittest executable. I think ref builds are only about things that can run in chrome (e.g., telemetry) not standalone executables.
AFAIK: If the unittest is based on a binary that exists in the reference build directory then we should be able to run it.

It depends on how old the ref build currently is.

sullivan@ do you have any insight on that?
Cc: nedngu...@google.com
Components: Speed>Benchmarks
We're working on a long-term plan for supported benchmark harnesses--essentially we will invest effort into getting each harness to work consistently with our toolchain and be easy to add new test cases to. When we get to the harness work for c++ perf tests, supporting a ref build is definitely possible, but we won't be able to work on c++ perf test harness until Q4 since we have a lot to do on higher-priority harnesses.

In parallel, we also need to find a solution to device redundancy. Whatever we come up with here may change the way we use ref builds.

So I'd say this is in scope but it will be several months before we can work on it. Ned, does that line up with your thinking?
Cc: martiniss@chromium.org eyaich@chromium.org
NextAction: 2017-06-01
I think this probably require a change to the perf recipe logic rather than test harness itself. I can imagine a model where we rely on a "reference isolate" to get a pinned version of binary to run the tests on.

However, on the recipe side, we are still working migrating perf tests to swarming & sort out issues due to swarming, so probably may revisit this in a quarter from now.
The NextAction date has arrived: 2017-06-01
Owner: kraynov@chromium.org
Most likely Won't Fix, but assigning to myself as the author of tracing_perftests.
Status: WontFix (was: Available)
Infeasible

Sign in to add a comment