New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 725212 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Jun 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 1
Type: Bug


Sign in to add a comment

Compare Windows 10 benchmarks across versions

Project Member Reported by charliea@chromium.org, May 22 2017

Issue description

Right now, the perf lab machines run on Windows 10 version "Threshold 1" (10240), which was released on July 29, 2015.

We believe that a certain bug causing power noise in our benchmarks could be resolved if we updated these machines to a newer version. Right now, the newest version available in Google is "Redstone 1" (14393) released on August 2, 2016.

In order to update these machines, we need to evaluate whether doing so will have a significant impact on any benchmarks that we run. This is a similar problem to the one that we had when updating our Android bots from non-swarming to swarming (https://bugs.chromium.org/p/chromium/issues/detail?id=705134), so I plan to use a similar process.

I'm going to compare the performance of the Windows High-DPI FYI bot before and after the update to Redstone 1 across a few benchmarks:

- system_health.memory_desktop (swarming@  http://crbug.com/705135 )
- speedometer (swarming@  http://crbug.com/705138 )
- memory.desktop (swarming@  http://crbug.com/705136 )
- smoothness.top_25_smooth (swarming@  http://crbug.com/705139 )
- loading.desktop (swarming@  http://crbug.com/705140 )
- thread_times.key_silk_cases (swarming@  http://crbug.com/714750 )

 
Blockedon: 725213
Blockedon: 725214
Blockedon: 725215
Blockedon: 725216
Blockedon: 725217
Blockedon: 725218
FYI - some are currently running on 10586 (Threshold 2). If you need exact numbers/bot names, lmk.
I was only planning on comparing between the FYI bot (running RS1) and the main perf bots (running whatever their existing versions are), so I don't think it should make too much of a difference.
Owner: ----
Status: Untriaged (was: Assigned)
Clearing myself as the owner, as rolling the version of Windows that we use in the lab didn't turn out to be a blocker for more stable power benchmarks.
Status: WontFix (was: Untriaged)
Marking this as WontFix. Feel free to reopen if it's still relevant, but in the short term it's not clear what our update strategy is for Windows lab machines.

Sign in to add a comment