Compare Windows 10 benchmarks across versions |
||||||||
Issue descriptionRight now, the perf lab machines run on Windows 10 version "Threshold 1" (10240), which was released on July 29, 2015. We believe that a certain bug causing power noise in our benchmarks could be resolved if we updated these machines to a newer version. Right now, the newest version available in Google is "Redstone 1" (14393) released on August 2, 2016. In order to update these machines, we need to evaluate whether doing so will have a significant impact on any benchmarks that we run. This is a similar problem to the one that we had when updating our Android bots from non-swarming to swarming (https://bugs.chromium.org/p/chromium/issues/detail?id=705134), so I plan to use a similar process. I'm going to compare the performance of the Windows High-DPI FYI bot before and after the update to Redstone 1 across a few benchmarks: - system_health.memory_desktop (swarming@ http://crbug.com/705135 ) - speedometer (swarming@ http://crbug.com/705138 ) - memory.desktop (swarming@ http://crbug.com/705136 ) - smoothness.top_25_smooth (swarming@ http://crbug.com/705139 ) - loading.desktop (swarming@ http://crbug.com/705140 ) - thread_times.key_silk_cases (swarming@ http://crbug.com/714750 )
,
May 22 2017
,
May 22 2017
,
May 22 2017
,
May 22 2017
,
May 22 2017
,
May 23 2017
FYI - some are currently running on 10586 (Threshold 2). If you need exact numbers/bot names, lmk.
,
May 24 2017
I was only planning on comparing between the FYI bot (running RS1) and the main perf bots (running whatever their existing versions are), so I don't think it should make too much of a difference.
,
Jun 12 2017
Clearing myself as the owner, as rolling the version of Windows that we use in the lab didn't turn out to be a blocker for more stable power benchmarks.
,
Jun 23 2017
Marking this as WontFix. Feel free to reopen if it's still relevant, but in the short term it's not clear what our update strategy is for Windows lab machines. |
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by charliea@chromium.org
, May 22 2017