New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 699645 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Last visit > 30 days ago
Closed: Mar 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

crosperf settings_factory unit test fails

Project Member Reported by manojgupta@chromium.org, Mar 8 2017

Issue description

The test was not updated with the changes to experiment iterations.

RUNNING: ./settings_factory_unittest.py
FF..
======================================================================
FAIL: test_init (__main__.BenchmarkSettingsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "././settings_factory_unittest.py", line 22, in test_init
    self.assertEqual(res.GetField('iterations'), 1)
AssertionError: 0 != 1

======================================================================
FAIL: test_init (__main__.GlobalSettingsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "././settings_factory_unittest.py", line 59, in test_init
    self.assertEqual(res.GetField('iterations'), 1)
AssertionError: 0 != 1

----------------------------------------------------------------------
Ran 4 tests in 0.001s


 
Cc: llozano@chromium.org
laszio@, I was talking to Luis and I think we should have at least 2 iterations of each benchmark e.g. webgl_aquarium etc. Otherwise, crosperf can't report p-values.
Can you update default iterations to minimum 2?

Comment 2 by laszio@chromium.org, Mar 10 2017

Would you mind to create a new bug and assign it to me? This seems has nothing to do with this bug.

Comment 3 by laszio@chromium.org, Mar 10 2017

or let's save a bug:
https://chromium-review.googlesource.com/c/452796/
Opened 700467
Project Member

Comment 5 by bugdroid1@chromium.org, Mar 14 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/toolchain-utils/+/a314af4dbfb1cd331ab84ddeb946f8657b56c921

commit a314af4dbfb1cd331ab84ddeb946f8657b56c921
Author: Ting-Yuan Huang <laszio@chromium.org>
Date: Tue Mar 14 20:36:08 2017

crosperf: update unittest for 8332364c

BUG= chromium:699645 
TEST=./crosperf/setttings_factory_unittest.py

Change-Id: Iedab40ce6d6cf4e272a080134730a175f769c4e7
Reviewed-on: https://chromium-review.googlesource.com/452540
Commit-Ready: Ting-Yuan Huang <laszio@chromium.org>
Tested-by: Ting-Yuan Huang <laszio@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>

[modify] https://crrev.com/a314af4dbfb1cd331ab84ddeb946f8657b56c921/crosperf/settings_factory_unittest.py

Status: Verified (was: Untriaged)

Sign in to add a comment