New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 809827 link

Starred by 5 users

Issue metadata

Status: Assigned
Owner:
Cc:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

caroline-tot-chrome-pfq-informational failure with error "Timed out waiting for unnamed condition"

Project Member Reported by xiaoyinh@chromium.org, Feb 7 2018

Issue description

caroline-tot-chrome-pfq-informational failure in build #929, #927, #926, #925 with the same error "Timed out waiting for unnamed condition" but in different hwtest. 

This appears to happen in different DUT, but the failure point looks the same:

02/06 15:41:17.483 ERROR|             utils:2756| Timed out waiting for unnamed condition
02/06 15:41:17.496 WARNI|              test:0637| The test failed with the following exception
Traceback (most recent call last):
  File "/usr/local/autotest/common_lib/test.py", line 598, in _exec
    _cherry_pick_call(self.initialize, *args, **dargs)
  File "/usr/local/autotest/common_lib/test.py", line 746, in _cherry_pick_call
    return func(*p_args, **p_dargs)
  File "/usr/local/autotest/cros/graphics/graphics_utils.py", line 83, in initialize
    self._player.find_connected_inputs()
  File "/usr/local/autotest/cros/input_playback/input_playback.py", line 402, in find_connected_inputs
    properties = self._find_device_properties(event)
  File "/usr/local/autotest/cros/input_playback/input_playback.py", line 197, in _find_device_properties
    utils.poll_for_condition(find_exit)
  File "/usr/local/autotest/common_lib/utils.py", line 2757, in poll_for_condition
    raise TimeoutError(desc)
TimeoutError: Timed out waiting for unnamed condition

LOG: https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/174990937-chromeos-test

one of the failed build#929: https://uberchromegw.corp.google.com/i/chromeos.chrome/builders/caroline-tot-chrome-pfq-informational/builds/929
 
This does not appear to be a specific test failure as it occurs across a range of tests:
graphics_Gbm
graphics_Idle.arc
cheets_PlayStoreTest
graphics_Drm.bvt
cheets_NotificationTest
cheets_KeyboardTest

It also occurs across a range of devices. Counts from the last half dozen failures:

11 chromeos6-row2-rack23-host11/
10 chromeos6-row2-rack23-host6/
5 chromeos6-row2-rack23-host9/
1 chromeos6-row2-rack23-host13/
3 chromeos6-row2-rack23-host15/
1 chromeos6-row2-rack23-host17/

Looking at the suite details I don't see any obvious pattern, e.g.
https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/suiteDetails?suiteId=175110757

The failing code (which appears to be the same for all test failures) is called from here:

def find_connected_inputs(self):
        """Determine the nodes of all present input devices, if any.
        Cycle through all possible /dev/input/event* and find which ones
        are touchpads, touchscreens, mice, keyboards, etc.
        These nodes can be used for playback later.
        If the type of input is already emulated, prefer that device. Otherwise,
        prefer the last node found of that type (e.g. for multiple touchpads).
        Record the found devices in self.devices.

It looks like it relies on a timeout waiting for a tmp file to be written.

My guess is that a recent change is causing that to take longer or fail more frequently before. We should look for the first occurence and see if we can identify any suspicious changes.


Cc: abhishekbh@chromium.org shenghao@chromium.org adlr@chromium.org
The first PFQ failure with this was @ R66-10363.0.0-b897, chrome @ 533716
https://uberchromegw.corp.google.com/i/chromeos.chrome/builders/caroline-tot-chrome-pfq-informational/builds/897

The first paladin failure wash @R66-10365.0.0-rc1, chrome 66.0.3329.0
https://luci-milo.appspot.com/buildbot/chromeos/caroline-paladin/2541

Given that this is flaky the correlation is pretty close, but it may have been introduce earlier than R66-10363.0.0-b897, making this challenging to track down.

That said, Chrome 66.0.3329.0 first made it through the PFQ @R66-10334.0.0, a week prior to when we started seeing these failures, so we can probably assume this was not a chrome issue.

+CrOS sheriffs

Does anyone know a straightforward way to get the CrOS changes leading up to R66-10363.0.0-b897 ?



Cc: cywang@chromium.org
+cywang@, +kathrelkeld@ for most recent-ish changes to client/cros/input_playback. Any idea what is actually failing here?

Cc: victorhsieh@chromium.org hashimoto@chromium.org xiaoyinh@chromium.org kazu@google.com
 Issue 809151  has been merged into this issue.
Owner: kathrelk...@chromium.org
Status: Assigned (was: Untriaged)
The failing part is when we're waiting for evtest to output info for a single input.  I'll dig into this one.
Thanks! At minimum it would be great to progagate up a better error than "unnamed condition" :)

On one of the failing machines, I found something odd: /dev/input/event4 exists but returns "Invalid argument" when passed to evtest.  This doesn't happen on other Caroline machines - and not all the time on this one machine.  Not sure why that is, but I've put up a CL to ignore this case.

https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/907949
Triage nag: This Chrome OS bug has an owner but no component. Please add a component so that this can be tracked by the relevant team.
<UI triage> Bug owners, please add the appropriate component to your bug. Thanks!

Sign in to add a comment