New issue
Advanced search Search tips

Issue 778843 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Sep 24
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: Bug

Blocked on:
issue 888035



Sign in to add a comment

build249-m9 flaking on maps_pixel_test and other tests because of problem with KVM adapter

Project Member Reported by khushals...@chromium.org, Oct 26 2017

Issue description

Here's a couple of recent builds that failed.

https://build.chromium.org/p/chromium.gpu.fyi/builders/Mac%20Release%20%28Intel%29/builds/8386
https://build.chromium.org/p/chromium.gpu.fyi/builders/Mac%20Release%20%28Intel%29/builds/8388

And the trace from the failure:

/b/s/w/ir/third_party/catapult/telemetry/telemetry/testing/run_browser_tests.py:349
    context.test_class.SetUpProcess()
  SetUpProcess at /b/s/w/ir/content/test/gpu/gpu_tests/maps_integration_test.py:42
    color_profile_manager.ForceUntilExitSRGB()
  ForceUntilExitSRGB at /b/s/w/ir/content/test/gpu/gpu_tests/color_profile_manager.py:22
    color_profile_manager_mac.GetDisplaysToProfileURLMap()
  GetDisplaysToProfileURLMap at /b/s/w/ir/content/test/gpu/gpu_tests/color_profile_manager_mac.py:83
    device_id = device_info['DeviceID']
TypeError: 'NoneType' object has no attribute '__getitem__'

Looks like it is failing to get the color profile for the device. I've seen this happen a few times before.

ccameron@, assigning to you because colors. Would you be able to take a look?
 
Oop, I wrote that code, too....
Components: Internals>GPU>Testing Blink>WebGL
Labels: -Pri-1 Pri-2
I was going to close this out as not reproducible but it just happened again:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac%20FYI%20Release%20%28Intel%29/5130

Downgrading to P2, it's only happening 2/200 builds, but should still be investigated.

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac%20FYI%20Release%20%28Intel%29/4942

This is happening pretty often recently on Mac Debug (Intel). 7/200 builds.

3 more instances just today:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac%20Debug%20(Intel)/84124 maps_pixel_test
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac%20Debug%20(Intel)/84137 screenshot_sync_tests
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac%20Debug%20(Intel)/84161 maps_pixel_test
There were about 3 yesterday too, here's one:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac%20Debug%20%28Intel%29/84088 pixel_test

Note it's happening on maps_pixel_test, pixel_test, and screenshot_sync_tests.
Cc: ccameron@chromium.org
Components: -Blink>WebGL Infra>Labs
Owner: ----
Status: Available (was: Assigned)
Summary: build249-m9 flaking on maps_pixel_test and other tests because of problem with KVM adapter (was: maps_pixel_test flakes on Mac Release (Intel).)
All of these failures are happening on the same bot, build249-m9.
https://chromium-swarm.appspot.com/bot?id=build249-m9&sort_stats=total%3Adesc

I've taken it out of the Swarming pool.

Labs team: could you please examine build249-m9? Most likely its connection to the KVM is broken. Thanks.

Owner: jo...@chromium.org
Status: Assigned (was: Available)
Should we consider setting the system color space from within Chrome? (it's a bit more reliable and many fewer lines of code)
We could consider that. The reason we factored it out into a separate Python script was to be able to restore it reliably even if the browser crashed, but we found that changing it back caused test flakiness, so these bots are always setting the system color space and leaving it set that way. The browser could just as well do that.

On the other hand we probably want some sort of early warning if a bot can't set the system color space, because that probably indicates a hardware misconfiguration.


Blockedon: 888035
Apologies for the delay. Opened a request for hwops to check and verify the KVM interconnects.
Status: Fixed (was: Assigned)
Hwops re-seated connections. Verified KVM output, and the bot is back in swarming.

Thanks.

Sign in to add a comment