mash: Add autotest for starting chrome --mash in smoke suite HWTests |
|||||||||||
Issue descriptionWe've had bugs like issue 659804 that prevent mash from starting on a Chromebook. Most mustash developers use the Ozone X11 build on their linux desktops, but real hardware runs Ozone DRM and has IsRunningOnChromeOS true. We need a test that verifies that chrome --mash starts properly on a device. I'm going to investigate adding an autotest to our smoke suite that starts chrome with --mash and verifies that it starts without crashing. I'll aim to put it on one of the Chrome OS PFQ informational builders, which test ToT chrome. These are watched by the Chrome on Chrome OS gardeners and will provide ~3 hour regression ranges for Chrome CLs. Once we get that test running and stable we can look into propagating it to other bots. PFQ informational bots: https://uberchromegw.corp.google.com/i/chromeos.chrome/waterfall?reload=300
,
Nov 29 2016
Some progress here thanks to lots of in-person help from achuith. We're close to adding a test like the one in comment 1. This will start chrome with --mash and attempt login. This requires loading the OOBE screen when the browser first starts. I've got a fix for an OOBE startup CHECK failure in the CQ: https://codereview.chromium.org/2536113003/ When deploying a build of chrome with symbols (i.e. deploy_chrome with --nostrip) chrome starts more slowly. This might cause timeouts during tests. We had to add some sleeps in the catapult code initially, but were able to remove them after using a stripped binary. When running a test with extra_browser_args='--mash' you cannot also have a /etc/chrome_dev.conf file with --mash in it. This causes chrome to crash, see issue 669705 . With those changes we can successfully launch chrome and login. We can see the test launcher talking to chrome over devtools/JS. Cryptohome gets mounted. However the test still fails due to a problem with "SystemInfo.getInfo". On device: localhost chrome # pwd /usr/local/autotest/tests localhost tests # ../bin/autotest --verbose desktopui_StartChromeWithMash/control <snip> <lots of successful "sent" JS commands> <cryptohome gets mounted> <browser logs in, mash is running on the screen> <snip> 15:38:12 INFO | Browser is up! 15:38:12 INFO | OS: chromeos 15:38:12 DEBUG| sent [{ "id": 0, "method": "SystemInfo.getInfo" }] 15:38:17 ERROR| Failure while starting browser backend. Traceback (most recent call last): File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/browser/browser.py", line 56, in __init__ self._LogBrowserInfo() File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/browser/browser.py", line 114, in _LogBrowserInfo if self.supports_system_info: File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/browser/browser.py", line 307, in supports_system_info return self._browser_backend.supports_system_info File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py", line 271, in supports_system_info return self.GetSystemInfo() != None File "/usr/local/telemetry/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py", line 277, in GetSystemInfo return self._system_info_backend.GetSystemInfo() File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome/system_info_backend.py", line 25, in GetSystemInfo res = websocket.SyncRequest(req, timeout) File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 110, in SyncRequest res = self._Receive(timeout) File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 149, in _Receive data = self._socket.recv() File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket.py", line 596, in recv opcode, data = self.recv_data() File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket.py", line 606, in recv_data frame = self.recv_frame() File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket.py", line 637, in recv_frame self._frame_header = self._recv_strict(2) File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket.py", line 746, in _recv_strict bytes = self._recv(shortage) File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket.py", line 739, in _recv raise WebSocketConnectionClosedException() WebSocketConnectionClosedException Commenting out the call to self._LogBrowserInfo() makes the test pass. This is suspicious for something in chrome not correctly handling the request. I'm not sure what SystemInfo.getInfo() is doing, but that's what I'm going to look at next.
,
Nov 30 2016
Notes to self: 1. Stripped vs. unstripped is not an issue. The timeouts are not necessary -- after commenting out LogBrowserInfo() the test passes with an unstripped binary. 2. SystemInfo.getInfo appears to be implemented in generated devtools code. Unfortunately the generated files don't explain what generates them, but I think the implementation is here: https://cs.chromium.org/chromium/src/content/browser/devtools/protocol/system_info_handler.cc It seems to be asking the GPU process for model-specific information. I can imagine that might not work on mustash. Options: A) Figure out how to set Browser.supports_system_info false without calling GetSystemInfo, or B) Pass --gpu-no-complete-info-collection to skip this data collection (not sure if this is sufficient), or C) Fix mustash to allow getting this info from GPU
,
Nov 30 2016
Does the SystemInfo problem merit a separate tracking bug?
,
Nov 30 2016
Filed Issue 669965 mash: Cannot query GPU info with telemetry SystemInfo.getInfo, GPUDataManager observer not called
,
Nov 30 2016
,
Dec 1 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/0a30ff51f532b3bd281c7fdf668ca09f4b7b72f1 commit 0a30ff51f532b3bd281c7fdf668ca09f4b7b72f1 Author: James Cook <jamescook@chromium.org> Date: Wed Nov 30 20:34:15 2016 mash: Add desktopui_MashLogin autotest This tests that chrome --mash starts up, reaches the first OOBE screen, and can log in. The purpose is to catch startup crashes specific to mash and the Ozone DRM graphics backend (which is only used on real hardware). This test starts on the bvt-perbuild suite. I will move it to bvt-cq once we prove it is stable. BUG= chromium:669243 TEST=desktopui_MashLogin Change-Id: I4c2a2035d5a3fc92b12556ade35d536b5671fe93 Reviewed-on: https://chromium-review.googlesource.com/415267 Commit-Ready: James Cook <jamescook@chromium.org> Tested-by: James Cook <jamescook@chromium.org> Reviewed-by: James Cook <jamescook@chromium.org> [add] https://crrev.com/0a30ff51f532b3bd281c7fdf668ca09f4b7b72f1/client/site_tests/desktopui_MashLogin/control [add] https://crrev.com/0a30ff51f532b3bd281c7fdf668ca09f4b7b72f1/client/site_tests/desktopui_MashLogin/desktopui_MashLogin.py
,
Dec 1 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/76c19cf522a8ed07e625269930f3b04f6debec0f commit 76c19cf522a8ed07e625269930f3b04f6debec0f Author: James Cook <jamescook@chromium.org> Date: Wed Nov 30 21:07:59 2016 mash: Add desktopui_MashLogin autotest This tests that chrome --mash starts up, reaches the first OOBE screen, and can log in. The purpose is to catch startup crashes specific to mash and the Ozone DRM graphics backend (which is only used on real hardware). This test starts on the bvt-perbuild suite. I will move it to bvt-cq once we prove it is stable. BUG= chromium:669243 TEST=desktopui_MashLogin CQ-DEPEND=CL:415222 Change-Id: Ic58c1c66e7365e2a9f4049e0325ba31597edbb86 Reviewed-on: https://chromium-review.googlesource.com/415222 Commit-Ready: James Cook <jamescook@chromium.org> Tested-by: James Cook <jamescook@chromium.org> Reviewed-by: Achuith Bhandarkar <achuith@chromium.org> Reviewed-by: James Cook <jamescook@chromium.org> [modify] https://crrev.com/76c19cf522a8ed07e625269930f3b04f6debec0f/chromeos-base/autotest-chrome/autotest-chrome-9999.ebuild
,
Dec 5 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/498cf62596b579862eac02e27bb85d734b51bbeb commit 498cf62596b579862eac02e27bb85d734b51bbeb Author: jamescook <jamescook@chromium.org> Date: Mon Dec 05 23:34:39 2016 Don't CHECK in service manager when loading service from disk package This was causing CHECK failures on chrome --mash startup when running official builds. The StartWithFilePath() flow redirects through a delegate in the chrome_mash case, so this code is OK in production for chrome_mash. rockot@ is going to look into how to disable this code path for content but allow it for chrome_mash. BUG= 669243 ,649673 TEST=chrome --mash starts up in official builds Review-Url: https://codereview.chromium.org/2552873002 Cr-Commit-Position: refs/heads/master@{#436451} [modify] https://crrev.com/498cf62596b579862eac02e27bb85d734b51bbeb/services/service_manager/service_manager.cc
,
Dec 13 2016
,
Jan 9 2017
Test is in bvt-cq now.
,
Mar 4 2017
,
Apr 17 2017
,
May 30 2017
,
Aug 1 2017
,
Oct 14 2017
,
Feb 26 2018
|
|||||||||||
►
Sign in to add a comment |
|||||||||||
Comment 1 by achuith@chromium.org
, Nov 28 2016