crosvm fails to start on grunt: ForkingWhileMultiThreaded |
||||||||
Issue descriptioncrosvm can be run manually on grunt, but when invoked via concierge it fails to start: https://stainless.corp.google.com/search?exclude_cts=true&exclude_non_release=true&board=%5Egrunt%24&test=%5Etast%5C.vm%5C.&view=matrix&col=build&row=test&first_date=2018-08-01&last_date=2018-08-07 The error thrown is: 2018-08-07T15:56:16.332439-07:00 INFO localhos[18141]: crosvm[4]: [src/linux.rs:1012] crosvm entering multiprocess mode 2018-08-07T15:56:16.374346-07:00 ERR localhos[18141]: crosvm[4]: [src/main.rs:464] error registering block device: ProxyDeviceCreation(ForkingJail(ForkingWhileMultiThreaded)) 2018-08-07T15:56:46.326921-07:00 ERR vm_concierge[18113]: VM failed to start in 30 seconds ...which shouldn't happen.
,
Aug 27
ick. Is the thread creation in minigbm avoidable? Or do we need to rework the jailing sequence?
,
Aug 27
To avoid the thread creation, we would either have to adjust minigbm or adjust the amdgpu dri driver. Ideally minigbm should not have depended on dri, but I recall this being the worse of two evils. Adding gurchetan, who might be able to give us context on how feasible it would be too avoid thread creating in minigbm.
,
Aug 27
Can you trace where is the new thread being created? I assume it's when creating a new context: https://chromium.googlesource.com/chromiumos/platform/minigbm/+/master/dri.c#148 https://cgit.freedesktop.org/mesa/mesa/tree/src/gallium/state_trackers/dri/dri_context.c#n193 If that's the case, we may be able to disable this.
,
Aug 29
,
Aug 29
Grunt launches on M70, so we really need this one fixed ASAP unless we want to disable it for launch.
,
Aug 29
FYI here's a completed untested change that disables new thread creation when creating a new context: crrev.com/c/1194659
,
Sep 10
update?
,
Oct 1
,
Oct 1
Issue 890929 has been merged into this issue.
,
Oct 3
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/e56bba39be7005599cef6acd0c78496f30a0a80b commit e56bba39be7005599cef6acd0c78496f30a0a80b Author: Zach Reizner <zachr@google.com> Date: Wed Oct 03 20:59:14 2018 vm_host_tools: add crosvm_wl_dmabuf to IUSE This USE flag is used in an #if statement in concierge. BUG= chromium:872064 TEST=emerge-grunt vm_host_tools Change-Id: Ic7298649288f0623e3def47c4d064987a4ee27d9 Reviewed-on: https://chromium-review.googlesource.com/1257594 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Chirantan Ekbote <chirantan@chromium.org> [modify] https://crrev.com/e56bba39be7005599cef6acd0c78496f30a0a80b/chromeos-base/vm_host_tools/vm_host_tools-9999.ebuild
,
Oct 6
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/04a2b3dcaf1abdd105b8f1539e311ac321e28a7d commit 04a2b3dcaf1abdd105b8f1539e311ac321e28a7d Author: Zach Reizner <zachr@google.com> Date: Sat Oct 06 02:37:58 2018 grunt: disable crosvm-wl-dmabuf USE flag The multiprocess mode of crosvm does not work when it uses minigbm on radeonsi cards (such as on grunt), which spawns a background thread. The background thread trips a protective assert when crosvm forks device processes which behave badly when in multithreaded processes. BUG= chromium:872064 TEST=on grunt: vmc start termina Change-Id: I12c6fa63a7e9064215bfd93b7d8992b1dd1bb527 Reviewed-on: https://chromium-review.googlesource.com/1257650 Commit-Ready: Zach Reizner <zachr@chromium.org> Tested-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Drew Davenport <ddavenport@chromium.org> Reviewed-by: Daniel Kurtz <djkurtz@chromium.org> [modify] https://crrev.com/04a2b3dcaf1abdd105b8f1539e311ac321e28a7d/overlay-grunt/profiles/base/make.defaults
,
Oct 13
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/aff70f4b3c6de6496aa5639c598e7907297ee30f commit aff70f4b3c6de6496aa5639c598e7907297ee30f Author: Chirantan Ekbote <chirantan@chromium.org> Date: Sat Oct 13 06:07:15 2018 vm_host_tools: Enable crosvm_wl_dmabuf by default Without this flag, concierge will not enable wayland dmabufs by default, regressing graphics performance by requiring lots of buffer copying. BUG= chromium:872064 TEST="vmc start termina" and see that crosvm has the --wayland-dmabuf flag set Change-Id: Icca6f6985b2cca5163cc422934d262ddc91734e5 Signed-off-by: Chirantan Ekbote <chirantan@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/1279267 Reviewed-by: Zach Reizner <zachr@chromium.org> [modify] https://crrev.com/aff70f4b3c6de6496aa5639c598e7907297ee30f/chromeos-base/vm_host_tools/vm_host_tools-9999.ebuild
,
Oct 15
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/3b44ab29e878cf5558c13c068ddb5d7bfd0f3947 commit 3b44ab29e878cf5558c13c068ddb5d7bfd0f3947 Author: Zach Reizner <zachr@google.com> Date: Mon Oct 15 00:50:25 2018 grunt: disable crosvm-wl-dmabuf USE flag The multiprocess mode of crosvm does not work when it uses minigbm on radeonsi cards (such as on grunt), which spawns a background thread. The background thread trips a protective assert when crosvm forks device processes which behave badly when in multithreaded processes. BUG= chromium:872064 TEST=on grunt: vmc start termina Change-Id: I12c6fa63a7e9064215bfd93b7d8992b1dd1bb527 Reviewed-on: https://chromium-review.googlesource.com/1257650 Commit-Ready: Zach Reizner <zachr@chromium.org> Tested-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Drew Davenport <ddavenport@chromium.org> Reviewed-by: Daniel Kurtz <djkurtz@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/c/1279933 Reviewed-by: Martin Roth <martinroth@chromium.org> Tested-by: Martin Roth <martinroth@chromium.org> [modify] https://crrev.com/3b44ab29e878cf5558c13c068ddb5d7bfd0f3947/overlay-grunt/profiles/base/make.defaults
,
Oct 15
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform2/+/10afe0c2942c7032a0cf8f3888f849b76f48c816 commit 10afe0c2942c7032a0cf8f3888f849b76f48c816 Author: Zach Reizner <zachr@google.com> Date: Mon Oct 15 23:01:56 2018 vm_tools: concierge: check for crosvm-wl-dmabuf use flag If the crosvm-wl-dmabuf USE flag is unset, crosvm will not recognize the wayland-dmabuf argument, causing it to fail. This change removes that argument from the crosvm invocation in the absense of the crosvm-wl-dmabuf USE flag to prevent that failure. BUG= chromium:872064 TEST=on grunt: vmc start termina CQ-DEPEND=CL:1257594 Change-Id: Ie450133c3731d6d0c6cc6a7bfb035377411887d4 Reviewed-on: https://chromium-review.googlesource.com/1257614 Commit-Ready: Zach Reizner <zachr@chromium.org> Tested-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Chirantan Ekbote <chirantan@chromium.org> [modify] https://crrev.com/10afe0c2942c7032a0cf8f3888f849b76f48c816/vm_tools/host.gypi [modify] https://crrev.com/10afe0c2942c7032a0cf8f3888f849b76f48c816/vm_tools/concierge/virtual_machine.cc [modify] https://crrev.com/10afe0c2942c7032a0cf8f3888f849b76f48c816/common-mk/common.gypi
,
Oct 19
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/4ef106fc61f412045df549a4a9bd068189e955e8 commit 4ef106fc61f412045df549a4a9bd068189e955e8 Author: Stephen Barber <smbarber@chromium.org> Date: Fri Oct 19 05:38:02 2018 vm_host_tools: fix crosvm-wl-dmabuf USE flag The USE flag should match the one in crosvm's ebuild. BUG= chromium:872064 TEST=crosvm starts on grunt Change-Id: I9e94e4d94bd033a3349009528192479e76316a82 Reviewed-on: https://chromium-review.googlesource.com/1287769 Commit-Ready: Stephen Barber <smbarber@chromium.org> Tested-by: Stephen Barber <smbarber@chromium.org> Reviewed-by: Dylan Reid <dgreid@chromium.org> Reviewed-by: Daniel Verkamp <dverkamp@chromium.org> [modify] https://crrev.com/4ef106fc61f412045df549a4a9bd068189e955e8/chromeos-base/vm_host_tools/vm_host_tools-9999.ebuild
,
Oct 19
Discussed offline, vm.CrostiniStartEverything is failing since sommelier will still try to use dmabuf for wayland apps. https://stainless.corp.google.com/browse/chromeos-autotest-results/249822248-chromeos-test/
,
Oct 26
Ping. I think we need a sommelier fix here.
,
Nov 1
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform2/+/f797eeec7b10cd81057719050e6f6fbfd3e16bcd commit f797eeec7b10cd81057719050e6f6fbfd3e16bcd Author: Zach Reizner <zachr@google.com> Date: Thu Nov 01 04:59:38 2018 vm_tools: sommelier: check that virtwl-dmabuf works, fallback to virtwl On some hosts, dmabufs over virti-wayland are not supported, but sommelier is compiled with the virtwl-dmabuf shm-driver as the default on all termina images. In order to gracefully handle situtations where the default will not work, this CL adds a quick check that VIRTWL_IOCTL_NEW_DMABUF does not return ENOTTY before using virtwl-dmabuf. If that ioctl does return ENOTTY, sommelier falls back to using virtwl shm-driver. BUG= chromium:872064 TEST=wayland_demo Change-Id: Ie5ddf575ba9995aa6e4e102c1ce2102cd2c935bd Reviewed-on: https://chromium-review.googlesource.com/1306905 Commit-Ready: Zach Reizner <zachr@chromium.org> Tested-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Stephen Barber <smbarber@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/f797eeec7b10cd81057719050e6f6fbfd3e16bcd/vm_tools/sommelier/sommelier.c
,
Jan 8
Observing this issue on 11151.100.0, 71.0.3578.120(FSI for Grunt). As M71 is already moved to stable channel, removing M71 label and adding M72 & M73. Please feel free to change milestone labels if anyone feel otherwise. Thanks!
,
Jan 8
|
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by smbar...@chromium.org
, Aug 25This appears to be minigbm calling into dri_init for wayland-dmabuf. strace of crosvm: ioctl(11, DRM_IOCTL_AMDGPU_INFO or DRM_IOCTL_SIS_FB_FREE, 0x7ffe575d8100) = 0 ioctl(10, DRM_IOCTL_GET_CAP, 0x7ffe575d80f8) = 0 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [CHLD], 8) = 0 mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f2335a0f000 mprotect(0x7f2335a0f000, 4096, PROT_NONE) = 0 clone(child_stack=0x7f233620eef0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f233620f9d0, tls=0x7f233620f700, child_tidptr=0x7f233620f9d0) = 6 rt_sigprocmask(SIG_SETMASK, [CHLD], NULL, 8) = 0 futex(0x7f233811a840, FUTEX_WAKE_PRIVATE, 2147483647) = 0 uname({sysname="Linux", nodename="localhost", ...}) = 0 stat("/usr/lib64/dri/radeonsi_dri.so", {st_mode=S_IFREG|0755, st_size=30148424, ...}) = 0