The failure you're seeing is due to the maximum number of memory slots in x86 kvm: https://elixir.bootlin.com/linux/v4.4.130/source/arch/x86/include/asm/kvm_host.h#L38 not due to maximum number of FDs. The correct solution is not to limit the number of allocations but to limit the number of concurrent memory mappings at once, wsich should never be that high, even in egregiously badly written applications that make 600 cursors on start up.

Can we add better logging for when we reach this limit? A misbehaving VM can always trigger it.

My improvements to xwayland and sommelier will make it much less likely that we reach this limit but I also want to make sure that a misbehaving app can't cause other applications from working correctly.   

<triage>Is this fixed?</triage>

Nope. Logging is still pretty bad and the concurrent mapping limit has not been implemented.