Analysis:
1. Service main thread receives connection to device closed event.
2. Service main thread calls media::VideoCaptureDevice::StopAndDeAllocate()
3. The implementation of that posts to its internal v4l2_thread and blocks 
   on the thread join. This means the service main thread is now blocked.
4. The task on the v4l2 thread eventually runs the destructor of 
   VideoCaptureGpuJpegDecoderImpl, which posts a task to the service main thread and blocks on the completion of that task.
=> Deadlock, because that task is never run, because the service main thread is 
   blocked at step 3.

The underlying root cause of this is a design issue.

The destructor VideoCaptureJpegDecoderImpl::~VideoCaptureJpegDecoderImpl() blocks the calling thread in order to guarantee that no more frame callbacks are sent on the "decoder thread" after the destructor returns.

Method VideoCaptureDeviceLinux::StopAndDeAllocate() also blocks the calling thread in order to guarantee that no more frame callbacks are sent on the "v4l2 thread" after the destructor returns.

In the non service case, this happens to work out okay, because the "decoder thread" (Browser::IO) happens to be different from the thread that calls StopAndDeAllocate() (Device thread aka. Audio thread). But in the service case, both are the same, i.e. the main service thread.

IMO, blocking threads is a really bad idea, because it creates issues like this, where there are requirements on threading that are difficult to anticipate and for which there is no place or contract to document them. The clean solution here would be to replace the blocking methods/destructors with asynchronous shutdown methods that generate an event when it can be guaranteed that no more frame callbacks are sent.

Unfortunately, changing the design from blocking synchronous methods to asynchronous methods with an event to wait on, may require changing the design through the whole call stack, which can be a big undertaking. Also there is not necessarily consensus among developers that this is the right approach, see e.g. discussion on https://chromium-review.googlesource.com/c/chromium/src/+/533681.

The only practical solution here seems to be to spin up a new thread either for the calls to media::VideoCaptureDevice (i.e. a "device thread", but this would be a lot of work), or to serve as the "decoder thread", which is probably much less work. I will prepare a CL for the latter.

Note: This was not caught and is currently not verifiable by our integration tests using the fake capture device, because unlike most actual device implementations, the fake capture device does not create its own thread.

We need to update the fake capture device to use its own thread, maybe make it configurable. This way we can get coverage for such threading issues.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3a4ffadf90e0d4c523816a69fe740e522a4b3a56

commit 3a4ffadf90e0d4c523816a69fe740e522a4b3a56
Author: Christian Fremerey <chfremer@chromium.org>
Date: Wed Jun 20 20:15:38 2018

[Video Capture Service] Operate jpeg decoder IO to gpu process on separate thread

Adds a dedicated thread to the video capture service for operating the IO
between the capture device and the gpu process for acceleraged Mjpeg decoding.

This fixes a deadlock during shutdown of video capture in the video capture
service after a session that uses the accelerated jpeg decoder. See the crbug
for details.

The deadlock and the fix for it can currently only be provoked/verified manually
by running a ChromeOS device with a camera attached and command-line flag
--enable-features=MojoVideoCapture. Getting coverage in automated integration
testing is possible but requires modification of the fake video capture device
implementation, see crbug/852606.

Bug:  852606 , 820608
Change-Id: Id9a57e386d2a741f15fd1de72539f4a9885624da
Reviewed-on: https://chromium-review.googlesource.com/1102878
Reviewed-by: Dan Sanders <sandersd@chromium.org>
Reviewed-by: Emircan Uysaler <emircan@chromium.org>
Commit-Queue: Christian Fremerey <chfremer@chromium.org>
Cr-Commit-Position: refs/heads/master@{#568966}
[modify] https://crrev.com/3a4ffadf90e0d4c523816a69fe740e522a4b3a56/media/capture/video/video_capture_jpeg_decoder_impl.cc
[modify] https://crrev.com/3a4ffadf90e0d4c523816a69fe740e522a4b3a56/media/capture/video/video_capture_jpeg_decoder_impl.h
[modify] https://crrev.com/3a4ffadf90e0d4c523816a69fe740e522a4b3a56/media/mojo/clients/mojo_jpeg_decode_accelerator.cc
[modify] https://crrev.com/3a4ffadf90e0d4c523816a69fe740e522a4b3a56/media/mojo/clients/mojo_jpeg_decode_accelerator.h
[modify] https://crrev.com/3a4ffadf90e0d4c523816a69fe740e522a4b3a56/services/video_capture/device_factory_media_to_mojo_adapter.cc
[modify] https://crrev.com/3a4ffadf90e0d4c523816a69fe740e522a4b3a56/services/video_capture/device_factory_media_to_mojo_adapter.h
[modify] https://crrev.com/3a4ffadf90e0d4c523816a69fe740e522a4b3a56/services/video_capture/device_factory_provider_impl.cc
[modify] https://crrev.com/3a4ffadf90e0d4c523816a69fe740e522a4b3a56/services/video_capture/device_factory_provider_impl.h
[modify] https://crrev.com/3a4ffadf90e0d4c523816a69fe740e522a4b3a56/services/video_capture/device_media_to_mojo_adapter.cc
[modify] https://crrev.com/3a4ffadf90e0d4c523816a69fe740e522a4b3a56/services/video_capture/device_media_to_mojo_adapter.h
[modify] https://crrev.com/3a4ffadf90e0d4c523816a69fe740e522a4b3a56/services/video_capture/device_media_to_mojo_adapter_unittest.cc
[modify] https://crrev.com/3a4ffadf90e0d4c523816a69fe740e522a4b3a56/services/video_capture/test/mock_device_test.cc