Video feed from Brio 4K camera is flickering on Jaq device |
|||||||||||||||
Issue descriptionChrome Version: 66.0.3359.10 /10452.1.0 Jaq OS: chrome 1.Not reproducible in other cameras in the same chromebook 2. Not reproducible using the same camera in Squawks, Chell, Clapper with same chrome version ***** This this is a combination of using only the specified camera and a specific chromebook. [as of now]***** What steps will reproduce the problem? Pre-condition: Connect a 4k camera [Logitech Brio 4K Stream Edition (046d:086b)] (1) Make sure 4k camera is selected in chrome://settings/content/camera (2) Make an Apprtc, meet or hangouts with another device [I used Macbook pro with stable chrome version 65.0.3325.146] What is the expected result? Video feed from 4k camera is good and free from any glitches. What happens instead? Video feed from 4k camera keeps on flickering. This is not a pleasant experience at all. Will be nice to get it fixed. Note: Same issue is Reproducible in 65.0.3325.148 / 10323.52.0, 64.0.3282.190 / 10176.76.0 Other observation: When doing an appear.in call video feed is blacked out rather than flickering.
,
Mar 12 2018
,
Mar 12 2018
Could you include chrome://media-internals video tab contents? Also does it happen using different resolutions? This should also happen on samples pages by the looks it since you are not connected in a call (you are seeing this in the raw video feed from the camera)
,
Mar 12 2018
Medial internals is attached in the link: https://cloud.google.com/console/storage/chromiumos-test-logs/bugfiles/cr/820961/ To capture the flickering in a big screen I did like that. The behavior is the same even after joining the webrtc calls. I echo your statement, this happened in the samples page first [Not exactly the same behavior]. To cross check I have checked in the live apps. In samples pages, video feed is blacked out completely. On live apps video feed is flickering.
,
Mar 12 2018
Same environment with QVGA seem to work quite good. No flickering is observed.
VGA: 2*2 video is generated. {this could be the reason why appear.in and apprtc call with the flag hd=false call is blacked out}
HD: Video flickers
Full HD: Video Flickers.
Does not support any later resolutions.
When a loop back calls are made, video flickering is even worse. [tried with vp9, vp8 and h264].
,
Mar 20 2018
,
Mar 23 2018
,
Mar 26 2018
,
Mar 28 2018
I can reproduce this on minnie and see the same tearing on camera preview. On other x86 3.14 device is camera does not work at all. Investigating if this is an issue with the 3.14 kernel....
,
Apr 10 2018
,
Apr 17 2018
Ping!
,
May 16 2018
any updates on this ticket?
,
May 16 2018
Sorry I didn't have cycle for this bug. I suspect this is an issue with the UVC kernel drivers in older kernels. To fix the issue we'll need to find out the potential missing CLs from upstream and backport them to 3.14 kernel.
,
May 16 2018
Thanks for the update. Feel free to contact if you need any help from me.
,
May 17 2018
Ricky, are you going to look at this or we should find someone else?
,
May 17 2018
Feel free to re-assign the bug. I probably won't have cycle for this one in the short term.
,
May 21 2018
,
May 23 2018
This problem is also reproduced on veyron_jerry in R68-10707.0.0 / 68.0.3437.0. I can see the same things as #5 about resolutions. Also, flickering occurs in native camera app too.
,
May 23 2018
Thanks Keiichi for looking at this. Assigning to you. :)
,
May 24 2018
By changing the value in /sys/module/uvcvideo/parameters/trace to 255, I got messages like the following when it's flickering: ... [ 2054.365312] uvcvideo: uvc_v4l2_poll [ 2054.396609] uvcvideo: Marking buffer as bad (error bit set). [ 2054.396621] uvcvideo: Marking buffer as bad (error bit set). [ 2054.396628] uvcvideo: Marking buffer as bad (error bit set). [ 2054.396633] uvcvideo: Marking buffer as bad (error bit set). [ 2054.396661] uvcvideo: Marking buffer as bad (error bit set). [ 2054.396667] uvcvideo: Frame complete (EOF found). [ 2054.396671] uvcvideo: EOF in empty payload. ... The message 'uvcvideo: Marking buffer as bad (error bit set).' comes from https://cs.corp.google.com/chromeos_public/src/third_party/kernel/v3.14/drivers/media/usb/uvc/uvc_video.c?l=989. It does not appear when the screen is not flickering, for example, when I use another camera or choose QVGA for resolution. In addition, I checked this 4K camera works well on Guado, which also uses 3.14 kernel.
,
May 25 2018
,
May 28 2018
When I added a line ctrl->dwMaxPayloadTransferSize = 1024; to https://cs.corp.google.com/chromeos_public/src/third_party/kernel/v3.18/drivers/media/usb/uvc/uvc_video.c?type=cs&g=0&l=240 , flickering didn't occur in HD camera capturing. In addition, when I added a line ctrl->dwMaxPayloadTransferSize = 512; to the same place, FullHD worked fine instead. (I tested both on https://webrtc.github.io/samples/src/content/getusermedia/resolution/ ) Thus, I guess this problem is related to a problem of this 4K camera's USB bandwidth. My guess may be supported by a result I reported in crbug.com/812529#c67 Some special handler for this device might be needed.
,
May 28 2018
I can think of some things to check further: 1) What's the original value of dwMaxPayloadTransferSize on this camera? 2) What's the value in case of some other cameras? 3) Where does the value originally come from? +Some of my trusted kernel folks, in case this makes more sense to anyone. ;)
,
May 29 2018
> 1) What's the original value of dwMaxPayloadTransferSize on this camera?
> 2) What's the value in case of some other cameras?
The original values of dwMaxPayloadTransferSize are:
| HD |FullHD |
Brio 4K | 2688 | 3072 |
Logitech C930e | 2688 | 3072 |
USB2.0 UVC HD Webcam | 3072 | 3072 |
"USB2.0 UVC HD Webcam" is the embedded webcam of jerry.
> 3) Where does the value originally come from?
The values come from camera devices.
They are obtained by ___uvc_query_ctrl with a query UVC_GET_CUR.
https://cs.corp.google.com/chromeos_public/src/third_party/kernel/v3.14/drivers/media/usb/uvc/uvc_video.c?l=174
Although uvc_video.c has a part computing dwMaxPayloadTransferSize (https://cs.corp.google.com/chromeos_public/src/third_party/kernel/v3.14/drivers/media/usb/uvc/uvc_video.c?l=154),
this part is not used for this camera because of UVC_FMT_FLAG_COMPRESSED flag in https://cs.corp.google.com/chromeos_public/src/third_party/kernel/v3.14/drivers/media/usb/uvc/uvc_video.c?l=122.
,
May 29 2018
https://cs.corp.google.com/chromeos_public/src/third_party/kernel/v3.14/drivers/media/usb/uvc/uvc_video.c?g=0&l=1608 AFAIU, this loop decides the best USB interface setting for transmitting data. |bandwidth| is the maximum bytes that the camera can transmit in one transfer. The USB interface setting is good if its |psize| (=max_bpi) is larger or equal to |bandwidth|. And, smaller psize is better. When Brio 4K is connected to jerry, |psize| of each setting is: 192, 382, 512, 640, 800, 944, 1280, 1600, 1984, 2688, 3072. When I select HD on resolution test, |bandwidth| is 2688. Then, 2688 is selected as |best_psize| here. This causes flickering. On the other hand, when I force |bandwidth| to be 1024 like #22, 1280 is selected. This setting does not cause flickering.
,
May 29 2018
I was kind of suspecting that Rockchip USB controller has problems with bigger packet sizes, but from #24, it seems like Logitech C930e also supports bigger sizes and it doesn't suffer from this issue. It looks like for some reason, only with this particular camera and this particular USB controller, bigger packets are broken.
,
May 29 2018
,
May 29 2018
The USB host controller in all veyron devices (dwc2) is a very inexpensive controller and requires software to do LOTS of the work on very tight deadlines (need to be able to handle interrupts in < 125 us). Also: historically the driver for dwc2 has been pretty buggy, though we've definitely made improvements there over the years. The first thing I'd try is to see if turning off DDRFreq and/or bumping up the CPU Freq helps. I'll try to come up with some commands for you shortly (my PC isn't responding to ssh right now), but basically: * Whenever we do a DDR Freq transition we'll certainly fail to service certain USB scenarios in time. Doing a DDR Freq transition on veyron involves disabling interrupts, sending an message to all other CPUs to tell them to stop (and waiting with interrupts disabled till they do), retraining DDR, then starting it all up again. Luckily USB is fairly robust, but with ISOC transfers we sometimes get drops. * We definitely have a much better chance of servicing USB interrupts in time if we're at a higher CPU frequency. In fact, at anything close to the top we almost never miss them. Unfortunately if the CPU isn't busy (except for the bursty servicing of interrupts), we might not put the CPU freq up high enough by default. ...we actually have some special case (read: hacky) code in the veryron kernel to make USB audio more robust. See <https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/302392>. If that's not getting triggered by your camera driver then we could add another hack like that. It's not pretty, but... --- If it's not a performance problem, then the next step would be to start digging through dwc2. There are also a few patches that have landed upstream we could try picking?
,
May 29 2018
Thanks Doug, those are some really good hints. We tried testing with cpufreq locked at max frequency, without any change in the symptoms, but maybe we could try again, to make sure we didn't miss something. We also haven't tried disabling DDR freq.
,
May 29 2018
@29: OK. Yeah, ddrfreq is actually a bigger deal than cpufreq. You'd set things here: cd /sys/class/devfreq/ff610000.dmc ...one word of warning is that I believe there's a bug where if you switch ddrfreq to userspace you can't set it back to the normal governor without rebooting (just a heads up). --- Skimming through the commits upstream, most are for gadget mode, but a shot in the dark could be: c3b22fe2e4cc usb: dwc2: Fix Control Write issue in DMA mode
,
May 30 2018
Thanks, Doug. I did some additional tests. When the resolution is HD, locking cpufreq at the maximum value made it a little bit better. The frequency of flickering was reduced, but not zero. In the case of full HD, it was still quite bad. For DDR freq, what I did are: - Chenge value in /sys/class/devfreq/ff610000.dmc/governor: 'simple_ondemand' -> 'userspace' - Change values in min_freq and max_freq to 666000000, which is the maximum value in available_frequencies But, I couldn't find any change on video flickering. > See <https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/302392>. This code was actually triggered in my case. So, this problem might be different from audio's one. > c3b22fe2e4cc usb: dwc2: Fix Control Write issue in DMA mode I applied this change, but this code was not executed in my case.
,
May 30 2018
> > See <https://chromium- > > review.googlesource.com/c/chromiumos/third_party/kernel/+/302392>. > This code was actually triggered in my case. > So, this problem might be different from audio's one. Makes sense, since webcams typically also include microphones over the USB audio class.
,
May 31 2018
I compared the behavior of BRIO in uvc_video with that of C930e. I found the following differences (though I'm not sure that they are helpful): - The numbers of payloads per one frame are different. It is around 44 for C930e, but around 260 For BRIO (when I choosed FullHD) I measured this number by counting continuous payloads that have the same fid (Frame Identifier), which is a bit toggled at each frame start boundary. I have no idea where does this difference come from. - Values of wCompQuality are different. 61 is used for C930e, and 0 for BRIO. wCompQuality is a field to specify the compression quality to cameras. I dumped this value in uvc_get_video_ctrl and uvc_set_video_ctrl. For C930e, this value ranges from 2 to 61. So, the maximum value is used. On the other hand, for BRIO, both of maximum/minimum value are 0. 0 is valid in USB device class specification, so I think this is not a problem. In addition, for C930e, some special handling is done in /drivers/usb/core/quirks.c. I added the same handling for BRIO, but nothing became better.
,
May 31 2018
Seems like we just have some debugging to do then. Two options would be: 1. dig into dwc2 and see if there's any traces that could help. 2. try to capture a USB trace with something like a Beagle USB analyzer. ...oh, I guess one last shot in the dark. Any chance <https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/321932> is relevant? At one point when I was poking at webcams on veyron I was tracing through the code and I found that it was taking a really long time in __usb_hcd_giveback_urb(), as measured by <http://crosreview.com/321980>. I found that I could hack it to be faster by decreasing the max number of UVC packets. If I remember correctly there was an issue where UVC was by default allocating memory that was could only be accessed in a super slow way. IIRC (it's been a few years) if I hacked uvc_video.c to use kmalloc() instead of usb_alloc_coherent() (AKA I tweaked the code around CONFIG_DMA_NONCOHERENT) then the speed problem went away... At the time I wasn't able to dig more... Anyway, possibly that might be relevant...
,
Jun 1 2018
I'm starting to think that this goes deeply beyond my understanding into the USB stack and probably also beyond what a human eye can understand without a USB analyzer. Do we have someone with USB analyzer (and USB experience) that could help us in debugging this?
,
Jun 1 2018
Thanks, Doug! You were right. The hack in https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/321932 fixed this problem. By changing UVC_MAX_PACKETS to 4, flickering went away both in HD and FullHD. As you mentioned, this hack made urb->complete faster. When UVC_MAX_PACKETS was 32, it took around 250-350us on average in FullHD resolution. Sometimes, it took over 500us. By decreasing UVC_MAX_PACKETS to 4, the time became about 130-150us.
,
Jun 1 2018
@35: Funny. Interestingly, if I were to think of the person who I would ask DMA memory allocation in the UVC video driver Tomasz would be the person who would jump to my mind (sorry Tomasz). ...so tag you're it I guess? ;-)
,
Jun 1 2018
Okay, maybe I was too fast with my previous comment. Great finding. It looks like it may be related to what Doug said about dwc2 earlier: > The USB host controller in all veyron devices (dwc2) is a very inexpensive > controller and requires software to do LOTS of the work on very tight > deadlines (need to be able to handle interrupts in < 125 us) If that's true, then the only things we could do that come to my mind are: - optimizing uvc driver's urb->complete, e.g. by trying to use cached (kmalloc?) memory, - have a way to limit the amount of data being transferred in one urb on a per-controller basis, - anything else?
,
Jun 1 2018
@38: I think you got the main two options. In theory I think you could also go into __usb_hcd_giveback_urb() and try taking out the local_irq_save() / local_irq_restore(). As per the comments there: /* * We disable local IRQs here avoid possible deadlock because * drivers may call spin_lock() to hold lock which might be * acquired in one hard interrupt handler. * * The local_irq_save()/local_irq_restore() around complete() * will be removed if current USB drivers have been cleaned up * and no one may trigger the above deadlock situation when * running complete() in tasklet. */ I haven't gone to check whether that comments still matters. It also seems like a bit of a scary change...
,
Jun 1 2018
I just found a FROMLIST series that moves most of the work from UVC urb->complete() into a work queue (we thought of this idea with Keiichi and then I suddenly recalled seeing such series...): https://www.mail-archive.com/linux-media@vger.kernel.org/msg128359.html (patchwork search https://patchwork.kernel.org/project/linux-media/list/?submitter=Kieran+Bingham&q=uvc) I guess it could significantly improve things.
,
Jun 4 2018
By applying the series of patches that Tomasz mentioned in #40, flickering went away on the resolution test. I created CLs for backporting (http://crosreview.com/1084420 and other 5 CLs in the relation chain.) However, in the native camera app, flickering still remains. The occurrence frequency of flickering became lower, but it's reproducable. So, I need some measurement and investigation.
,
Jun 26 2018
In addition to 6 patches mentioned in #40, I created a CL http://crosreview.com/1092619 to improve the execution speed of the interrupt handler. It makes the execution time of uvc_video_complete, which is a function called for each URB in interrupt handlers, 2x shorter.
,
Jun 26 2018
Keiichi, do you think we can land the FROMLIST CLs? I think I gave CR+2 to them. Please give V+1 and CQ+1 whenever you think we're okay.
,
Jun 26 2018
Tomasz, thank you for letting me know it. I just started to give CQ+1 to them. I think it takes some time because of dependencies among CLs.
,
Jun 29 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/2c7a65cdba60ae1adfc6e7861349b359e6a9ec64 commit 2c7a65cdba60ae1adfc6e7861349b359e6a9ec64 Author: Kieran Bingham <kieran.bingham@ideasonboard.com> Date: Fri Jun 29 08:08:41 2018 BACKPORT: FROMLIST: media: uvcvideo: Refactor URB descriptors We currently store three separate arrays for each URB reference we hold. Objectify the data needed to track URBs into a single uvc_urb structure, allowing better object management and tracking of the URB. All accesses to the data pointers through stream, are converted to use a uvc_urb pointer for consistency. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> (am from https://patchwork.kernel.org/patch/10311065/) Conflicts: drivers/media/usb/uvc/uvcvideo.h BUG= chromium:820961 TEST=compile Change-Id: I421d73abd4dac6036f78e76c9666112accf6c96f Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/1084420 Reviewed-by: Tomasz Figa <tfiga@chromium.org> [modify] https://crrev.com/2c7a65cdba60ae1adfc6e7861349b359e6a9ec64/drivers/media/usb/uvc/uvcvideo.h [modify] https://crrev.com/2c7a65cdba60ae1adfc6e7861349b359e6a9ec64/drivers/media/usb/uvc/uvc_video.c
,
Jun 29 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/f88a99ee99cec45b64beca2a597faf190cd5a820 commit f88a99ee99cec45b64beca2a597faf190cd5a820 Author: Kieran Bingham <kieran.bingham@ideasonboard.com> Date: Fri Jun 29 08:08:42 2018 BACKPORT: FROMLIST: media: uvcvideo: Convert decode functions to use new context structure The URB completion handlers currently reference the stream context. Now that each URB has its own context structure, convert the decode (and one encode) functions to utilise this context for URB management. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> (am from https://patchwork.kernel.org/patch/10311069/) Conflicts: drivers/media/usb/uvc/uvc_isight.c drivers/media/usb/uvc/uvc_video.c drivers/media/usb/uvc/uvcvideo.h BUG= chromium:820961 TEST=compile Change-Id: I51044ad0894ebb685d3fff60d1637ed0353f9548 Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/1084525 Reviewed-by: Tomasz Figa <tfiga@chromium.org> [modify] https://crrev.com/f88a99ee99cec45b64beca2a597faf190cd5a820/drivers/media/usb/uvc/uvcvideo.h [modify] https://crrev.com/f88a99ee99cec45b64beca2a597faf190cd5a820/drivers/media/usb/uvc/uvc_isight.c [modify] https://crrev.com/f88a99ee99cec45b64beca2a597faf190cd5a820/drivers/media/usb/uvc/uvc_video.c
,
Jun 29 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/da32908612b79c196315a061bdb031bc024215e8 commit da32908612b79c196315a061bdb031bc024215e8 Author: Kieran Bingham <kieran.bingham@ideasonboard.com> Date: Fri Jun 29 08:08:44 2018 BACKPORT: FROMLIST: media: uvcvideo: Protect queue internals with helper The URB completion operation obtains the current buffer by reading directly into the queue internal interface. Protect this queue abstraction by providing a helper uvc_queue_get_current_buffer() which can be used by both the decode task, and the uvc_queue_next_buffer() functions. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> (am from https://patchwork.kernel.org/patch/10311063/) Conflicts: drivers/media/usb/uvc/uvc_queue.c drivers/media/usb/uvc/uvc_video.c drivers/media/usb/uvc/uvcvideo.h BUG= chromium:820961 TEST=compile Change-Id: I4d42e4a6c948a291924429fc1abbdfff07da4771 Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/1084526 Reviewed-by: Tomasz Figa <tfiga@chromium.org> [modify] https://crrev.com/da32908612b79c196315a061bdb031bc024215e8/drivers/media/usb/uvc/uvcvideo.h [modify] https://crrev.com/da32908612b79c196315a061bdb031bc024215e8/drivers/media/usb/uvc/uvc_queue.c [modify] https://crrev.com/da32908612b79c196315a061bdb031bc024215e8/drivers/media/usb/uvc/uvc_video.c
,
Jun 29 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/d8d183db71787058fc3631ec94f1e1381fe61364 commit d8d183db71787058fc3631ec94f1e1381fe61364 Author: Kieran Bingham <kieran.bingham@ideasonboard.com> Date: Fri Jun 29 08:08:45 2018 BACKPORT: FROMLIST: media: uvcvideo: queue: Simplify spin-lock usage Both uvc_start_streaming(), and uvc_stop_streaming() are called from userspace context, with interrupts enabled. As such, they do not need to save the IRQ state, and can use spin_lock_irq() and spin_unlock_irq() respectively. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> (am from https://patchwork.kernel.org/patch/10311049/) Conflicts: drivers/media/usb/uvc/uvc_queue.c BUG= chromium:820961 TEST=compile Change-Id: Ie51deed3ac18d1f6d95fd816750848ed40b0f5e4 Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/1084527 Reviewed-by: Tomasz Figa <tfiga@chromium.org> [modify] https://crrev.com/d8d183db71787058fc3631ec94f1e1381fe61364/drivers/media/usb/uvc/uvc_queue.c
,
Jun 29 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/195452c99fc1a2b57cac8e04fe942c961f93248d commit 195452c99fc1a2b57cac8e04fe942c961f93248d Author: Kieran Bingham <kieran.bingham@ideasonboard.com> Date: Fri Jun 29 08:08:47 2018 BACKPORT: FROMLIST: media: uvcvideo: queue: Support asynchronous buffer handling The buffer queue interface currently operates sequentially, processing buffers after they have fully completed. In preparation for supporting parallel tasks operating on the buffers, we will need to support buffers being processed on multiple CPUs. Adapt the uvc_queue_next_buffer() such that a reference count tracks the active use of the buffer, returning the buffer to the VB2 stack at completion. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> (am from https://patchwork.kernel.org/patch/10311053/) Conflicts: drivers/media/usb/uvc/uvc_queue.c drivers/media/usb/uvc/uvcvideo.h BUG= chromium:820961 TEST=compile Change-Id: Ie2f3b3b4bf980e10c3bdbf3b65d93aa53e1521a7 Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/1084528 Reviewed-by: Tomasz Figa <tfiga@chromium.org> [modify] https://crrev.com/195452c99fc1a2b57cac8e04fe942c961f93248d/drivers/media/usb/uvc/uvcvideo.h [modify] https://crrev.com/195452c99fc1a2b57cac8e04fe942c961f93248d/drivers/media/usb/uvc/uvc_queue.c
,
Jun 29 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/7ccd6a2f66c490ab8652fc38dc7585997ea3ec2e commit 7ccd6a2f66c490ab8652fc38dc7585997ea3ec2e Author: Kieran Bingham <kieran.bingham@ideasonboard.com> Date: Fri Jun 29 08:08:48 2018 BACKPORT: FROMLIST: media: uvcvideo: Move decode processing to process context Newer high definition cameras, and cameras with multiple lenses such as the range of stereo-vision cameras now available have ever increasing data rates. The inclusion of a variable length packet header in URB packets mean that we must memcpy the frame data out to our destination 'manually'. This can result in data rates of up to 2 gigabits per second being processed. To improve efficiency, and maximise throughput, handle the URB decode processing through a work queue to move it from interrupt context, and allow multiple processors to work on URBs in parallel. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> (am from https://patchwork.kernel.org/patch/10311057/) Conflicts: drivers/media/usb/uvc/uvc_video.c drivers/media/usb/uvc/uvcvideo.h BUG= chromium:820961 TEST=Test FullHD resolution at https://webrtc.github.io/samples/src/content/getusermedia/resolution/ with (internal camera|Logitech Webcam C930e|Logitech Brio 4K) on (Jerry|Minnie|Samus) Change-Id: Iaa05f77feae3d006614424608fd9ded43e08bf11 Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/1084529 Reviewed-by: Tomasz Figa <tfiga@chromium.org> [modify] https://crrev.com/7ccd6a2f66c490ab8652fc38dc7585997ea3ec2e/drivers/media/usb/uvc/uvcvideo.h [modify] https://crrev.com/7ccd6a2f66c490ab8652fc38dc7585997ea3ec2e/drivers/media/usb/uvc/uvc_video.c
,
Jul 1
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/ac1a6a581b93aac38cc50d6e33e774f94a07959d commit ac1a6a581b93aac38cc50d6e33e774f94a07959d Author: Keiichi Watanabe <keiichiw@chromium.org> Date: Sun Jul 01 00:21:36 2018 BACKPORT: FROMLIST: media: uvcvideo: Cache URB header data before processing On some platforms with non-coherent DMA (e.g. ARM), USB drivers use uncached memory allocation methods. In such situations, it sometimes takes a long time to access URB buffers. This can be a cause of video flickering problems if a resolution is high and a USB controller has a very tight time limit. (e.g. dwc2) To avoid this problem, we copy header data from (uncached) URB buffer into (cached) local buffer. This change should make the elapsed time of the interrupt handler shorter on platforms with non-coherent DMA. We measured the elapsed time of each callback of uvc_video_complete without/with this patch while capturing Full HD video in https://webrtc.github.io/samples/src/content/getusermedia/resolution/. I tested it on the top of Kieran Bingham's Asynchronous UVC series https://www.mail-archive.com/linux-media@vger.kernel.org/msg128359.html. The test device was Jerry Chromebook (RK3288) with Logitech Brio 4K. I collected data for 5 seconds. (There were around 480 callbacks in this case.) The following result shows that this patch makes uvc_video_complete about 2x faster. | average | median | min | max | standard deviation w/o caching| 45319ns | 40250ns | 33834ns | 142625ns| 16611ns w/ caching| 20620ns | 19250ns | 12250ns | 56583ns | 6285ns In addition, we confirmed that this patch doesn't make it worse on coherent DMA architecture by performing the same measurements on a Broadwell Chromebox with the same camera. | average | median | min | max | standard deviation w/o caching| 21026ns | 21424ns | 12263ns | 23956ns | 1932ns w/ caching| 20728ns | 20398ns | 8922ns | 45120ns | 3368ns Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org> (am from https://patchwork.kernel.org/patch/10491117/) Conflicts: drivers/media/usb/uvc/uvc_video.c BUG= chromium:820961 TEST=Use native camera app with Brio 4K on jerry/guado TEST=Capture FullHD video on https://webrtc.github.io/samples/src/content/getusermedia/resolution/ with Logitech Brio 4K on jerry/guado Change-Id: Ia7c32fe1db0db834cd9d897d87b45eada583f146 Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/1092619 Commit-Ready: Tomasz Figa <tfiga@chromium.org> Reviewed-by: Douglas Anderson <dianders@chromium.org> Reviewed-by: Tomasz Figa <tfiga@chromium.org> [modify] https://crrev.com/ac1a6a581b93aac38cc50d6e33e774f94a07959d/drivers/media/usb/uvc/uvc_video.c
,
Jul 2
,
Jul 2
Just a note for anyone verifying this: The issue is related to hardware limitations of the USB controller on Veyron devices (dwc2) and it's not possible to eliminate the flickering completely. The fixes above made the flickering rate much lower and size of the corrupted frame part much smaller, to the point that it's barely noticeable. I think we can call it fixed, since the camera should be usable now. |
|||||||||||||||
►
Sign in to add a comment |
|||||||||||||||
Comment 1 by vasanthakumar@chromium.org
, Mar 12 2018