New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 688861 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner: ----
Closed: Mar 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Feature



Sign in to add a comment

Support depth capture using Creative Senz3D

Project Member Reported by aleksand...@intel.com, Feb 5 2017

Issue description

Creating this bug to collect more information.
The camera is a bit rare/out of stock and I'm not able to find a way to purchase it.

Philipp reported the issue with the tweet here [1].

Philip, if you have time to help on this, appreciate it as purchasing the camera might prove difficult:
After installing driver how does the capture of depth device look here (please select Video source)?
https://webrtc.github.io/samples/src/content/devices/input-output/

Screenshot would help in identifying if it is splitting values to RGB or using unsupported 16/8 bit format.
Thanks.

Explanation:
We don't use external SDK here but use formats provided by driver. e.g. for 16-bit depth camera, format GUIDS and fourcc values are added as [2].

By grepping the code around, found that depth format provided Senz3D is not using 16-bit format but YUY2 and that needs to be interpretted from decoded RGB color as [3]:
"
Raw Depth - the 10 bit number is encoded with 8 bits in the red channel and the remaining 2 in the green channel. This allows you to access the full depth resolution in external applications by decoding the R and G channel. (Note the B channel is set to 0).
"

[1]
https://twitter.com/HCornflower/status/828279895327379459

[2]
https://cs.chromium.org/search/?q=kMediaSubTypeY16&type=cs

[3]
http://www.roborealm.com/help/Creative_Senz3D.php

 
Hi!

on Windows 10, it only shows up as an ordinary webcam even after installing the drivers.

On Linux, it shows up as two devices in enumerateDevices:
  {"deviceId":"someid","kind":"videoinput","label":"Creative Senz3D VF0780 (041e:4096)","groupId":""},  
  {"deviceId":"anotherid","kind":"videoinput","label":"Creative Senz3D VF0780 (041e:4096)","groupId":""}
The first is the normal webcam, the second (order seems to be random) returns a 640x240 video stream which is all green. This is semi-consistent with the 320x240 resolution quoted for the sensor.
uniformgreen.png
7.5 KB View Download
Hi.
Thanks for help.
Let me try to prepare a short binary / script for reading out the values on Linux. Green color means that format definition is missing, from Chromium (we could add it there) and potentially from kernel (in this case we cannot do much).

> Let me try to prepare a short binary / script for reading out the values on Linux
Found it. The output from this command should tell us how to implement it:

sudo uvcdynctrl --device=video<index of depth device> -f

Thanks.

*In details:*
sudo uvcdynctrl -l
Listing available devices: for me lists 3 devices. After calling following for video0 and video1, using video2 provides the information about the depth capture device and there is information about pixel formats; supported and unsupported.

sudo uvcdynctrl --device=video2 -f
Listing available frame formats for device video2:
Pixel format: YUYV (YUYV 4:2:2; MIME type: video/x-raw-yuv)
  Frame size: 640x480
    Frame rates: 30
Pixel format: INVZ (Depth 16-bit (INVZ))
  Frame size: 640x480
    Frame rates: 60, 30, 10
  Frame size: 640x240
    Frame rates: 110, 60, 30, 10
Pixel format: INZI (Depth:IR 16:8 24-bit (INZI))
  Frame size: 640x480
    Frame rates: 60, 30, 10
  Frame size: 640x240
    Frame rates: 110, 60, 30, 10
Pixel format:  (435a4e49-b602-480f-978c-e4e88ae)
  Frame size: 640x480
    Frame rates: 60, 30, 10
  Frame size: 640x240
    Frame rates: 110, 60, 30, 10
Pixel format: INVI (Infrared 8-bit (INVI))
  Frame size: 640x480
    Frame rates: 200, 120, 60, 30
Pixel format:  (52494150-8536-4841-b6bf-8fc6ffb)
  Frame size: 640x480
    Frame rates: 60, 30, 10
  Frame size: 640x240
    Frame rates: 110, 60, 30, 10

Comment 4 by fi...@appear.in, Feb 6 2017

video1 (which I assume is the cam-camera):
Listing available frame formats for device video1:
Pixel format: MJPG (Motion-JPEG; MIME type: image/jpeg)
  Frame size: 640x360
    Frame rates: 30, 25, 15
  Frame size: 640x480
    Frame rates: 30, 25, 15
  Frame size: 1280x720
    Frame rates: 30, 25, 15
Pixel format: YUYV (YUYV 4:2:2; MIME type: video/x-raw-yuv)
  Frame size: 160x120
    Frame rates: 30, 25, 15
  Frame size: 320x240
    Frame rates: 30, 25, 15
  Frame size: 640x360
    Frame rates: 30, 25, 15
  Frame size: 640x480
    Frame rates: 30, 25, 15

video2:
Pixel format: YUYV (YUYV 4:2:2; MIME type: video/x-raw-yuv)
  Frame size: 640x240
    Frame rates: 60, 50, 30, 25

Cc: aleksand...@intel.com
Owner: ----
Status: Available (was: Unconfirmed)
YUYV(YUY2) is already supported. The problem is that we don't know if there is custom unpacking that the vendor SDK is doing... or if there is a bug with YUYV/YUY2 implementation.
I am marking is as available for now... someone with the camera or with the newer version BlasterX Senz3D could try it meanwhile. Once I get the new camera I'll come back to this otherwise.

Philipp, thanks for help. Additional info might help too:

The output of this command:
v4l2-ctl --device=/dev/video2 --list-formats-ext


Comment 6 by fi...@appear.in, Feb 8 2017

doesn't seem to provide much more information:
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'YUYV'
	Name        : YUYV 4:2:2
		Size: Discrete 640x240
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.020s (50.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)

Status: WontFix (was: Available)
Received the new version: Creative BlasterX Senz3D. It works - it is based on RealSense SR300.
Marking as WontFix as there is no plan to support it.

Sign in to add a comment