USB 3D Camera Experimentation

USB 3D Camera Module

USB 3D Camera Module (parallal SBS)

UPDATED: 2021/09/29

For experimentation with 3D video, I purchased a USB Dual Synchronized 3D Camera Webcam Module from Amazon:

https://www.amazon.com/gp/product/B07MJFTYZ3

The camera module is intended for surveillance, but my uses are 3D stereoscopic capture. I'll discuss the different approaches I took to get the camera module to record 3D photos and videos. 

Camera Module

The camera module generates 960p 3D Stereo HD SBS output at 2560x960 pixels for both video and still photos. Video is 60 fps. The resolution is 1280x960 per eye, slightly better than the FujiFilm W3 3D camera video at 1280x720. There is no microphone for audio recording with the Webcam module and the camera lens is fixed focus M9 90 degrees. The image sensor is 1/3 inch CMOS OV9750. 

The camera lenses interaxial distance is 60 mm. This interaxial generates pleasing 3D images when the subject distance ranges from 2 to 6 feet depending on the background distance behind the subject. 

My task was to write or find software that will operate the camera over a wired USB connection. My goal is to capture 3D still images and video.

Write Code

First I wrote a code sketch with the Processing Foundation https://www.processing.org IDE using the Processing Video library to control the camera. Using Video capture sample code I could see the camera in a list available to the code, but could not turn on or display the video stream from the module. The code reports an internal data stream error.  This code does work for other 2D USB Webcams that I experimented on.

Here is the code for you to try with 2D USB cameras:

// USB 3D Camera video capture

// Java mode Processing IDE version 3.5.4

// Processing Video Library 2.0

// Library uses Gstreamer Version 1.16.2

import processing.video.*;

Capture cam;

int index = 1;

void setup() {

  //size(640, 240);

  //size(1280, 480); // for laptop display

  size(2560, 960);  // for USB 3D Camera screen size on a 2160p laptop display

  //size(1920,1080);

  frameRate(30);

  String[] cameras = Capture.list();

  if (cameras.length == 0) {

    println("There are no cameras available for capture.");

    exit();

  } else {

    println("Available cameras:");

    for (int i = 0; i < cameras.length; i++) {

      println("index="+ i + " "+cameras[i]);

    }

    // The camera can be initialized directly using an 

    // element from the array returned by list():

    println("using index="+ index+ " camera "+cameras[index]);;

    println("width="+width + " height="+height);

    cam = new Capture(this, width, height, cameras[index]);

    //cam.frameRate(60);

    cam.start();

  }

}

void draw() { 

  image(cam, 0, 0);

}

void captureEvent(Capture c) {

  cam.read();

}

Write Code Success

I reached out to Processing Video Library Github participants and Lucas Fredericks suggested a code fix in green highlight below, that works with the USB 3D Camera:

// USB 3D Camera video capture

// Java mode Processing IDE version 3.5.4

// Processing Video Library 2.0

// Library uses Gstreamer Version 1.16.2

import processing.video.*;

Capture cam;

int index = 1;

void setup() {

  //size(640, 240);

  size(1280, 480);

  //size(2560, 960);  // screen size that will fit 2160p laptop display

  //size(1920,1080);

  //frameRate(30);

  String[] cameras = Capture.list();

  if (cameras.length == 0) {

    println("There are no cameras available for capture.");

    exit();

  } else {

    println("Available cameras:");

    for (int i = 0; i < cameras.length; i++) {

      println("index="+ i + " "+cameras[i]);

    }

    // The camera can be initialized directly using an 

    // element from the array returned by list():

    println("using index="+ index+ " camera "+cameras[index]);;

    println("width="+width + " height="+height);

    //cam = new Capture(this, width, height, cameras[index]);

    cam = new Capture(this, width, height, "pipeline: ksvideosrc device-index=1 ! image/jpeg, width=2560, height=960, framerate=60/1 ! jpegdec ! videoconvert");

    cam.start();

  }

}

void draw() { 

  image(cam, 0, 0);

}

void captureEvent(Capture c) {

  cam.read();

}

Zoom

I tried the Zoom app on Windows 10 and with the settings menu found the 3D camera and saw video. I did not try it in a Zoom session. The camera module generates a SBS video stream.

VLC Media Player

Next I used open source VLC Media Player (version 3.0.16) running on Windows 10. In the main menu I initialized with menu item Media, Open Capture Device and made the settings captured in the screen shot below.

A mouse right click starts a recording (red record button) and again to stop recording (gray stop button). I played back the video full screen SBS and viewed with a stereoscope. Now I have a way to record videos in 3D using the Webcam.

Android Apps

I want to use my Android phone with the camera module. After trying several apps without success, I discovered UsbWebCamera app by Serenegiant on the Google play store. It has a subscription payment featue that may put off users, but I tried this version anyway, before discovering the UsbWebCameraPro version that I recommend to use instead.

With the paid versions you can set the camera resolution which you cannot do in the free version. This feature makes it possible to use this camera.

To connect the USB 3D Camera to my phone (Samsung S20) I used an OTG cable micro-USB male to USC-C male (Host end) manufactured by Cable Creation on Amazon. It is about 8 inches long, just the right size for use with my phone. 

Later I tried a less powerful Samsung S8 phone, but video was not captured successfully using the subscription version of the app. When I switched to the Pro version on the S8 I could record video successfully.

The UsbWebCamera settings are the following:

• Resolution 2560x960 at 60 fps
• Audio settings: Audio source Disabled (Disabled, mic, or USB Audio Input). Set to mic to use the phone's microphone during video recording.
• Max video capture duration - using default 30 seconds
• Still capture mode - PNG for uncompressed still image
• Play shutter sound for photos

Camera Mount on a Phone Case

To use the camera with my phone I mounted in with rubber bands on the phone case. Using the volume up button I can capture either still photos or videos in 3D when in photo or video mode respectively.

Camera Mount on the OWL Viewer

I mounted the camera module on the back side of London Stereoscopic Company OWL Viewer: Virtual Reality Kit magnetic holding plate. https://shop.londonstereo.com/VRKIT.html

Looking through the lens you can see the camera view in stereo.

OWL Viewer with mounted Camera module 

OWL Viewer with camera module and S8 phone

Photo Example

Photo from Camera unedited

Aligned and Edited Photo

Anaglyph Aligned Photo

Stereo Photo Maker Screenshot during Alignment

YouTube Example

Resting Cat

https://youtu.be/z4_WwwX1fKs  (no 3D metadata applied, you need a stereoscope to view)

https://youtu.be/s7NEPUrfZrY (added 3D metadata for viewing in anaglyph and 3D TVs)

To add 3D metadata I used the following code in a Windows batch file invoking ffmpeg.  See  https://www.ffmpeg.org/

ffmpeg -i %1 -map 0 -c copy -metadata:s:v stereo_mode=left_right "%~n1.mkv"

ffmpeg -i "%~n1.mkv" -aspect 1280:960 -map 0 -c copy -strict -1 "%~n1_3D.mp4"

pause

Advantages

1. The cameras are synced and run at 60 fps. This solves synchronization and wiring issues using separate dual cameras in a 3D camera rig.
2. No microphone - use a higher quality external microphone for audio recording or use the phone microphone
3. Near human eye separation distance

Disadvantages

1. Low resolution, small sensor
2. Fixed lens, not interchangeable
3. With so many Android phone variations, your phone may not work with this camera module

Improvements

• Build a small enclosure to house the camera module and clip on a phone case.

Conclusion

For me this was worthwhile experimentation because I learned more about 3D video and cameras. I'll be looking for a camera module like this in 4K and better lenses.