3D Camera With Bluetooth Remote Control

Twin Smartphone 3D Camera, Bluetooth Remote Controller, and OWL viewer

Twin Smartphone 3D Camera Lens in OWL viewer

Twin Smartphone 3D Camera in OWL viewer

During the summer 2019, I went to my first National Stereoscopic Association convention, 3D-Con, in Akron, Ohio, USA. It was a great experience. I learned a lot about 3D stereo photography, saw wonderful 3D Art, photos, and videos. I met expert 3D enthusiasts who graciously shared their knowledge and Art. 

3D Camera Phone Rig

Before attending the convention I was experimenting with making a 3D camera rig using twin smartphones mounted with magnets. During the convention my ideas for this 3D camera project came together and I was able to complete a working system upon my return home.

Convention vendors sold all kinds of 3D related gear and Art. At the London Stereoscopic Company booth they sold the "OWL - Stereoscopic Viewer" for viewing stereo cards. I bought my OWL viewer on-line a couple of years ago, but I found out they now offer a Virtual Reality kit that uses a magnetic board for securing a phone to the viewer. I bought the magnet board holding plate from LSC to hold my twin phones in the OWL.

My 3D camera rig consists of two identical Samsung Galaxy S6 smartphones mounted in portrait mode, side by side, in the OWL viewer and held in place by the magnet. I placed a adhesive metal sheet bracket on each phone case to cling to the magnet. One of the S6 phones was used before I upgraded and the other I bought used on EBay. I cut two openings in my OWL viewer for charging cables and a hole for a 1/4 inch tripod mount.

In the OWL the phone camera lenses appear above the OWL frame unobstructed. Looking through the OWL lenses you can see in stereo the camera screen images. This turns out to be a good way to preview the depth of the scene you want to capture. The preview helps you determine if you are too close or too far from your subject besides helping with composition framing. The OWL rig mounted smart phone camera lens separation (inter-axial) is 75 mm in portrait mode. By coincidence the FujiFilm FinePix Real 3D W3 camera has the same lens separation distance. See the rig photos above.

Camera App

Open Camera Remote App Icon

I could have used the Samsung camera app that came with the phone, but I wanted a wireless trigger for both focus and shutter controls.

At the convention there was a discussion group about using smartphones to capture 3D. The Hyper3DPhone app was identified as a possible app for twin Android smartphone cameras. It works by pairing two phones over a Bluetooth connection and triggering one phone sends a message to the other to also take a picture. Although it worked well for triggering two phones nearly simultaneously, its weakness was that I had to touch the camera screen to take a photo and there was a lack of camera control features I use in other camera apps.

For several years I have been extending features of the open source Android Open Camera app to include wireless remote control. My modified app is named Open Camera Remote and you can install it from the Google Play Store . My app uses WiFi broadcast messages to trigger focus and shutter remotely on multiple phones connected to a local area network. Newer versions of the camera app permit control using wired and wireless Bluetooth keyboards, mice, and other game controllers. In addition I added a tiny HTTP Web Server in the app retrieve photos and videos taken by the camera and stored on the phone. This was my choice for a camera app in my 3D OWL rig.

Wireless Focus/Shutter Trigger

3D Camera in Owl Viewer with Bluetooth controller work in progress.

With this 3D camera rig I use two Bluetooth camera controllers. The Open Camera Remote app also allows shutter control over WiFi for which I would need access to a local private WiFi network, and a third phone to use as a remote control trigger. WiFi has a longer range advantage over Bluetooth. 

Bluetooth Camera Remote Shutter

I used two Bluetooth camera shutter remotes as pictured above for my DIY controller. I needed two, one for each smartphone, because two phones cannot share the same Bluetooth remote connection. I took the circuit boards out of the remote and wired them to switches and battery power in a case shown below. 

Home-made Bluetooth Remote for Twin Cameras

The remote control circuit boards were wired to trigger simultaneously with the push of a single button switch. Two diode OR circuits permit a single switch for focus and another OR circuit for shutter release to bypass the built-in controller switches. The controller's large button does focus (volume up, Android key code 24) and the smaller button does shutter release (enter, Android key code 66). The camera app settings have to be set to use the volume up switch for focus.

Synchronization of the cameras is just OK with variable results. Good synchronization of the two cameras depends mostly on how long it takes for the cameras to focus before taking a picture. To make sure focus does not interfere with synchronization, first I touch the screen to select my subject that the camera will focus on. When both cameras are in focus, then I press the shutter button to take a picture. I use the focus switch to focus repeatedly on the subject to make sure focus is achieved before pressing the shutter control switch.

I got better synchronization when I placed the camera apps in manual focus mode. But I found it too cumbersome to make sure the focus distance was the same for each camera.

Here is a video demonstrating the 3D Camera in operation.

Viewing 3D Using the OWL

After taking a picture you can view it in 3D with the OWL by tapping the right bottom corner preview icon on each phone to launch your photo viewer of choice. I use the app I wrote 3D/VR Stereo Photo Viewer in 2D mode for viewing my photos. Other photo viewing apps work just as well. The Back button on the phone returns to the camera app.

3D Smartphone Camera Tripod Mount in OWL 3D Viewer

Since writing this blog last October, it's now mid May, 2020, I have used the rig with twin Samsung Galaxy S8 phones that have a much better camera with improved results. I also experimented with the 3D Stereo Photo Viewer app running on a Rokit 3D Pro phone to retrieve photos from the twin phones for display on its glasses free 3D display screen. That's another future blog posting.

WiFi Remote Capture

Updated 2019-08-28

Introduction

WiFi Remote Capture app is a companion app to Open Camera Remote. It triggers photo or video recording capture for multiple compatible phone cameras simultaneously when Open Camera Remote is active. It emulates wired focus and shutter control cables connected to multiple cameras.

The app transmits capture commands to all the Open Camera Remote apps running on phones that are connected to the same local WiFi network. You can use this app for 3D stereo photography, multi-camera angle video shoots, multi-camera VR panoramic image capture, etc. The local network must support Broadcast messages.

At a social gathering, party, photo shoot, or other group event all the participants need to use the Open Camera Remote app to take photos or videos and all participants have to connect to the same local WiFi network. With the WiFi Remote Capture app you can then trigger all the participants cameras nearly simultaneously to take pictures at the same instant.

The app works with two cameras:

Open Camera Remote App (phone camera)

The Open Camera Remote app listens for broadcast message commands when connected to a local WiFi network. The Open Camera Remote app should be set in either photo or video recording mode before capture with that mode.

The camera app is available at https://play.google.com/store/apps/details?id=net.sourceforge.opencameraremote

Samsung NX500/NX1 Cameras

These cameras have to be modified with NXKS2 firmware that runs a broadcast receiver server. The app will send camera focus/shutter and video record/pause/stop commands using broadcast message commands over a connected local WiFi network. See https://github.com/ajavamind/NX500Capture for information about the camera firmware updates and setup you need to control your NX camera from this app.

Operation

The Focus Hold button can be pressed and released several times and will cause the Open Camera Remote app to show the network address assigned to the mobile phone. This is a good indication that BROADCAST messages are reaching the Open Camera Remote app.

The button at the top of the display that shows the next filename to send to connected apps. You can toggle the button to send either a time stamp or number for the filename. The time stamp selection is recommended.

You can change to video or photo modes by pressing the three dots in the upper right corner of the display.

Press the shutter button to send a focus and then press again to send the shutter trigger message.

Helpful Information

It is necessary that you close and restart the WiFi Remote Capture and Open Camera Remote apps after you change your WiFi connection in Settings for all your devices using these apps.

Camera Synchronization

Here are some tips for triggering multiple camera phones (Open Camera Remote app) nearly simultaneously. These changes are not all necessary, but give you a chance to more closely synchronize all your phone cameras at the same time.

1. Use a local WiFi router network not connected to the Internet.

2. In Google Chrome, log off your account to prevent background operations from slowing down network signaling and processor time used by the camera app. This step is not required.

3. In Settings Turn on Airplane mode. This will turn off all radios. Next in Settings turn on WiFi only. This will let you use WiFi without any connections to your mobile phone carrier.

4. Exit all applications. Next, optimize using Settings->Device Maintenance->Battery->Save Power to put to sleep background apps.

5. Start the Open Camera Remote app.

6. Use a separate phone or tablet running WiFi Remote Capture for triggering your multiple phones.

7. Make sure each camera phone can focus on the subject(s). If a camera phone has to hunt to get into focus, it will be delayed and not fire its shutter with the other phones. This means use manual or auto-focus, not continuous focus options in the Open Camera Remote app.

When finished taking photos all your photos, be sure to restore your previous Settings by turning off Airplane mode and logging back into your Google Chrome account.

Troubleshooting

The Pixel XL phone with Google WiFi assistant has this feature turned on by default. This phone feature adds extra WiFi security and VPN that blocks standard IP BROADCAST messages. Please turn off WiFi Assistant and retry.

Chromebook blocks BROADCAST messages.

Some Telecom supplied WiFi routers, for security reasons, block BROADCAST messages on the routers local network. In this case you must use your own local WiFi router. When BROADCAST messages are not passed through your router, there is no indication of a failure, so the apps will just not work and there is no feedback that this is a problem.

Your WiFi signal may be competing with other routers on the same channel, causing interference and dropped shutter focus/trigger messages. To avoid interference, you will need to reconfigure your router channel to a quiet 2.4 GHz channel or 5 GHz channels. A WiFi Analyzer app can help you find a quiet channel for your router. Please see your router documentation explaining how to change the default WiFi channel. Make sure your phones have a strong enough WiFi signal at each phone location and WiFi signals are not blocked by walls, etc.

Open Camera Remote

Last updated 2019_09_10
Previous update 2019_09_02

Introduction

Open Camera Remote is a free camera application for Android phones and tablets. It is a modified version of the full featured, open-source, free Open Camera application written by Mark Harman. For information about using the camera app features see Mark's website http://opencamera.org.uk/

Open Camera Remote adds features that give you remote control of your phone camera focus and shutter trigger over a local WiFi network. It does this by accepting broadcast control messages to operate focus and shutter release for the camera when sent by a separate companion application WiFi Remote Capture. The remote control camera feature is always on, waiting for a trigger, you only need to connect your phone to a compatible WiFi network.

The main goal of the Open Camera Remote app is to trigger multiple cameras and capture photos or videos nearly simultaneously from many phones running this app and all connected on the same local WiFi network. Depending on your local network configuration, the number of camera phones triggered can range from up to 253 or 4093 phones maximum.

Some example applications for the Open Camera Remote app are:

• 3D Stereo photography using twin phone cameras
• Multi-camera photos for 360 panorama and composite stitched shots
• Multi-camera photo capture for 3D modeling
• Multi-camera videos
• Multi-photo 3D Lenticular image capture
• Large group event or party activities for simultaneous photo or video capture
• Photo booths using remote trigger with single or multiple cameras
• Wild life photography where you cannot get close without disturbing your subject (for example a bird nest).
• Multi-phone camera single or group selfies or environment shots.

Features

Remote Control

The camera app receives focus and shutter release control messages broadcast over a local WiFi network sent by the WiFi Remote Capture app. The message contains the filename to use for storing the photo or video in each receiving camera phone app. The Remote Capture app can send either time-stamp or numbers for the filename.

To help identify photos and videos from each phone, especially if you merge photos into a folder, you should set a filename suffix. Go to the gear Settings - More camera controls... enter Save photo suffix, Save video suffix. The suffix string appended to the filename helps you identify from which phone camera a photo or video was shot. This feature allows you to merge all the photos from multiple cameras into a single folder for editing without filename conflicts.

I use _l and _r for stereo left and right camera suffixes. For other multiple camera setups, I use a sequence _1, _2, etc. You can choose your own multi-camera photo identification strategy.

Sequential 3D Stereo Photography

You can take 3D Stereoscopic left and right eye photos using the "chacha" sequential capture method. Like the dance first you take the left eye photo, move about 2.5 inches (65 mm) to the right and then take the right eye photo. In the gear Settings select the 3D ChaCha Mode. When selected this option shows a red line on the left in the camera preview. After you take the left photo, the red line appears on the right side of the preview screen. The left and right eye photos have the fixed suffixes "_l"  and "_r" appended to the corresponding photo filename, overriding the settings suffix entered. The photos are stored in the Save Location you choose in gear Settings - More Camera Controls.

To help you align the right eye photo vertically with you left eye photo, select the Settings - Align With Last Photo option. When set this option shows the left eye photo in the camera preview so you can align with your right eye photo preview. You can place your phone in a stationary folded V-shaped cardboard rail to help with vertical alignment by sliding the camera phone in a level channel.

You can view your 3D photos directly by purchasing a 3D/VR Stereo Photo Viewer app and select this app as your photo viewer using the photo preview thumbnail button. The 3D viewer app also works with 2D photos and many formats of 3D photos, and can be used with Google Cardboard VR headsets. 

The 3D/VR Stereo Photo Viewer app has an option to launch the WiFi Remote Capture app from the viewer so you can trigger your twin camera phone rig. You will need to install the WiFi Remote Capture app on your phone.

The Open Camera Remote app does not align your left and right eye photos. For this you will need Stereo Photo Maker that runs on Windows or Mac computer or you can install the 3DSteroid Android app.

Note that the ChaCha option is independent of the remote control feature, and could be used at the same time. But I don't know a situation you might want to used these combined features except when your phone cameras move horizontally in unison.
  

Number Format For Filename

I also added an option that lets you select a time-stamp or number for the filename with the Time/Number format for filename option. There is also an option to reset the photo and video filename number. This is not used with remote control but is for another way to name photo or video filenames, besides a time-stamp.

HTTP Server

The app has a built-in minimal Web HTTP Server to help with listing and transferring photos or videos to another phone or computer on the local network. See below for more details.

Mouse Control

With version 1.46.6 I added the capability to use a wired mouse (connected to the USB phone port using an OTG [on-the-go] cable or a connected Bluetooth wireless mouse) to control the camera shutter with a left button click. The right mouse button will display the last photo. The mouse wheel allows zoom control of the camera.

You would use a wired mouse to your phone to take photos with a single phone camera mounted on a tripod when you don't want to use the WiFi remote feature.

Setup

For security reasons some local network WiFi routers block broadcast messages. This prevents Open Camera Remote from working as intended. For example, my Telecom Internet service provider modem and router blocks broadcast messages in my home local WiFi network. Public WiFi networks outside my home usually block broadcast messages too. 

The solution is to set up and use your own private WiFi network router that allows broadcast messaging. A private local WiFi network not connected to the Internet also has an advantage of faster simultaneous camera control because network messages are not delayed by other Internet traffic. All you have to do is change each of your phone's WiFi connection to your alternative network.

I also recommend you place your phone in "Airplane Mode" to turn off all radios in the phone, and then selectively turn on only WiFi and make your connection. This step is not required but it helps to give you more simultaneous camera triggers by avoiding delays.

For my local network I bought a tiny home/mobile WiFi router, TP-Link 150 Mbps Wireless N Nano Router (model No. TL-WR702N). It can be powered with a battery and taken wherever I go to shoot multiple simultaneous photos or videos. I do not connect it to the Internet. I use the default "Access Point Mode" and set up my own SSID network name, "Mobile Photo Net", and password with a laptop computer.  

TP-Link also sells a faster router, TP-Link 290A (model No. TL-WR902AC version 3.0). With this router, you can transfer photos faster from your phones using the built-in Web HTTP server provided in the Open Camera Remote app.

Each router has a CAT5 Ethernet port so you could have an Internet connection if you wish. But the strategy for simultaneous capture is to reduce competing internet messages that slow down camera response time. You may also want to shutdown other apps for the same reason. And if you may want to log out of other services too, like email and messaging. However these steps are not necessary, just an option to consider.

WiFi Remote Capture

The free companion app WiFi Remote Capture sends broadcast camera control messages. It uses a simple protocol documented at https://github.com/ajavamind/NX500Capture and describes the command format. Besides phones this app can trigger the Samsung NX500 and NX1 cameras remotely on the local network. The Samsung cameras have to install a firmware hack.

The WiFi Remote Capture app has focus and shutter buttons and can switch between photo and video modes. Use the focus button to toggle the phone display to show its URL.

The WiFi Remote Capture app also accepts Bluetooth connected device volume up key so you can use an external controller instead of tapping the screen. See Twin NX500 Camera 3D Stereo Rig Using Remote Capture

Using a Linux computer you can also send broadcast messages using the Bash command line to trigger phones on your local network. For example:

echo "V0003" | nc -w1 -u -b 192.168.1.255 8000

will capture a phone video and save to VID_0003_1.mp4 

And you can write  Python or Processing code to do the same.

To test to see if you can broadcast messages on your local network successfully, press the Remote Capture app, Focus button, several times. You should see the  phone's IP address displayed on each screen. This confirms your remote trigger. If you do not, there is something wrong with your WiFi connection or broadcast messages are blocked.

HTTP Server

Web Browser

With the app's built-in Web HTTP server you can transfer your photos or videos with a Web browser. This only works with your local network while the camera app is running while visible or in the background. Use the HTTP server at your own risk. 

Alternatively you can transfer your photos or videos to another phone or computer using a separate FTP app or other methods supplied by the phone manufacturer, and other apps in the Google Play Store.

By default the server is disabled. In gear Settings there is a option to turn on the HTTP server. 

Use the gear Settings - Pairing QR Code option to show the URL of the phone Web server as a QR code or text. To see a directory of the photos or videos, type in the text URL into a Web browser search line from another computer or phone. You should see a listing of each file. Click on the filename to download and display in the Browser download folder. I use Google Chrome. Unfortunately at this time the transfers can only be done one photo or video at a time.

With the QR Code display in Settings - Pairing QR Code you can scan the URL to avoid typing using a free Barcode Scanner app by the ZXing team. The app will automatically show the directory in a browser.

Viewing Photos with 3D/VR Stereo Photo Viewer

You can view and save your photos from the Open Camera Remote HTTP server directly with the 3D/VR Stereo Photo Viewer app. This viewer app has menu features to scan and Save QR Code URLs (using the external ZXing app) and store the URL for a single phone camera photo server, and left and right camera photo servers. Using the View URL Link menu selection in the viewer app you can transfer and view a photos directly from a Open Camera Remote app. There is also an option in the viewer Settings to automatically save transferred files in designated folders. It is recommended you use time-stamps exclusively to order your photos.

Troubleshooting

When the WiFi Remote Capture app cannot seem to trigger the Open Camera Remote app and you know your router supports broadcast messaging, you can try closing the Open Camera Remote app and restarting it. Closing and restarting the WiFi Remote Capture app may also help. When you change your WiFi connection these steps are necessary and required.

Open Source Code Attribution

The open source code for Open Camera Remote (modified from version 1.46 Open Camera, copyright 2013-2019 Mark Harman) is available at https://sourceforge.net/u/ajavamind/opencamera/ci/master/tree/

Open Camera also uses additional third party files see online help for full licences and attributions. Andy Modla wrote the modifications for Open Camera Remote, modifications copyright 2016-2019.

Released under the GPL v3 or later.

Google ZXing ARCODE license:

http://www.apache.org/licenses/LICENSE-2.0 

The Nanohttpd HTTP server is Copyright (c) 2012-2013 by Paul S. Hawke, 2001,2005-2013 by Jarno Elonen, 2010 by Konstantinos Togias All rights reserved.

Network library for processing that supports the UDP protocol was written and copyright by Andreas Schlegel, GNU Lesser General Public License version 2.1. http://www.sojamo.de/libraries/oscP5 

Modifications to this code were made by Andy Modla.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistribution of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

Redistribution in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

Neither the name of the NanoHttpd organization nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANT-ABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.