Another Stereoscopic Camera Project

[TomDev]

Hey all! Thought I'd post on here as I've been speaking to realizator for a while on Twitter talking about the StereoPi.

I'm in the process of building a stereoscopic camera with the StereoPi and wanted to pick people's brains about taking stills and saving them to device/transferring them over wifi.

The main scenario for me is that I'd like to be able to take my StereoPi based camera out and about to different places and take lots of photos, with them being stored on the StereoPi for me to transfer when I get home. I think stereomaton has come up with a solution for this, or part of one? It would be nice to hear what people's thoughts are on this concept and how they'd go about doing it.

I only just got my StereoPi yesterday, so am still playing around with it. I'm also in the process of designing a modular case for my StereoPi, I can post photos up if people would like (as long as no one steals it XD) that can be used with both the standard Raspberry Pi camera and the newer high quality Raspberry Pi camera.

Looking forward to talking to you all

[Realizator]

Hi Tom,
Thank you for your lightweight StereoPi models! I've added them to our Wiki!
Stereomaton helps us a lot with the feature adding and debugging of our SLP Raspbian image, which can fit most of your needs. And we are ready to help you with the customization you want!
And yes, we are looking forward to your new case models!

[TomDev]

Hey there!

No problem, hope they are useful for you and for all tinkerers!

Ah ok cool, what kind of advice can you offer? I'd ideally like a physical button somewhere on my case design that I can press to take a photo.

I've attached a few images of the modular case design. The High Quality camera attachment is a work in progress and isn't particularly a priority right now, but most of the work for the typical Raspberry Pi type cameras is done

I've uploaded a few images. I still need to bevel the top edges so they're better to hold whilst holding the camera. Let me know what you think!

EDIT: I tried uploading more but I could only upload three, odd!

[Realizator]

A-ha, you're planning to do a variable stereobase! Cool!
As for the button - we have a simple Python script for this, designed for the SLP Raspbian image.
If it will be a screen on the back of the case, or you will use a phone as a viewfinder? Or this is a remote setup for the birds photo hunting?

UPD> I've increased number of images per post from 3 to 10, specially for you

[stereomaton]

I think stereomaton has come up with a solution for this, or part of one? It would be nice to hear what people's thoughts are on this concept and how they'd go about doing it.

The code (and other details) of the stereomaton is available here: https://github.com/haum/stereomaton
However, it is mostly a collection of hacks. I never found time to work on a better version for the moment.

In a new version, the goal is to replace the screen by one with an HDMI interface (SPI screens are not adapted to display animated frames) and use nice Qt interface on it with live preview (in app or on overlay). Some configurable processing pipeline and selectable calibration files would be nice too. A must would be to be able to calibrate inside the software, and probably lens and camera orientation separately, so that the orientation could be quickly computed with a few shots of a charuco board when we change the base.

I'd ideally like a physical button somewhere on my case design that I can press to take a photo.

A hack for a button is available here: viewtopic.php?f=10&t=106
It is probably not the best solution, but useful.

I've attached a few images of the modular case design. [...] Let me know what you think!

It looks nice with apparently a compartment for a battery and adjustable baseline.
Let us see the prototype in action and share the case (?).

[TomDev]

A-ha, you're planning to do a variable stereobase! Cool!
As for the button - we have a simple Python script for this, designed for the SLP Raspbian image.
If it will be a screen on the back of the case, or you will use a phone as a viewfinder? Or this is a remote setup for the birds photo hunting?

UPD> I've increased number of images per post from 3 to 10, specially for you

Yeah, the idea is that you can attach/detach any number of 'wings' using the green pegs as shown in the images. Each wing has two carry slots for the pegs, and two fit pegs slots. The carry slots allow the pegs to be more hidden when not in use, whereas the fit slots allow wings to be attached to the main camera body or more wings. Technically you could just keep on attaching wings, although it would get tricky to hold and balance after a while. I think there are camera ribbon cables four metres long, so you could have a stereobase of eight metres

I'll double check the forum post, but I think I may have already seen it. I will probably have more questions!

Yeah I'll be using a screen, a Waveshare 2 inch screen, the form factor is nice and slim and it can be connected to the GPIO! Can you use a mobile device's screen as a viewfinder? Even if there is no wifi or mobile data? I'm thinking about using this out and about away from home.

I have uploaded more images!

[TomDev]

The code (and other details) of the stereomaton is available here: https://github.com/haum/stereomaton
However, it is mostly a collection of hacks. I never found time to work on a better version for the moment.

In a new version, the goal is to replace the screen by one with an HDMI interface (SPI screens are not adapted to display animated frames) and use nice Qt interface on it with live preview (in app or on overlay). Some configurable processing pipeline and selectable calibration files would be nice too. A must would be to be able to calibrate inside the software, and probably lens and camera orientation separately, so that the orientation could be quickly computed with a few shots of a charuco board when we change the base.

I'll check out the code and take maybe steal some bits Ah using a HDMI screen sounds like a good idea! Did you have trouble with your SPI screen then? I don't know what a charuco board is! I will go and have to have a google of one.

A hack for a button is available here: viewtopic.php?f=10&t=106
It is probably not the best solution, but useful.

So, basically the step by step guide MauriceCyril put up using your info is a good place to start? With this line, "urlopen('http://localhost/make_photo.php')" being the most important as this is the one that takes the photo? I'll look into this further. I'm a mobile applications developer by trade, so I was thinking eventually I could hook a bluetooth dongle/module to the StereoPi and remote control the board from a mobile app. I've made bluetooth sensors based on Arduinos before and hooked them up to mobile apps!

It looks nice with apparently a compartment for a battery and adjustable baseline.
Let us see the prototype in action and share the case (?).

Yeah, the baseline is adjustable, and the part where the StereoPi sits can be taken out for easy adjustments. The StereoPi slot also helps hold the battery in place. Still needs some work doing to make it better for use and to make sure the camera boxes stay in place properly. I may create a new type of wing or front, as I'm not sure the snap connector I've created will be sturdy enough.

I'll upload some photos and videos when I'm done making it and had it printed. I'm still attempting to figure out whether to use SLA printing for the case or get it laser cut from wood, then weatherproof the wood somehow (rubber seal/liquid/other).

As for sharing the case, not just yet. It's still not complete, but, if you ask nice, I might be persuaded to share it. Although I'd like to test it first and make sure it suits the purpose before distributing it. The case has my name embedded onto it also in the model, that would have to stay on for advertising purposes XD

[TomDev]

Hey guys, is it possible to create a directory on the StereoPi through Python? I'm getting authority issues here, was just wondering if I could take a photo and store it on device in a new directory! I'm getting the following:

Traceback (most recent call last):
File "/opt/StereoPi/scripts/trigger2.py", line 27, in <module>
file = open (imagePath, mode='w+')
IOError: [Errno 13] Permission denied: '/opt/StereoPi/image.jpg'

Should I use a chmod to make the file have better permissions? Or is it not advisable to write image data to the device in general?

Full code:

Code: Select all

trigger = 3
import RPi.GPIO as GPIO
from time import sleep
from urllib2 import urlopen
import io
import os
import os.path

GPIO.setmode (GPIO.BCM)
GPIO.setup(trigger, GPIO.IN, pull_up_down=GPIO.PUD_UP)

while True:
  path = '/opt/Tom/'
  imagePath = '/opt/StereoPi/image.jpg'
  
  GPIO.wait_for_edge(trigger, GPIO.FALLING)
  print "Cheese!"
  response = urlopen('http://localhost/make_photo.php')
  img = response.read()
  
  file = open (imagePath, 'w+')
  file.write(img)
  file.close()
  sleep(0.5)

[Realizator]

Hi Tom,
Why do you need to write files to the Linux partition?
Several reasons for not to use this:
1. Filesystem is read-only by default. It saves Linux in a case of occasional power off.
2. Space is limited, and it is not easy to increase it (as system use this default size to format all other space to FAT32).

But you can easily write your files (and create folders) on the FAT32 partition! It was designed right for these purposes!
By default, SLP mounts this partition as /media. (You can take a look at the code in this file: /opt/StereoPi/run.sh)
I.e. instead of this part in your code

Code: Select all

path = '/opt/Tom/'

you can use this:

Code: Select all

path = '/media/Tom/'

[TomDev]

Hi Tom,
Why do you need to write files to the Linux partition?
Several reasons for not to use this:
1. Filesystem is read-only by default. It saves Linux in a case of occasional power off.
2. Space is limited, and it is not easy to increase it (as system use this default size to format all other space to FAT32).

But you can easily write your files (and create folders) on the FAT32 partition! It was designed right for these purposes!
By default, SLP mounts this partition as /media. (You can take a look at the code in this file: /opt/StereoPi/run.sh)
I.e. instead of this part in your code

Code: Select all

path = '/opt/Tom/'

you can use this:

Code: Select all

path = '/media/Tom/'

I totally missed the partition stuff! Silly me, I'll do that instead, cheers Realizator! Is this where the images from the web interface are saved also? IPAddress/records? If not, then where is this located? Thanks!

[Realizator]

Yes, all records are there too.
TO check the paths, you can press "Media" icon in the admin panel and look at the path, where all photos and videos are saved.

[stereomaton]

I'll check out the code and take maybe steal some bits

It is shared to be useful.

Did you have trouble with your SPI screen then?

SPI is a "slow" bus, or more precisely it does not benefit from several accelerators for the video (partly due to poor driver for those screens and partly for a more fundamental reason). For example, there is a part of the memory that is reserved for direct access by some hardware parts without passing by the Linux kernel (which is the video split size we configure in the boot partition), and one of those direct accesses are the preview function of raspistill whose output is directly done in a part of this memory, that can then be used as an overlay (ie without slow copies) on the HDMI output. So you have a live preview for free on HDMI, but you would have to do slow copies (and probably resize) for an SPI screen. This is only one example, but it is representative of the advantage of using HDMI even if there is less choice at higher prices for small screens.

I don't know what a charuco board is! I will go and have to have a google of one.

It is a special pattern that can be quickly found by computer vision, that combine the ability to find without ambiguity the orientation of the pattern and the labeling of "corners" which can be used to finely detect these homologous points in the two images. With these points, we can compute the relative orientation of the cameras, and by extension the deformations to apply on the captured images to produce aligned pairs that are easier to view ourselves, but also easier to process for an algorithm.

So, basically the step by step guide MauriceCyril put up using your info is a good place to start? With this line, "urlopen('http://localhost/make_photo.php')" being the most important as this is the one that takes the photo? […] remote control the board from a mobile app.

Yes. Realizator answered almost at the same time as me and pointed a post that summarizes well the whole thread.

The trick used here is to load the "http://localhost/make_photo.php" url when a falling edge is detected on the GPIO.
You can call this page (localhost replaced by the IP of the board) on your mobile app to get the same result.

I'll upload some photos and videos when I'm done making it and had it printed. I'm still attempting to figure out whether to use SLA printing for the case or get it laser cut from wood, then weatherproof the wood somehow (rubber seal/liquid/other).

Cool. At least, your case was inspiring.

With the current idea, you have several 3D shapes that cannot be cut with laser cutter.
I recently tried masqued-SLA and the result is very precise, and pretty inexpensive for the result if you do not pay for the time.

[TomDev]

Ah, I didn't see this message sorry! Thanks for all the explanations

I might try looking at using a HDMI screen then, although the connectors are often bulky. It's why I wanted to use something like the 2inch Waveshare screen I found, to help minimise the space I need inside of the case.

I've done a lot of work and totally redesigned the front of the case now, it looks a lot better and is a lot more flexible. I'll upload some screenshots at some point so you can see. I think SLA will definitely be the way to go, I've spoken to a couple of printing companies and they've told me the same. I've build camera boxes for the spy cam also that you can buy for the Raspberry Pi, I've got the Stereobase down to 1.2cm. I think that's going to be the best I can do, but that still means you can take photos of things 36cm away. There's also a camera box for the high quality camera, albeit quite bulky compared to the other two. It would also need to be tripod mounted if you had big lenses on them!

I was trying to get bluetooth up and running with the StereoPi so I could trigger photo taking from my phone with no wi-fi, but I ran into quite a few issues. I followed various tutorials online and tried a couple of different modules, but there must be a file somewhere that I am not finding or some like of code that isn't working right. The StereoPi wouldn't connect to another bluetooth device at all, even though it would pair with one. Bluez seemed to work nicely for the pairing.

Realizator, the StereoPi should connect to a phone's mobile data correct? If the stereopi was streaming video all the time, I can imagine this would take up the mobile data quite quickly, but do you think it would be possible to turn streaming on and off with a button press, like taking a photo is? I'm thinking that way, when you want to take a photo, you could turn the streaming 'on' and see the feed on your mobile phone via mobile data. If there is a page in the documentation/wiki that I have missed for this, could you point me to it please?

If this kind of concept would work, I'm thinking maybe a screen would not be necessary at all, and that a user could maybe slot their phone onto the case and use that as a screen instead. Or, if there is another way of using a phone as a screen not over wi-fi/mobile data, then that would be great

[Realizator]

...
Realizator, the StereoPi should connect to a phone's mobile data correct? If the stereopi was streaming video all the time, I can imagine this would take up the mobile data quite quickly, but do you think it would be possible to turn streaming on and off with a button press, like taking a photo is? I'm thinking that way, when you want to take a photo, you could turn the streaming 'on' and see the feed on your mobile phone via mobile data. If there is a page in the documentation/wiki that I have missed for this, could you point me to it please?

Tom, StereoPi will use mobile data only it you set up a streaming to the Internet (like YouTube). If you just look at the livestream on your phone from the StereoPi, no mobile data is used. Actually your StereoPi's WiFi dongle and phone's WiFi have a bandwidth load.

[TomDev]

Tom, StereoPi will use mobile data only it you set up a streaming to the Internet (like YouTube). If you just look at the livestream on your phone from the StereoPi, no mobile data is used. Actually your StereoPi's WiFi dongle and phone's WiFi have a bandwidth load.

Really? How does this magic work? I have listed specific questions below.

I'm afraid I'm not familiar with wi-fi/mobile data based things like this. How can a mobile phone see what the StereoPi is streaming without using mobile data?

Does the StereoPi somehow create it's own network? I seem to remember seeing this said somewhere in some form or another, but I didn't understand it.

If there is no Wi-Fi network available, other than a mobile phone hotspot, would you still be able to see the video stream from the StereoPi? The phone needs to be connected to the StereoPi in some way to view the video feed right?

Sorry for my newbie questions, I'm very much a front end developer and 3D artist, rather than a tech focused back end developer, so all of this is new territory for me!

[stereomaton]

There is no magic, it is the way all networks work.

On your mobile, there are several network interfaces (depending on the context, there might be "data", wifi, bluetooth, usb, vpn...). Each one has its own range of IPs (address that corresponds to the device in this network + mask to know all IPs that are directly accessible through this interface)

When you start the wifi hotspot, you configure the wifi interface to run as hotspot ("server") and it will deliver an IP address to the stations ("clients") that connect to it. This IP is local to this network. The hotspot will also indicate the gateway address, which is the address to contact to go outside of the network (through this network; things can become a bit more complex with several networks).

If you want to have a connection between two devices of the network, it does not go outside. Notice that some hotspots disallow the communication between different stations for [very limited] security, for example in public wifi, and I do not remember if it is the case with Android. For sure, it allows the communication between the phone and the StereoPi.

If you want to reach forum.stereopi.com from the StereoPi, it will ask the gateway (which happens to be the phone) which will see its routage table and forward your packets (after tagging to be able to transmit the answers back) to its own gateway and so on until reaching the server (actually there are more complex routes in the middle). Only in this case the data plan is consumed if the traffic passes throught the "data" interface.
Unless if there was a weird configuration, the communication between the StereoPi and the phone, because it stays on the local network, will not consume data plan and more interestingly will work without Internet access.

Because of the versatility of IP, you can replace the hotspot of your phone by a dedicated hotspot or wired router that will forward the packets between the devices or between its own interfaces depending on its own routes (StereoPi in wifi to PC in Ethernet / StereoPi in Ethernet to PC in wifi / wifi<->wifi / Ethernet<->Ethernet, StereoPi to Internet, etc.) If the route between the two peers is well defined, you can use whatever topology you want, and this is one of the grounds that make Internet so powerful.

There is also what is called the loopback interface that allows to communicate by IP (using TCP/UDP sockets) from the StereoPi to the same StereoPi even without any external connections (it exists in Linux/Mac/Windows/FreeBSD/QNX/Etc too). The address is thus 127.0.0.1 (actually anything starting with 127) in ipv4 and ::1 if you use ipv6, also named localhost.

With more efforts, you can make the StereoPi act as a hotspot if you really want (or more likely need).

So more specifically for your questions:
How can a mobile phone see what the StereoPi is streaming without using mobile data? <= Because it uses direct connection through wifi hotspot, that do not need to pass through the Internet
Does the StereoPi somehow create it's own network? <= You can, but it is easier to use the hotspot functionality that most smartphones have.
If there is no Wi-Fi network available, other than a mobile phone hotspot, would you still be able to see the video stream from the StereoPi? <= Yes, because nothing else is needed (except if there is a weird configuration in the hotspot)
The phone needs to be connected to the StereoPi in some way to view the video feed right? <= Of course, there must be a route (that do not drop too much packets)