Some questions before pledging V2

StereoPi hardware discussion
Post Reply
mr.burns
Posts: 14
Joined: Fri Mar 19, 2021 5:16 pm

Some questions before pledging V2

Post by mr.burns »

Hi,

first of all I want to say thank you for this great project! I am stereo hobbyist and looked for such stereo camera for a long time!
Keep going on this great work please!

Basically I am intersted to use StereoPI V2 to built a still image stereo cam which I can use in the fields and later view these images using a side by side stereo viewer similar to this:

Image

Movie I am not so interested for now (may this will change in future).
I am software delveloper and have 3D printer, lasercutter and milling machine at hand, so I want to give StereoPI V2 a chance to built a great cam.
Before buying/pleding V2 I have some questions, may someone can help me with these:
  1. The V2 kit comes with 2 x IMX219 160 FoV camera. Each is 8MP - right?
    Can I shoot stereo still images at this high resolution - means I get images 2 x 8MP side by side (which will result in 16MP image)?
    What about even higher resulution lime 12.3 MP using the IMX477 (will result in 24MP image)?
  2. For still images I need high synchronization of left and right image to shoot also moving targets like people, flames, water, etc.
    Are both cameras synched by software or hardware? If hardware is it global shutter?
  3. I want to add a rotary encoder (or other switches) to change cam settings like, shutter speed, exposure, brightness, white balance, etc.
    How can I get there? Should I use 'normal' Raspi OS, libcamera and python or can I also use your SLP as basis and go ahead from it and still use Python?
    (for shutterspeed I read an post here (viewtopic.php?f=6&t=951&p=1997&hilit=shutter#p1997) - is it possible now to set the shutterspeed for both cams now?
  4. Is your SLP a fixed software or can I use and modify it for my needs?
  5. If I use a button to shoot the images what is the delay from triggering the button to final image?
  6. If I hold the shoot button can I do image series? E.g. shoot images every half a second as long as I hold the button?
  7. Mostly I will use the camera outside. What is the recommended battery and power device to drive the V2 kit, CM4 and the 2 cams?
  8. To preview the stereo image before shooting I saw in one of the videos a split screen on the cam display.
    Is it possible to see also the image of left or right cam istead of split screen?
  9. Since I want to shoot still images I have to set the focus somehow - which cams (supported by StereoPI) do have autofocus?
    Or can I use these?: https://www.arducam.com/raspberry-pi-camera/autofocus/
  10. Can I attach a bigger display? And can I attach a touch screen and use it as interface to change the cam settings?
  11. Can I attach two displays one for left cam one for right cam?
  12. If I want to use your SLP can I change camera settings using the GPIOs or do I need an external PC and access the web UI?
  13. Can I use WLAN to connect my mobile phone to the Web UI?
  14. I know for stereo images it is necessary to to some image processing after shooting to get rid of lens distortions, not properly aligned cameras, etc. I think it's called image rectification - correct me if I am wrong.
    Is this done by the StereoPI module? Or in your software?
  15. To develop a software for StereoPI V2 (e.g.to access the rotary encoder, switches, etc.) what is a good starting point?
    Are there tutorials for developing StereoPI in Python?
  16. What is the preferred programming language?
So that's it for now - hopefully someone can help me - thanks in advance :)

stereomaton
Posts: 215
Joined: Tue May 21, 2019 12:33 pm
Location: France

Re: Some questions before pledging V2

Post by stereomaton »

I am a user of the v1 and did not looked at the v2 much, but I can answer some questions.
By the way, it seems that you edited your post while I was writing an answer (with a pause of a few hours), because the numbers do not correspond anymore. I have to start again.

The StereoPi is not a ready-to-use stereoscopic camera in the sense a photographer would understand [like a successor of the Fuji W3], but it is a building block that could allow to do something like this with some efforts. With your background with software and the tools you have, it should not be out of reach. [By the way if you do this, thanks to share]
Last edited by stereomaton on Sun Mar 21, 2021 6:23 pm, edited 1 time in total.
Stereophotographer and hacker
Despite my quite active participation in the forum, I am not in the StereoPi team
StereoPi (v1) Standard Edition + CM3Lite module + a few cameras

stereomaton
Posts: 215
Joined: Tue May 21, 2019 12:33 pm
Location: France

Re: Some questions before pledging V2

Post by stereomaton »

So the answers: [this time I copied the questions :-)]

> The V2 kit comes with 2 x IMX219 160 FoV camera. Each is 8MP - right?
IMX219 is a 8Mpx sensor, yes.

> Can I shoot stereo still images at this high resolution - means I get images 2 x 8MP side by side (which will result in 16MP image)?
The v1 was able to get full-size not-squeezed side-by-side.

> What about even higher resulution lime 12.3 MP using the IMX477 (will result in 24MP image)?
I think Realizator got the full definition with this sensor too. I do not have them personally.

> For still images I need high synchronization of left and right image to shoot also moving targets like people, flames, water, etc. Are both cameras synched by software or hardware?
The sync is in between: the images are fetched at the same time from two not synchronized cameras. Since the process is done by dedicated hardware, the synchronization is very good but not guaranteed at the fine scale; and cameras could drift over time (there are hardware modifications possible to reduce this, although for most usages it is not necessary). You can have an idea of the sync (v1, no hardware modification) with this video: https://youtu.be/Vk8Fhci0IvA

> If hardware is it global shutter?
The IMX219 has rolling shutter.
There exist cameras compatible with StereoPi which have global shutter (but those I think of sold by ArduCam have 1Mpx and 2Mpx sensors)

> Is your SLP a fixed software or can I use and modify it for my needs?
SLP is a headless Raspbian with a software suite written by Realizator. You can tweak it, a large part is made of PHP scripts.

> If I use a button to shoot the images what is the delay from triggering the button to final image?
It depends on the method you use. In v1 it was not built-in and I proposed a way to modify the code in this forum (not the most reactive way, but simple modification; I do not have an estimation to give). There is a blog post about fast synchronization of multiple StereoPi which could be used with a single StereoPi triggered by a button, and this one is extremely reactive.

> If I hold the shoot button can I do image series? E.g. shoot images every half a second as long as I hold the button?
Again, the button was not built-in. Since you add the code you want, you could do this, or other reaction like start/stop on a timelapse.

> Mostly I will use the camera outside. What is the recommended battery and power device to drive the V2 kit, CM4 and the 2 cams?
Personally, I used (with v1) a 2A power pack initially aimed to recharge mobile phones (in the pocket with a cable to the stereopi for lighter weight in the hands)

> To preview the stereo image before shooting I saw in one of the videos a split screen on the cam display. Is it possible to see also the image of left or right cam istead of split screen?
This preview is direct image from the GPU to the screen. You could get the image in a program to display whatever you want, but you might need to break some functionalities of SLP for this. You can also access left and right cameras separately, but you loose the sync by doing so.

> Since I want to shoot still images I have to set the focus somehow - which cams (supported by StereoPI) do have autofocus?
Check ArduCam cameras, some of them have piezzo focus adjustment controllable via I²C. By the way, the auto part of the autofocus should be done in your software that controls the cameras.

> Or can I use these?: https://www.arducam.com/raspberry-pi-camera/autofocus/
Sorry, I do not have the courage to read it. But very likely.

> Can I attach a bigger display?
Yes. I used a SPI screen (cheap) but I do not recommend it because its framerate is limited, and HDMI screen which works well and without code.

> And can I attach a touch screen and use it as interface to change the cam settings?
Yes. But you would have to install the associated drivers (and code the interface)

> Can I attach two displays one for left cam one for right cam?
Not sure. You could perhaps have two SPI screens, or one SPI and one HDMI but I doubt it would be very friendly.

> If I want to use your SLP can I change camera settings using the GPIOs or do I need an external PC and access the web UI?
You can write a script (e.g. python) that read the GPIO and do local web calls (no need to be connected to anything, network works from one program to another via the loopback interface internally; you probably know it as a dev though)

> Can I use WLAN to connect my mobile phone to the Web UI?
Yes. In v1 it worked with USB wifi sticks.

> I know for stereo images it is necessary to to some image processing after shooting to get rid of lens distortions, not properly aligned cameras, etc. I think it's called image rectification - correct me if I am wrong. Is this done by the StereoPI module? Or in your software?
Not in v1. You could do it on PC afterwards or in the StereoPi with OpenCV for example (but again, you could have to break functionalities of SLP to access the cameras instead of the built-in scripts)

> To develop a software for StereoPI V2 (e.g.to access the rotary encoder, switches, etc.) what is a good starting point?
Rotary encoders might need special hardware to decode fast signals, but you could write a script to decode it through GPIOs. I guess that low speed would be okay. Switches are regular GPIOs. You could read tutorials about GPIOs with raspberry-pi, it works the same. There exist python libraries to abstract the operations.

> Are there tutorials for developing StereoPI in Python?
It is the same as with Raspberry-pi (or embedded Linux in general)

> What is the preferred programming language?
It depends on you ^^ and a little on what you want to achieve. Python might be a good choice for a lot of applications. In the article about a reactive button with low jitter, C was preferred because it gives deeper access to some functionalities of the kernel.
Stereophotographer and hacker
Despite my quite active participation in the forum, I am not in the StereoPi team
StereoPi (v1) Standard Edition + CM3Lite module + a few cameras

mr.burns
Posts: 14
Joined: Fri Mar 19, 2021 5:16 pm

Re: Some questions before pledging V2

Post by mr.burns »

Hi stereomaton,

thanks for your detailed reply and sorry that you had to do it twice. So it seems StereoPI could be a basis to start from.
What I still concern is the synchronization.
Currently I use two Canon IXUS 860 synched by https://chdk.fandom.com/wiki/CHDK which gives me a very good synchronization of around 1msec (1/1000 sec !!!). And that I want to get with StereoPI if possible.
The videolink you provided looks quite promising, but video is 'only' around 1/30 or 1/60 sec. If I have still images of fast moving objects (like flames, waterdrops from breaking waves in the air, etc.) I need synchronisation less than 1/100 or even less than 1/500 sec.
Is that possible with StereoPI?

Or is it planned to get hardware based synchronization (if this is possible)?

User avatar
Realizator
Site Admin
Posts: 900
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Some questions before pledging V2

Post by Realizator »

Hi, Mr.burns,
Stereomaton put such a detailed question, so I decided to not add anything :)
As for your synchronization question, if you plan to work with HQ cameras, they have a native, built-in hardware sync feature (with nanosecond precision). You should enable this in Raspbian settings and connect two pads on the cameras (soldering needed). I put tech details on this in our update here.
Eugene a.k.a. Realizator

mr.burns
Posts: 14
Joined: Fri Mar 19, 2021 5:16 pm

Re: Some questions before pledging V2

Post by mr.burns »

Hi Realizator,

yes stereomaton did a great job!

built-in hardware sync feature Wow - that sounds really great - thanks!
OK lets assume I will go with the HQ cams, there is no pledge for it (including CM4) - right?
What is the exact camera Item No. or description for this HQ cam? Is it IMX477? If yes I hope I can get it in somewhere Germany.

What about an autofocus for these HQ cams? Since we have a stereo pair of cams I believe it is very difficult to set the focus manually and simoultanously on two cams.

Arducam offers this one: https://www.arducam.com/product/arducam ... pberry-pi/.
Does this also support the high synchronization you mentioned and is compatible with StereoPI V2?

User avatar
Realizator
Site Admin
Posts: 900
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Some questions before pledging V2

Post by Realizator »

Mr.Burns,
HQ camera has the only version. Here is the official webpage: https://www.raspberrypi.org/products/ra ... ty-camera/

Yep, it is IMX477 based

Autofocus is another hardware, and syncing it is another trick. HQ cameras are syncing their image capture start time (i.e. triggering the sensor to get an image). As fr autofocus - IMHIO I'd use a manual focus, but this is on your own.

As for Arducam and focus sync. You see, the camera sensor is workong on gigahertz frequencies. And sync means just to send the same command to both cameras. But autofocus includes the mechanics (motors) with extremely high latency (in comparison with the electronic part). That's why I prefer manually adjusted and fixed focus. :)
Eugene a.k.a. Realizator

mr.burns
Posts: 14
Joined: Fri Mar 19, 2021 5:16 pm

Re: Some questions before pledging V2

Post by mr.burns »

Hi Realizator,

thanks for quick response.
To make it clear, I can use the Arducam IMX477 with their Autofocus system together with StereoPI - right?

I know that autofocus needs some time since it is a mechanical part, but thats fine. I can wait until both cams have finished autofocusing and then do a high synchronized still image shot. High synchronization I need only for the final image, if both cams need some time for focusing I am fine with that.

Arducam on their webpage provide some code using OpenCV for some contrast autofocus (https://www.arducam.com/raspberry-pi-camera/autofocus/)
Since I am not an expert in SPL, by using SPL can I still do some python programming to get both cams focusing?

User avatar
Realizator
Site Admin
Posts: 900
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Some questions before pledging V2

Post by Realizator »

Mr.Burns, as I remember, all Arducam products have some extra features, but also require special drivers compilation and installation tricks. As a result, you can't use classic tools like raspistill and raspivid, which are the default and friendly ways to take images and videos on Raspberry.
Our SLP image is based on these system tools (as we are keeping 100% compatibility with stock Raspberry OS).
I did not play with this Arducam model. But if you want to use it with SLP, it is better to make sure it can be used with raspistill and raspivid utilities. You can ask Arducam about this.
Otherwise, you will need to write all image capture code from scratch (using Arducam drivers), and this is a tricky way. This is not a problem for advanced users (with programming skills), but may cause a lot of issues for regular customers.
Eugene a.k.a. Realizator

mr.burns
Posts: 14
Joined: Fri Mar 19, 2021 5:16 pm

Re: Some questions before pledging V2

Post by mr.burns »

Hi,

thanks again. I took a deeper look in the Arducam's python code snippet they use to do the focus.
They use some keys on keyboard and send commands via i2c to the cam to change focus. I think I can adapt it to use GPIOs.

Image

It seems they rely on the builtin function raspistill (see green area).
Can I use python together with your SLP?

If I want to adapt Arducam's code - do I have to do it for two cams seperately? Or is StereoPI to treat as one cam?

Is there any guide to get more familiar with SLP and custom programming?

stereomaton
Posts: 215
Joined: Tue May 21, 2019 12:33 pm
Location: France

Re: Some questions before pledging V2

Post by stereomaton »

The frame rate of the video is not relevant in the estimation of the sync. They are just a lot of photos and a delay can be present at each shot. With the fast moving subject, we see that this delay is by far smaller than the time between two frames.
As I said, this small sync error might be small enough for most cases, and can be improved with hardware modification (using one oscillator for both cameras, or the sync signal for the HQ cameras)
Since you seem to have the protocol to measure it, maybe you could give us an estimate of the sync without hardware modification. I would not be surprised if it is less than a few milliseconds.

About autofocus, as I said, the auto part has to be done by your software. Giving focus orders via I2C is doable with the stereopi, but you have to compute which value to send (or write an interface to select the value manually, which is not automatic, but not fixed). Notice that to compute the focus value by analyzing the image, you have to get the image in your software and thus probably have to lost SLP functionalities since you get the control of the cameras. (You can send [manual] I2C focus commands without controlling the cameras' video stream, thus taking advantage of the software suite of SLP)

Of course you can use python on SLP. This distribution is mainly a Raspbian with some pre-installed software for streaming and a few other functionalities, plus the dt-blob.bin adapted to describe this board to the kernel.
To control the focus, you have to send the command to each camera, on the appropriate I2C bus.
For programming guide, there is nothing special with SLP. It is a regular embedded Linux, and more specifically a Raspbian. The board is compatible with raspberry-pi.
Stereophotographer and hacker
Despite my quite active participation in the forum, I am not in the StereoPi team
StereoPi (v1) Standard Edition + CM3Lite module + a few cameras

Post Reply