Autofocus camera

Raspberry Pi cameras (V1, V2, wide angle etc), HDMI capture modules etc.
Post Reply
peterfiabigron
Posts: 14
Joined: Fri Sep 16, 2022 8:19 am

Autofocus camera

Post by peterfiabigron »

Hello,

I am new to this community, just received my StereoPi v2 camera kit.
I am into the stereoscopic photography. Until now I used an "own" method to capture stereo photos of moving objects. I used two smartphones and made series of shots by holding shot buttom for some seconds on both phones at the same time. Afterwards, I selected the perfectly matching photos of the two phones from the e.g. 30 photo series, which generally resulted in a well sincronized stereophoto (even for e.g. water waves... , afterwards i postprocessed it with android "3d steroid pro" app). The real pain was the difficult handling of the two phones, and long postprocessing (selectimg matching photos...). In the pro side is that i had high quality, versatile, and ready to shoot cameras (in the sence that i did not have to focus those), and i could adjust the stereobase.

My aim would be something similar and more with this device, wihout the above described cons. Therefore i would like an autofocus, smarphone like camera pair.

My question would be whether one of these could be used with all of the functions including autofocus on stereopi v2?
https://www.arducam.com/product/64mp-af ... pberry-pi/
https://www.waveshare.com/wiki/IMX519-78_16MP_AF_Camera
Is there a possibility to use smartphone as controller touchscreen and use autofocus, and redirect autofocus by tap, when it is needed?

As I see from my test shots, for videos maybe the bigger fov (e.g. the 160) is more optimal for a more natural, more immersed feeling, but for the pictures the fov80 is more natural. Does it make sence to you? Or for videos actually i see a smaller fov image in the sereovideo? What is your opinion on this?

And do you think is there any possibility to have two pairs of cameras installed (e.g. a bigger fov and a smaller fov) and switch back and forth between them on the fly (e.g. with a switch, quicky between shots?

Sorry for the long post, but I am really enthusiast about this StereoPi V2 camera kit, and I would like to tailor it to perfectly match my plans.

Many thanks for the development and availability of this excellent device!
Gábor

User avatar
Realizator
Site Admin
Posts: 900
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Autofocus camera

Post by Realizator »

Wow, it's a nice trick with two phones! :-)

As for the autofocus cameras - let me explain how our system works under the hood.

Our ready-to-use image (SLP2) is built on the base of a previous video subsystem (MMAL/raspivid/raspistill). This approach allows us to work with all the regular Rbi-compatible cameras with no driver tricks. I.e., you can connect the V1 camera or clones (OV5647 sensor), V2 (IMX219), and HQ (IMX477).

If you need to connect some non-standard camera (like the mentioned autofocus modules), you must dig the driver to force all the stuff to work correctly with SLP2. That's hard. As a plan B, you can use the "stock" RaspiOS image, install all drivers and write some code. I could not play with these sensors and thus can't say if the two-camera mode is supported in their drivers/firmware.

As for the autofocus: it's a pain for the stereoscopic system when both cameras focus on their own. Ideally, you need to consider one camera as a "master" and another camera as a "slave" and "copy" the focus state to the slave. We have not implemented this feature, but this is possible if you can access the camera focus control with the software.

As for the FOV - you should know one trick. If you have 16:9 images (like 1280x720) and record the stereoscopic video with default settings, you'll get two images with 640x720 resolution. If you enable "decimation" mode, each video is taken in 1280x720 and then scaled to 640x480. Most players (like YouTube) are "scaling back" while playing, so you see the 16:9 image for each eye.

Another point: some sensors (like IMX219) are doing "cropping" at some resolutions in the video mode. You can dig the RPi Camera data sheets to find this info.

And the last info: in the "photo" mode, we are trying to use every available pixel of the sensor; that's why you are getting a 4:3 image for each eye. It means that when you are doing a 16:9 photo or image, you lose the image's top and bottom. But in reality, all sensors have a 4:3 side ratio.

I prefer to use 160 FOV since it gets a better sense of immersion.

As for the two pairs of cameras switching - this is hard. The camera's ribbons are passing a huge amount of data, and you need a special hardware solution to implement this switch.
Eugene a.k.a. Realizator

peterfiabigron
Posts: 14
Joined: Fri Sep 16, 2022 8:19 am

Re: Autofocus camera

Post by peterfiabigron »

Dear Eugene,

Thank you very much for your detailed explanation.
If I understand correctly, if an alternative camera is compatible with raspberry pi cm4 without any additional driver, than there is a high chance that i can use it with SLP2?

Regarding autofocus, then I assume it is best to wait until raspberry libcamera is capable of af. It would be nice to have AF lenses for HQ cameras in the future with this master-slave connection, but I assume it would require big power supply.

Interestingly, when I used decimation for photos, i ended up with compressed pictures (i.e. unnatural width of objects on the photo). But maybe it is caused by my phone, where i opened the pictures?

I will look for this rpi data sheet for cropping info, thanks.

May i ask, whether with a touchscreen mipi i2c (something like this: https://www.waveshare.com/product/raspb ... m?___SID=U) would it be possible to have the control panel on the screen along with the preview, to have complete control over the system and the photo/video shoots on it?

Many thanks again!
Gábor

User avatar
Realizator
Site Admin
Posts: 900
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Autofocus camera

Post by Realizator »

Hi Gábor,
peterfiabigron wrote:
Sat Oct 01, 2022 9:08 am
If I understand correctly, if an alternative camera is compatible with raspberry pi cm4 without any additional driver, than there is a high chance that i can use it with SLP2?
Yes, it is.
peterfiabigron wrote:
Sat Oct 01, 2022 9:08 am
Regarding autofocus, then I assume it is best to wait until raspberry libcamera is capable of af. It would be nice to have AF lenses for HQ cameras in the future with this master-slave connection, but I assume it would require big power supply.
Yes, I'm waiting for this solution too. As I know, the HQ camera's hardware sync is supported by the old video subsystem only (MMAL-based), and with libcamera the work is in progress.
peterfiabigron wrote:
Sat Oct 01, 2022 9:08 am
Interestingly, when I used decimation for photos, i ended up with compressed pictures (i.e. unnatural width of objects in the photo). But maybe it is caused by my phone, where I opened the pictures?
With the decimated photo you are getting the image with the width scaled by 0.5 If you take any image editor and scale the image x2 horizontally keeping the same height, you'll get the photo with the correct aspect ratio. As a relish, you have x2 real-world FOV with this image.
peterfiabigron wrote:
Sat Oct 01, 2022 9:08 am
May i ask, whether with a touchscreen mipi i2c (something like this: https://www.waveshare.com/product/raspb ... m?___SID=U) would it be possible to have the control panel on the screen along with the preview, to have complete control over the system and the photo/video shoots on it?
Well, this is not as simple :-) Our SLP2 image is based on a "lite" version of RpiOS without GUI. It means you can't just connect the touchscreen (recognized as a mouse by the system) and work with it. Some special logic is needed here, optimized for this tiny screen size. Right now the admin panel can be used on your phone to adjust all settings. Try to imagine that you see this interface on the 2" screen with a 320x240 resolution.
(actually, we did some experiments with buttons support and updated interface, but this is in the very early stage now)

As for the screen you've mentioned - this is a good item. But StereoPi was originally planned as a compact "remote" device, thus we removed the DSI interface, used on the regular RPi for the screen connection. For the StereoPi it is possible to use the HDMI touchscreen display, but this will require one more connection (display -> USB).
Eugene a.k.a. Realizator

peterfiabigron
Posts: 14
Joined: Fri Sep 16, 2022 8:19 am

Re: Autofocus camera

Post by peterfiabigron »

Dear Eugene,

Thank you, I am looking forward to see what future holds to RPI and to this project. I think the limit is the sky. :D
Thanks again for the great work and that you made many projects possible with this device :)

Post Reply