What metadata is needed for OpenCV to create stereo depth map?

OpenCV, Python and other image processing questions
Post Reply
sodyiumcl
Posts: 9
Joined: Tue May 31, 2022 11:36 am

What metadata is needed for OpenCV to create stereo depth map?

Post by sodyiumcl »

Hello,

The StereoPi is obviously great to obtaining stereo still images that could be used to create depth maps that tell me how far way objects are from the camera. That said, I would like to know what basic pieces of data are needed on a more general level when creating stereo images.

Let's say I use two identical cameras side-by-side to take simultaneous photos, what does OpenCV need to create a depth map from those stereo images? Do I simply need the distance between the two cameras or would more data be required? If so, that? Is there a list somewhere??

Thank you.

stereomaton
Posts: 215
Joined: Tue May 21, 2019 12:33 pm
Location: France

Re: What metadata is needed for OpenCV to create stereo depth map?

Post by stereomaton »

I am sure you can find tuto/book online about it, I did a while ago.
You can also look at the code from realizator https://github.com/realizator/stereopi-fisheye-robot and read the official documentation of the called functions.

From what I remember, you have three steps:
1) Align images
Find the transformation (non-linear to correct lens deformation if needed + linear perspective transform to correct for non-parallel sensors) and crop to get all homologous points aligned horizontally. This is usually achieved though calibration images and parameter optimization to minimize reprojection error on the feature points found in those images.
If the cameras are fixed (focal & position), you can apply the same transformation to other images to get good alignment.

2) Find disparity
The algorithms try to find which (horizontal) displacements allow to match the patterns in the other images. Some algos work with patches, some use finer grain, but the principle is the same: compare the two images. Thus the images should have enough texture, good quality, and so on, and in case of patches, their size (and other parameters) should be adapted to the content photographed.

3) Translate disparity to depth value if needed
The previous calibration can help to know the exact focal length and separation between the lenses which influence the disparity on the sensor (proportional to them and to 1/depth).
Stereophotographer and hacker
Despite my quite active participation in the forum, I am not in the StereoPi team
StereoPi (v1) Standard Edition + CM3Lite module + a few cameras

User avatar
Realizator
Site Admin
Posts: 900
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: What metadata is needed for OpenCV to create stereo depth map?

Post by Realizator »

Stereomaton, thank you for the detailed answer!

I'd add that as a result of your steps 2 and 3 OpenCV actually ends with a simple triangle calculations, and the idea of this approach is greatly described in this OpenCV guide.
Eugene a.k.a. Realizator

Post Reply