StereoPi Forum

Hello,

The StereoPi is obviously great to obtaining stereo still images that could be used to create depth maps that tell me how far way objects are from the camera. That said, I would like to know what basic pieces of data are needed on a more general level when creating stereo images.

Let's say I use two identical cameras side-by-side to take simultaneous photos, what does OpenCV need to create a depth map from those stereo images? Do I simply need the distance between the two cameras or would more data be required? If so, that? Is there a list somewhere??

Thank you.

I am sure you can find tuto/book online about it, I did a while ago.
You can also look at the code from realizator https://github.com/realizator/stereopi-fisheye-robot and read the official documentation of the called functions.

From what I remember, you have three steps:
1) Align images
Find the transformation (non-linear to correct lens deformation if needed + linear perspective transform to correct for non-parallel sensors) and crop to get all homologous points aligned horizontally. This is usually achieved though calibration images and parameter optimization to minimize reprojection error on the feature points found in those images.
If the cameras are fixed (focal & position), you can apply the same transformation to other images to get good alignment.

2) Find disparity
The algorithms try to find which (horizontal) displacements allow to match the patterns in the other images. Some algos work with patches, some use finer grain, but the principle is the same: compare the two images. Thus the images should have enough texture, good quality, and so on, and in case of patches, their size (and other parameters) should be adapted to the content photographed.

3) Translate disparity to depth value if needed
The previous calibration can help to know the exact focal length and separation between the lenses which influence the disparity on the sensor (proportional to them and to 1/depth).

Stereomaton, thank you for the detailed answer!

I'd add that as a result of your steps 2 and 3 OpenCV actually ends with a simple triangle calculations, and the idea of this approach is greatly described in this OpenCV guide.