Getting Actual Distance Data (in cm) from Depth Map

OpenCV, Python and other image processing questions
Post Reply
angelyn.mercado
Posts: 5
Joined: Thu Dec 05, 2019 5:22 am

Getting Actual Distance Data (in cm) from Depth Map

Post by angelyn.mercado »

Hi! Our team has just bought 3 StereoPi kits for our robots. We are trying to make an altitude sensor. We thought about using the Depth Map to get depth data but we're not sure how to get the exact distance (in cm) of each pixel in the depth map. Any suggestions on how to do this? Thank you!

stereomaton
Posts: 195
Joined: Tue May 21, 2019 12:33 pm
Location: France

Re: Getting Actual Distance Data (in cm) from Depth Map

Post by stereomaton »

I never did that, but if I remember my readings well, Z=F.B/D (depth = focal × base ÷ disparity) when infinity is at 0 disparity.
You can calibrate your cameras to get the focal length and the stereo base in the right units.
Alternatively, you can consider that F.B is constant (the distance between the cameras is fixed and the lens is not changed) and thus match a curve on measured data.

I wonder which accuracy you can get with stereo measures.
Stereophotographer and hacker
Despite my quite active participation in the forum, I am not in the StereoPi team
StereoPi Standard Edition + CM3Lite module + a few cameras

User avatar
Realizator
Site Admin
Posts: 636
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Getting Actual Distance Data (in cm) from Depth Map

Post by Realizator »

Hi Angelyn,
Actually I planned to cover this question in upcoming second part of my "StereoPi Fisheye robot" article. I will do it later, but most of the code you need already published in my Github repo.

Key points for you:

1. Connecting the real world with the calculated world
First step for "connecting" virtual world, calculated from video frames, is correct settings of a square size on a chessboard at the moment of camera calibration.
For example, in this calibration script "4_calibration_fisheye.py" you can find this code:

Code: Select all

# Chessboard parameters
rows = 6
columns = 9
square_size = 2.5
You need to set adequate square size for correct calculations in the future. You can find a lot of this question discussions, like this one.

2. Depth map is not a distance
Depth map is actually a disparity between two points on a left and right images. For calculating real distance you can use a simple trigonometry calculations. Luckily, OpenCV has a special function for this, which recalculate disparity map to the real-world 3D coordinates X, Y and Z for each image point. This is "reprojectImageTo3D" function.

3. Code example
Actually I've already uploaded 7th script called "7_2d_map.py" to my repo, but not yet commented it.
To say briefly, we take a middle horisontal part of 3D map, choose the closest points to the camera, reconstruct it in 3D, equalize Z coordinates and get 2D obstacles map for indoor robot. As I understand, you do not need this approach, and can just use "reprojectImageTo3D" directly.

After calibration and 3D reconstruction you can use some object at, say, 2 meters from the camera, to calibrate your actual measurements.

One more notice: "front" direction in 3D reconstructed world is not coinciding with the "front" direction of your stereocamera. You will find this in your experiments.

p.s. Stereomaton, you've posted your answer while I was writing my. :-) You are right about calculations, and, as I mentioned, this feature is implemented in OpenCV. Also focal length is calculated automatically while calibration procedure.
Eugene a.k.a. Realizator

angelyn.mercado
Posts: 5
Joined: Thu Dec 05, 2019 5:22 am

Re: Getting Actual Distance Data (in cm) from Depth Map

Post by angelyn.mercado »

Wow! Thanks for your responses! I actually tried the triangulation formula but wasn't sure what to do with all the values since we get one per pixel.

I will check out the "reprojectImageTo3D" function and will get back with my results. I'll also look into the 7_2d_map.py. Your codes are very user-friendly!

Thank you again!

User avatar
Realizator
Site Admin
Posts: 636
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Getting Actual Distance Data (in cm) from Depth Map

Post by Realizator »

Yes, Angelyn, please share your results! If you will have any questions - please ask it here. I will add all information from our discussion to upcoming article, and hope it will be useful for other StereoPi users too.
Eugene a.k.a. Realizator

Post Reply