Getting Actual Distance Data (in cm) from Depth Map
-
- Posts: 6
- Joined: Thu Dec 05, 2019 5:22 am
Getting Actual Distance Data (in cm) from Depth Map
Hi! Our team has just bought 3 StereoPi kits for our robots. We are trying to make an altitude sensor. We thought about using the Depth Map to get depth data but we're not sure how to get the exact distance (in cm) of each pixel in the depth map. Any suggestions on how to do this? Thank you!
-
- Posts: 215
- Joined: Tue May 21, 2019 12:33 pm
- Location: France
Re: Getting Actual Distance Data (in cm) from Depth Map
I never did that, but if I remember my readings well, Z=F.B/D (depth = focal × base ÷ disparity) when infinity is at 0 disparity.
You can calibrate your cameras to get the focal length and the stereo base in the right units.
Alternatively, you can consider that F.B is constant (the distance between the cameras is fixed and the lens is not changed) and thus match a curve on measured data.
I wonder which accuracy you can get with stereo measures.
You can calibrate your cameras to get the focal length and the stereo base in the right units.
Alternatively, you can consider that F.B is constant (the distance between the cameras is fixed and the lens is not changed) and thus match a curve on measured data.
I wonder which accuracy you can get with stereo measures.
Stereophotographer and hacker
Despite my quite active participation in the forum, I am not in the StereoPi team
StereoPi (v1) Standard Edition + CM3Lite module + a few cameras
Despite my quite active participation in the forum, I am not in the StereoPi team
StereoPi (v1) Standard Edition + CM3Lite module + a few cameras
- Realizator
- Site Admin
- Posts: 900
- Joined: Tue Apr 16, 2019 9:23 am
- Contact:
Re: Getting Actual Distance Data (in cm) from Depth Map
Hi Angelyn,
Actually I planned to cover this question in upcoming second part of my "StereoPi Fisheye robot" article. I will do it later, but most of the code you need already published in my Github repo.
Key points for you:
1. Connecting the real world with the calculated world
First step for "connecting" virtual world, calculated from video frames, is correct settings of a square size on a chessboard at the moment of camera calibration.
For example, in this calibration script "4_calibration_fisheye.py" you can find this code:
You need to set adequate square size for correct calculations in the future. You can find a lot of this question discussions, like this one.
2. Depth map is not a distance
Depth map is actually a disparity between two points on a left and right images. For calculating real distance you can use a simple trigonometry calculations. Luckily, OpenCV has a special function for this, which recalculate disparity map to the real-world 3D coordinates X, Y and Z for each image point. This is "reprojectImageTo3D" function.
3. Code example
Actually I've already uploaded 7th script called "7_2d_map.py" to my repo, but not yet commented it.
To say briefly, we take a middle horisontal part of 3D map, choose the closest points to the camera, reconstruct it in 3D, equalize Z coordinates and get 2D obstacles map for indoor robot. As I understand, you do not need this approach, and can just use "reprojectImageTo3D" directly.
After calibration and 3D reconstruction you can use some object at, say, 2 meters from the camera, to calibrate your actual measurements.
One more notice: "front" direction in 3D reconstructed world is not coinciding with the "front" direction of your stereocamera. You will find this in your experiments.
p.s. Stereomaton, you've posted your answer while I was writing my. You are right about calculations, and, as I mentioned, this feature is implemented in OpenCV. Also focal length is calculated automatically while calibration procedure.
Actually I planned to cover this question in upcoming second part of my "StereoPi Fisheye robot" article. I will do it later, but most of the code you need already published in my Github repo.
Key points for you:
1. Connecting the real world with the calculated world
First step for "connecting" virtual world, calculated from video frames, is correct settings of a square size on a chessboard at the moment of camera calibration.
For example, in this calibration script "4_calibration_fisheye.py" you can find this code:
Code: Select all
# Chessboard parameters
rows = 6
columns = 9
square_size = 2.5
2. Depth map is not a distance
Depth map is actually a disparity between two points on a left and right images. For calculating real distance you can use a simple trigonometry calculations. Luckily, OpenCV has a special function for this, which recalculate disparity map to the real-world 3D coordinates X, Y and Z for each image point. This is "reprojectImageTo3D" function.
3. Code example
Actually I've already uploaded 7th script called "7_2d_map.py" to my repo, but not yet commented it.
To say briefly, we take a middle horisontal part of 3D map, choose the closest points to the camera, reconstruct it in 3D, equalize Z coordinates and get 2D obstacles map for indoor robot. As I understand, you do not need this approach, and can just use "reprojectImageTo3D" directly.
After calibration and 3D reconstruction you can use some object at, say, 2 meters from the camera, to calibrate your actual measurements.
One more notice: "front" direction in 3D reconstructed world is not coinciding with the "front" direction of your stereocamera. You will find this in your experiments.
p.s. Stereomaton, you've posted your answer while I was writing my. You are right about calculations, and, as I mentioned, this feature is implemented in OpenCV. Also focal length is calculated automatically while calibration procedure.
Eugene a.k.a. Realizator
-
- Posts: 6
- Joined: Thu Dec 05, 2019 5:22 am
Re: Getting Actual Distance Data (in cm) from Depth Map
Wow! Thanks for your responses! I actually tried the triangulation formula but wasn't sure what to do with all the values since we get one per pixel.
I will check out the "reprojectImageTo3D" function and will get back with my results. I'll also look into the 7_2d_map.py. Your codes are very user-friendly!
Thank you again!
I will check out the "reprojectImageTo3D" function and will get back with my results. I'll also look into the 7_2d_map.py. Your codes are very user-friendly!
Thank you again!
- Realizator
- Site Admin
- Posts: 900
- Joined: Tue Apr 16, 2019 9:23 am
- Contact:
Re: Getting Actual Distance Data (in cm) from Depth Map
Yes, Angelyn, please share your results! If you will have any questions - please ask it here. I will add all information from our discussion to upcoming article, and hope it will be useful for other StereoPi users too.
Eugene a.k.a. Realizator