3D world coordinates from stereo calibration

OpenCV, Python and other image processing questions
Post Reply
latif
Posts: 6
Joined: Thu Oct 08, 2020 3:48 pm

3D world coordinates from stereo calibration

Post by latif »

I want to get a 3D world coordinate of a point in an image from its pixel coordinate. For this, I have studied that we need extrinsic and intrinsic parameters of a stereo setup.
Now, I have successfully done the stereo calibration. I have set two same cameras 14 cm apart and prepare a stereo setup. I have followed the code provided on your forum up to 4_fisheye_calibration.py. I have calibrated stereo camera setup and obtain RMS error of 0.45. and I have saved the projection matrices for both left and right camera. my projection matrix look like
leftProjection matrix
[[1.17125554e+03 0.00000000e+00 4.23688858e+02 0.00000000e+00]
[0.00000000e+00 1.17125554e+03 9.46732804e+01 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 1.00000000e+00 0.00000000e+00]]
rightProjection matrix
[[ 1.17125554e+03 0.00000000e+00 5.48487792e+02 -9.48032081e+03]
[ 0.00000000e+00 1.17125554e+03 9.46732804e+01 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 1.00000000e+00 0.00000000e+00]]

Now I just want to find the 3D world coordinates of any point in the image with known coordinates. I am using these projection matrices and use them in the cv2.triangulatepoints() function. It gives me a 4x1 matrix. This function gives answers in homogenous coordinate. So I convert this in 3D coordinates by dividing the 4th value in this matrix with the first three values of the matrix. and this will give me world coordinates of the specific points according to my knowledge as far as I have gone through this cv2.triangulatepoints() function.

I am getting a difference of almost half value for X, Y and Z coordinates.i.e for the original distance of 3 meters it is only showing 1.5 meters in the result.
I am following his approach (https://stackoverflow.com/questions/461 ... n-2-7-with)

I have attached the picture from my code for the cv2.triangulatepoints() function.

P.S: The problem is the wrong coordinates that I am getting by this approach. Also, I am not sure if I am using the correct method or not to find the 3D world coordinates. I am stuck at this point. I have no idea what to do next

Kindly guide me if I am following the wrong approach
Attachments
COORDINATES.PNG
COORDINATES.PNG (402.9 KiB) Viewed 245 times
Last edited by latif on Wed Oct 28, 2020 7:23 pm, edited 1 time in total.

stereomaton
Posts: 184
Joined: Tue May 21, 2019 12:33 pm
Location: France

Re: 3D world coordinates from stereo calibration

Post by stereomaton »

I have no experience in the domain, but I see that the only input of the function are your calibration matrices and the coordinates of your homologous points.
Thus, I would suspect that the calibration is not good enough.

I suppose that the RMS value you talk about is computed as reprojection error of the calibration points, which denotes that the computed matrices correspond to those points, but were those sufficiently significant? Did you checked the rectified images?
I suppose that you use the coordinates in the rectified images, do you also use the matrices corrected for the new projection?
With a rough calculus, using a 1171px focal, your points seem too far apart for the 14cm/3m pair you gave, or the base is bigger or the object is closer. Also, the "center" terms of the matrices look a bit suspect to me, but not sure.
Stereophotographer and hacker
Despite my quite active participation in the forum, I am not in the StereoPi team
StereoPi Standard Edition + CM3Lite module + a few cameras

User avatar
Realizator
Site Admin
Posts: 600
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: 3D world coordinates from stereo calibration

Post by Realizator »

Hi Latif and Stereomaton,
Stereomaton, thank you for your reply!
Latif, it will be easier for me to use the same data you are using, to compare our results step-by-step.
Can you please share your set of calibration images? I will try to follow all your steps from the very beginning.
You can share a link to your files here, or send it to me in a direct message (email to my nickname @ gmail.com).
Eugene a.k.a. Realizator

latif
Posts: 6
Joined: Thu Oct 08, 2020 3:48 pm

Re: 3D world coordinates from stereo calibration

Post by latif »

Thank You for your response Stereomaton and realizator
Realizator, here is the google drive link my folder. it contains everything that I have tried. and please read the ( PLEASE READ THIS FILE.txt). I have mentioned everything in detail here.

https://drive.google.com/file/d/1foS7qx ... sp=sharing

Thanks

User avatar
Realizator
Site Admin
Posts: 600
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: 3D world coordinates from stereo calibration

Post by Realizator »

Hi Latif,
I got your files. Please give me some time to review and your data. I'll be back with my answer soon!
Eugene a.k.a. Realizator

latif
Posts: 6
Joined: Thu Oct 08, 2020 3:48 pm

Re: 3D world coordinates from stereo calibration

Post by latif »

Sure Realizator.. please take your time. I am eagerly waiting for your response.

Thanking you in anticipation!

Post Reply