I want to get a 3D world coordinate of a point in an image from its pixel coordinate. For this, I have studied that we need extrinsic and intrinsic parameters of a stereo setup.
Now, I have successfully done the stereo calibration. I have set two same cameras 14 cm apart and prepare a stereo setup. I have followed the code provided on your forum up to 4_fisheye_calibration.py. I have calibrated stereo camera setup and obtain RMS error of 0.45. and I have saved the projection matrices for both left and right camera. my projection matrix look like
leftProjection matrix
[[1.17125554e+03 0.00000000e+00 4.23688858e+02 0.00000000e+00]
[0.00000000e+00 1.17125554e+03 9.46732804e+01 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 1.00000000e+00 0.00000000e+00]]
rightProjection matrix
[[ 1.17125554e+03 0.00000000e+00 5.48487792e+02 9.48032081e+03]
[ 0.00000000e+00 1.17125554e+03 9.46732804e+01 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 1.00000000e+00 0.00000000e+00]]
Now I just want to find the 3D world coordinates of any point in the image with known coordinates. I am using these projection matrices and use them in the cv2.triangulatepoints() function. It gives me a 4x1 matrix. This function gives answers in homogenous coordinate. So I convert this in 3D coordinates by dividing the 4th value in this matrix with the first three values of the matrix. and this will give me world coordinates of the specific points according to my knowledge as far as I have gone through this cv2.triangulatepoints() function.
I am getting a difference of almost half value for X, Y and Z coordinates.i.e for the original distance of 3 meters it is only showing 1.5 meters in the result.
I am following his approach (https://stackoverflow.com/questions/461 ... n27with)
I have attached the picture from my code for the cv2.triangulatepoints() function.
P.S: The problem is the wrong coordinates that I am getting by this approach. Also, I am not sure if I am using the correct method or not to find the 3D world coordinates. I am stuck at this point. I have no idea what to do next
Kindly guide me if I am following the wrong approach
3D world coordinates from stereo calibration
3D world coordinates from stereo calibration
 Attachments

 COORDINATES.PNG (402.9 KiB) Viewed 415 times
Last edited by latif on Wed Oct 28, 2020 7:23 pm, edited 1 time in total.

 Posts: 184
 Joined: Tue May 21, 2019 12:33 pm
 Location: France
Re: 3D world coordinates from stereo calibration
I have no experience in the domain, but I see that the only input of the function are your calibration matrices and the coordinates of your homologous points.
Thus, I would suspect that the calibration is not good enough.
I suppose that the RMS value you talk about is computed as reprojection error of the calibration points, which denotes that the computed matrices correspond to those points, but were those sufficiently significant? Did you checked the rectified images?
I suppose that you use the coordinates in the rectified images, do you also use the matrices corrected for the new projection?
With a rough calculus, using a 1171px focal, your points seem too far apart for the 14cm/3m pair you gave, or the base is bigger or the object is closer. Also, the "center" terms of the matrices look a bit suspect to me, but not sure.
Thus, I would suspect that the calibration is not good enough.
I suppose that the RMS value you talk about is computed as reprojection error of the calibration points, which denotes that the computed matrices correspond to those points, but were those sufficiently significant? Did you checked the rectified images?
I suppose that you use the coordinates in the rectified images, do you also use the matrices corrected for the new projection?
With a rough calculus, using a 1171px focal, your points seem too far apart for the 14cm/3m pair you gave, or the base is bigger or the object is closer. Also, the "center" terms of the matrices look a bit suspect to me, but not sure.
Stereophotographer and hacker
Despite my quite active participation in the forum, I am not in the StereoPi team
StereoPi Standard Edition + CM3Lite module + a few cameras
Despite my quite active participation in the forum, I am not in the StereoPi team
StereoPi Standard Edition + CM3Lite module + a few cameras
 Realizator
 Site Admin
 Posts: 603
 Joined: Tue Apr 16, 2019 9:23 am
 Contact:
Re: 3D world coordinates from stereo calibration
Hi Latif and Stereomaton,
Stereomaton, thank you for your reply!
Latif, it will be easier for me to use the same data you are using, to compare our results stepbystep.
Can you please share your set of calibration images? I will try to follow all your steps from the very beginning.
You can share a link to your files here, or send it to me in a direct message (email to my nickname @ gmail.com).
Stereomaton, thank you for your reply!
Latif, it will be easier for me to use the same data you are using, to compare our results stepbystep.
Can you please share your set of calibration images? I will try to follow all your steps from the very beginning.
You can share a link to your files here, or send it to me in a direct message (email to my nickname @ gmail.com).
Eugene a.k.a. Realizator
Re: 3D world coordinates from stereo calibration
Thank You for your response Stereomaton and realizator
Realizator, here is the google drive link my folder. it contains everything that I have tried. and please read the ( PLEASE READ THIS FILE.txt). I have mentioned everything in detail here.
https://drive.google.com/file/d/1foS7qx ... sp=sharing
Thanks
Realizator, here is the google drive link my folder. it contains everything that I have tried. and please read the ( PLEASE READ THIS FILE.txt). I have mentioned everything in detail here.
https://drive.google.com/file/d/1foS7qx ... sp=sharing
Thanks
 Realizator
 Site Admin
 Posts: 603
 Joined: Tue Apr 16, 2019 9:23 am
 Contact:
Re: 3D world coordinates from stereo calibration
Hi Latif,
I got your files. Please give me some time to review and your data. I'll be back with my answer soon!
I got your files. Please give me some time to review and your data. I'll be back with my answer soon!
Eugene a.k.a. Realizator
Re: 3D world coordinates from stereo calibration
Sure Realizator.. please take your time. I am eagerly waiting for your response.
Thanking you in anticipation!
Thanking you in anticipation!