Page 1 of 1

3D world coordinates from stereo calibration

Posted: Wed Oct 28, 2020 4:28 pm
by latif
I want to get a 3D world coordinate of a point in an image from its pixel coordinate. For this, I have studied that we need extrinsic and intrinsic parameters of a stereo setup.
Now, I have successfully done the stereo calibration. I have set two same cameras 14 cm apart and prepare a stereo setup. I have followed the code provided on your forum up to I have calibrated stereo camera setup and obtain RMS error of 0.45. and I have saved the projection matrices for both left and right camera. my projection matrix look like
leftProjection matrix
[[1.17125554e+03 0.00000000e+00 4.23688858e+02 0.00000000e+00]
[0.00000000e+00 1.17125554e+03 9.46732804e+01 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 1.00000000e+00 0.00000000e+00]]
rightProjection matrix
[[ 1.17125554e+03 0.00000000e+00 5.48487792e+02 -9.48032081e+03]
[ 0.00000000e+00 1.17125554e+03 9.46732804e+01 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 1.00000000e+00 0.00000000e+00]]

Now I just want to find the 3D world coordinates of any point in the image with known coordinates. I am using these projection matrices and use them in the cv2.triangulatepoints() function. It gives me a 4x1 matrix. This function gives answers in homogenous coordinate. So I convert this in 3D coordinates by dividing the 4th value in this matrix with the first three values of the matrix. and this will give me world coordinates of the specific points according to my knowledge as far as I have gone through this cv2.triangulatepoints() function.

I am getting a difference of almost half value for X, Y and Z coordinates.i.e for the original distance of 3 meters it is only showing 1.5 meters in the result.
I am following his approach ( ... n-2-7-with)

I have attached the picture from my code for the cv2.triangulatepoints() function.

P.S: The problem is the wrong coordinates that I am getting by this approach. Also, I am not sure if I am using the correct method or not to find the 3D world coordinates. I am stuck at this point. I have no idea what to do next

Kindly guide me if I am following the wrong approach

Re: 3D world coordinates from stereo calibration

Posted: Thu Oct 29, 2020 9:16 am
by stereomaton
I have no experience in the domain, but I see that the only input of the function are your calibration matrices and the coordinates of your homologous points.
Thus, I would suspect that the calibration is not good enough.

I suppose that the RMS value you talk about is computed as reprojection error of the calibration points, which denotes that the computed matrices correspond to those points, but were those sufficiently significant? Did you checked the rectified images?
I suppose that you use the coordinates in the rectified images, do you also use the matrices corrected for the new projection?
With a rough calculus, using a 1171px focal, your points seem too far apart for the 14cm/3m pair you gave, or the base is bigger or the object is closer. Also, the "center" terms of the matrices look a bit suspect to me, but not sure.

Re: 3D world coordinates from stereo calibration

Posted: Thu Oct 29, 2020 11:26 am
by Realizator
Hi Latif and Stereomaton,
Stereomaton, thank you for your reply!
Latif, it will be easier for me to use the same data you are using, to compare our results step-by-step.
Can you please share your set of calibration images? I will try to follow all your steps from the very beginning.
You can share a link to your files here, or send it to me in a direct message (email to my nickname @

Re: 3D world coordinates from stereo calibration

Posted: Fri Oct 30, 2020 6:53 am
by latif
Thank You for your response Stereomaton and realizator
Realizator, here is the google drive link my folder. it contains everything that I have tried. and please read the ( PLEASE READ THIS FILE.txt). I have mentioned everything in detail here. ... sp=sharing


Re: 3D world coordinates from stereo calibration

Posted: Mon Nov 02, 2020 12:02 pm
by Realizator
Hi Latif,
I got your files. Please give me some time to review and your data. I'll be back with my answer soon!

Re: 3D world coordinates from stereo calibration

Posted: Tue Nov 03, 2020 6:09 am
by latif
Sure Realizator.. please take your time. I am eagerly waiting for your response.

Thanking you in anticipation!