Trouble with stereoPi calibration

OpenCV, Python and other image processing questions
Post Reply
sirperfluous
Posts: 1
Joined: Sat Apr 18, 2020 7:17 am

Trouble with stereoPi calibration

Post by sirperfluous »

So I am new to open CV and stereoPi, and I was trying to get a basic depth map calibration set up by following along with the blog posts. After a few tries I think I got pretty good calibration results and decent looking rectified images, but I am still having trouble getting a good looking depth map.

https://imgur.com/a/WW9UEN6
Link to my files: https://drive.google.com/open?id=1yzseJ ... o7rHVGvGnL

Any advice for how to improve my depth map?

User avatar
Realizator
Site Admin
Posts: 900
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Trouble with stereoPi calibration

Post by Realizator »

Hi sirperfluous ,
Sorry for my late answer. Actually I downloaded your image set, and calibrated it with our newer code (https://github.com/realizator/stereopi-fisheye-robot) with adjusted settings. And I recommend you to use this code instead of old code from "https://github.com/realizator/stereopi-tutorial". Looks like we need to merge both codes and do a single updated code.
So while calibrating your image I faced "assertion failed" problem described in our article.
And I fell into a rabbit hole:
- investigated this calibration issue on our forum
- contacted the author of this answer
- found 6 yrs old commit to the OpenCV (fisheye calibration)
- asked a question to the author of this commit on Github
- got answer with the links to the original Matlab logic, ported to OpenCV
- installed Matlab and to get info from the help file
- investigation in a process now...

At this moment I recommend you to use new code and capture new calibration images (with default settings). If you will try to calibrate current set - please remove scene_640x240_15.png file (it causes assertion fail), and add code from this comment, as several more images has "flipped checkerboard" problem.
Eugene a.k.a. Realizator

stereomaton
Posts: 215
Joined: Tue May 21, 2019 12:33 pm
Location: France

Re: Trouble with stereoPi calibration

Post by stereomaton »

[Suggestions for realizator]
In the linked function, the vector ekk looks like a vector of the absolute projection errors (I do not remember which projections are supposed to be done, I suppose one is left points on right image given the current parameters relating the two cameras, but for sure ekk concatenates the errors of two projections).
The assert makes the function fail when this error is large for at least one point. I do not understand the math behind this function at first glance, but I guess that ekk should logically be close to zero, except if there are bad inputs in the first place.
I think it would be impractical to try to find errors based on this metrics, you better check the input data another way.

In particular, I would guess that if you drew the detected chessboard with cv2.drawChessboardCorners, you would see either bad detection or detected points that are not homologous in the two images (rotated, skipped corner or whatever). You might see errors in other pairs too, but if the resulting error is small enough it might just screw up the computed parameters silently.

In stereomaton v2, if ever I manage to find time to work on this project, I planned to use ChArUco board to define the points used in the stereo calibration instead. This board has oriented, numbered and encoded tags so that each corner has a unique identifier whatever the orientation of the pattern relative to the camera (works even if a part of the board is cut). It also has large corners where subpixel position refinement can be computed. A prior check of the points can be done based on their ID (is this ID detected on both images of this pair) which, in my opinion, will remove most of the errors that several users reported in the forum.
See https://docs.opencv.org/trunk/df/d4a/tu ... ction.html
Stereophotographer and hacker
Despite my quite active participation in the forum, I am not in the StereoPi team
StereoPi (v1) Standard Edition + CM3Lite module + a few cameras

Post Reply