Function Reference

get_extrinsics

Get extrinsics function calibrates the extrinsic parameters (rotation and translation of cameras in the world space). It takes the video capture object generated by OpenCV and details about the chessboard being detected. Once specified the 3D co-ordinates of the chessboard pattern edges, the cameras get R and t values by treating the chessboard as world origin.

Note: The chessboard pattern should be visible in all cameras in the scene.

  • Low pass filter: The detected points of chessboard pattern are passed through a low pass filter to reduce temporal noise. (Exponential averaging)

  • Initialization: Position of cameras is initialised by triangulation and solvePnP.

  • Bundle adjustment: Extrinsics are further refined by minimizing reprojection error.

The optimised extrinsics are stored in the camera object.

triangulate

Once the cameras have been calibrated, the triangulation function can be used to get the 3D position any point in the scene. Given 2D camera points between two or more cameras, the 3D position can be calculated using the pinhole camera model. Preserving the order of the cameras, we match points between the cameras by epipolar geometry.

  • Low pass filter: The detected points of chessboard pattern are passed through a low pass filter to reduce temporal noise. (Exponential averaging)

  • Initialization: 3D position is calculated pairwise between 2 cameras detecting same number of points.

    1. Undistort Points: The points are undistorted before triangulation using camera Intrinsics.

    2. Triangulation: The 3D position can be calculated using DLT (Direct Linear Transforms). # https://temugeb.github.io/computer_vision/2021/02/06/direct-linear-transorms.html

  • Bundle adjustment: 3D points are further refined by minimizing reprojection error simultaneously between all cameras detecting the points.

get_points_parallel

Given a list of OpenCV camera capture object, this function calls all of them in parallel to get images. The images are passed through a detector which returns feature points. Features can be: * IR LEDs * Aruco markers * Chessboard pattern * Any other feature

cost_func

Calculated reprojection errors from the triangulated 3D points and the detected 2D points. The 3D points are projected to 2D using the camera intrinsics and extrinsics. The difference in the pixel values is passed through L2 Norm to measure error.