各相机模型(针孔+鱼眼)综述
[TOC]
Overview
Lens Projections
 About the various projections of the photographic objective lenses
 Models for the various classical lens projections
 鱼眼相机成像、校准和拼接(笔记)
 Computer Generated Angular Fisheye Projections
 Perspective and fisheye imaging process
Optics: Terminology
 Dioptric: All elements are refractive (lenses)
 Catoptric: All elements are reflective (mirrors)
 Catadioptric: Elements are refractive and reflective (mirrors + lenses)
Camera Models
Papers
 Straight Lines Have to be Straight: Automatic Calibration and Removal of Distortion from Scenes of Structured Environments
 A Generic Camera Model and Calibration Method for Conventional, WideAngle, and FishEye Lenses
 Single View Point Omnidirectional Camera Calibration from Planar Grids
 The Double Sphere Camera Model
Related Code
 Distortion Models in ROS (distortion_models.h)
 plumb_bob: a 5parameter polynomial approximation of radial and tangential distortion
 rational_polynomial: an 8parameter rational polynomial distortion model
 equidistant (lunar)
 hengli/camodocal
 Pinhole camera model
 Unified projection model
 Equidistant fisheye model

uzhrpg/rpg_vikit/vikit_common: support pinhole, atan and omni camera

ethzasl/aslam_cv2: support pinhole and unified peojection and radtan, fisheye and equidistant distortion

cggos/okvis_cg: support pinhole peojection and radtan and equidistant distortion
 ptam_cg/src/ATANCamera.cc: support ATAN camera model
Pinhole camera
 pinhole model (rectilinear projection model) + radialtangential distortion
The Pinhole camera model is the most common camera model for consumer cameras. In this model, the image is mapped onto a plane through perspective projection. The projection is defined by the camera intrinsic parameters such as focal length, principal point, aspect ratio, and skew.
Fisheye camera
OpenCV fisheye camera model
 pinhole model (rectilinear projection model) + fisheye distortion
The Fisheye camera model is a camera model utilized for wide field of view cameras. This camera model is neccessary because the pinhole perspective camera model is not capable of modeling image projections as the field of view approaches 180 degrees.
Given a point $ X=[x_c \quad y_c \quad z_c] $ from the camera $z_c=1$ plane in camera coordinates, the pinhole projection is:
\[\begin{cases} r = \sqrt{x_c^2 + y_c^2} \\ \theta = atan2(r, z_c) = atan2(r, 1) = atan(r) \end{cases}\]in another way
\[f = r' \cdot tan(\theta) \quad \text{where} \quad r' = \sqrt{u^2 + v^2}\]fisheye distortion:
\[\theta_d = \theta (1 + k1 \cdot \theta^2 + k2 \cdot \theta^4 + k3 \cdot \theta^6 + k4 \cdot \theta^8)\]The distorted point coordinates are
\[\begin{cases} x_d = \frac{\theta_d \cdot x_c} {r} \\ y_d = \frac{\theta_d \cdot y_c} {r} \end{cases}\]convert into pixel coordinates, the final pixel coordinates vector
\[\begin{cases} u = f_x (x_d + \alpha y_d) + c_x \\ v = f_y \cdot y_d + c_y \end{cases}\]write in matrix form
\[\begin{aligned} \left[\begin{array}{c}u\\v\\1\end{array}\right] = \left[\begin{array}{ccc} f_x & \alpha & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{array}\right] \left[\begin{array}{c}x_d\\y_d\\1\end{array}\right] \end{aligned}\]ATAN model
 pinhole model (rectilinear projection model) + FOV distortion
 ptam_cg/src/ATANCamera.cc
This is an alternative representation for camera models with large radial distortion (such as fisheye cameras) where the distance between an image point and principal point is roughly proportional to the angle between the 3D point and the optical axis. This camera model is first proposed in Straight Lines Have to be Straight: Automatic Calibration and Removal of Distortion from Scenes of Structured Environments.
Given a point $ X=[x_c \quad y_c \quad z_c] $ from the camera $z_c=1$ plane in camera coordinates, the pinhole projection is:
\[r = \sqrt{\frac{x_c^2 + y_c^2}{z_c^2}} = \sqrt{x_c^2 + y_c^2}\]FOV distortion:
\[r_d = \frac{1}{\omega}arctan( 2 \cdot r \cdot tan(\frac{\omega}{2}) )\]where $\omega$ is the FOV distortion coefficient
The distorted point coordinates are
\[\begin{cases} x_d = \frac{r_d}{r} \cdot x_c \\ y_d = \frac{r_d}{r} \cdot y_c \end{cases}\]convert into pixel coordinates, the final pixel coordinates vector
\[\begin{aligned} \left[\begin{array}{c}u\\v\\1\end{array}\right] = \left[\begin{array}{ccc} f_x & 0 & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{array}\right] \left[\begin{array}{c}x_d\\y_d\\1\end{array}\right] \end{aligned}\]