[TOC]

# Overview

## Lens Projections

• Perspective and fisheye imaging process

## Optics: Terminology

• Dioptric: All elements are refractive (lenses)
• Catoptric: All elements are reflective (mirrors)
• Catadioptric: Elements are refractive and reflective (mirrors + lenses)

## Papers

• Straight Lines Have to be Straight: Automatic Calibration and Removal of Distortion from Scenes of Structured Environments
• A Generic Camera Model and Calibration Method for Conventional, Wide-Angle, and Fish-Eye Lenses
• Single View Point Omnidirectional Camera Calibration from Planar Grids
• The Double Sphere Camera Model
• Supported models in Kalibr

• Distortion Models in ROS (distortion_models.h)
• plumb_bob: a 5-parameter polynomial approximation of radial and tangential distortion
• rational_polynomial: an 8-parameter rational polynomial distortion model
• equidistant (lunar)
• hengli/camodocal
• Pinhole camera model
• Unified projection model
• Equidistant fish-eye model
• uzh-rpg/rpg_vikit/vikit_common: support pinhole, atan and omni camera

• ethz-asl/aslam_cv2: support pinhole and unified peojection and radtan, fisheye and equidistant distortion

• cggos/okvis_cg: support pinhole peojection and radtan and equidistant distortion

• ptam_cg/src/ATANCamera.cc: support ATAN camera model

# Pinhole camera

• pinhole model (rectilinear projection model) + radial-tangential distortion

The Pinhole camera model is the most common camera model for consumer cameras. In this model, the image is mapped onto a plane through perspective projection. The projection is defined by the camera intrinsic parameters such as focal length, principal point, aspect ratio, and skew.

# Fisheye camera

## OpenCV fisheye camera model

• pinhole model (rectilinear projection model) + fisheye distortion

The Fisheye camera model is a camera model utilized for wide field of view cameras. This camera model is neccessary because the pinhole perspective camera model is not capable of modeling image projections as the field of view approaches 180 degrees.

Given a point $X=[x_c \quad y_c \quad z_c]$ from the camera $z_c=1$ plane in camera coordinates, the pinhole projection is:

$\begin{cases} r = \sqrt{x_c^2 + y_c^2} \\ \theta = atan2(r, |z_c|) = atan2(r, 1) = atan(r) \end{cases}$

in another way

$f = r' \cdot tan(\theta) \quad \text{where} \quad r' = \sqrt{u^2 + v^2}$

fisheye distortion:

$\theta_d = \theta (1 + k1 \cdot \theta^2 + k2 \cdot \theta^4 + k3 \cdot \theta^6 + k4 \cdot \theta^8)$

The distorted point coordinates are

$\begin{cases} x_d = \frac{\theta_d \cdot x_c} {r} \\ y_d = \frac{\theta_d \cdot y_c} {r} \end{cases}$

convert into pixel coordinates, the final pixel coordinates vector

$\begin{cases} u = f_x (x_d + \alpha y_d) + c_x \\ v = f_y \cdot y_d + c_y \end{cases}$

write in matrix form

\begin{aligned} \left[\begin{array}{c}u\\v\\1\end{array}\right] = \left[\begin{array}{ccc} f_x & \alpha & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{array}\right] \left[\begin{array}{c}x_d\\y_d\\1\end{array}\right] \end{aligned}

## ATAN model

This is an alternative representation for camera models with large radial distortion (such as fisheye cameras) where the distance between an image point and principal point is roughly proportional to the angle between the 3D point and the optical axis. This camera model is first proposed in Straight Lines Have to be Straight: Automatic Calibration and Removal of Distortion from Scenes of Structured Environments.

Given a point $X=[x_c \quad y_c \quad z_c]$ from the camera $z_c=1$ plane in camera coordinates, the pinhole projection is:

$r = \sqrt{\frac{x_c^2 + y_c^2}{z_c^2}} = \sqrt{x_c^2 + y_c^2}$

FOV distortion:

$r_d = \frac{1}{\omega}arctan( 2 \cdot r \cdot tan(\frac{\omega}{2}) )$

where $\omega$ is the FOV distortion coefficient

The distorted point coordinates are

$\begin{cases} x_d = \frac{r_d}{r} \cdot x_c \\ y_d = \frac{r_d}{r} \cdot y_c \end{cases}$

convert into pixel coordinates, the final pixel coordinates vector

\begin{aligned} \left[\begin{array}{c}u\\v\\1\end{array}\right] = \left[\begin{array}{ccc} f_x & 0 & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{array}\right] \left[\begin{array}{c}x_d\\y_d\\1\end{array}\right] \end{aligned}

^