Conversion of world coordinate system, camera coordinate system and image coordinate system

Coordinate transformation

Before, I just stayed at the stage of being able to use , I have never understood the principle of calculation , I read through the article of the boss today , The writing is concise and comprehensive , Thank you thank you ~~ It is hereby recorded that , For personal notes only

Post a link , Thank you very much. ~
https://blog.csdn.net/weixin_44278406/article/details/112986651
https://blog.csdn.net/guyuealian/article/details/104184551

Four different types of coordinate systems

Convert three-dimensional objects into two-dimensional coordinates on photos , It is transformed by four coordinate systems .

1. World coordinate system

The world coordinate system is a special coordinate system , It establishes the reference frame needed to describe other coordinate systems . Can use the world coordinate system to describe the position of other coordinate systems , Instead of using bigger 、 The external coordinate system describes the world coordinate system . In a non-technical sense , The world coordinate system establishes the largest coordinate system we care about , It doesn't have to be the whole world .
use $(X_w,Y_w,Z_w)$ To express , The world coordinate system can get the camera coordinate system through rotation and Translation .

2. Camera coordinate system

At the geometric center of the camera lens （ Light heart ） Origin , The coordinate system satisfies the right-hand rule , use $(X_c,Y_c,Z_c)$ To express ; The optical axis of the camera is in the coordinate system Z Axis ,X Axis horizontal ,Y Vertical axis .

3. Image physical coordinate system

With CCD The center of the image is the origin , Coordinates from $(x,y)$ Express , Unit of image coordinate system , Usually mm , The coordinate origin is the intersection of the camera optical axis and the imaging plane （ In general , This intersection is close to the center of the image ）

CCD, English full name ：Charge coupled Device, Chinese full name ： Charge coupled device , Can be called CCD image sensor .CCD It's a semiconductor device , It can convert optical images into digital signals . CCD The tiny photosensitive material implanted on the is called pixel （Pixel）. A piece of CCD The more pixels it contains , The higher the picture resolution it provides .

4. Image pixel coordinate system

Actually , When we mention an image , It usually refers to the pixel coordinate system of the image . The origin of the pixel coordinate system is in the upper left corner , And the unit is pixels .

Set the origin of the image coordinate system $O_1$ , Convert to $O_0$ In the coordinate system of the origin . Reason for use ：

• If you use the image coordinate system , Company mm, In fact, it is not easy to measure specific images , If according to the unified pixel standard , It is easier to measure the quality of the image
• If you use the image coordinate system , Then there are four quadrants , There will be a problem of positive and negative numbers , But after converting to pixel coordinate system , All are integers . In subsequent operations and operations , Are simplified a lot .

Coordinate transformation

Pinhole Model （The basic pinhole model）. This model is mathematically from three-dimensional space to two-dimensional plane （image plane or focal plane） The central projection of , By a $3\times 4$ Projection matrix $P=K[R|t]$ To describe ,$K$ For camera internal reference （internal camera parameters）,$[R|t]$ For external reference （external parameters）.

World coordinates → Camera coordinates （ Rigid transformation ）

$$\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\\ 1\end{array}\right]=\left[\begin{array}{cc}R& t\\ & \\ 0_{1\ast 3}& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]$$
$X_c,Y_c,Z_c$ Represents camera coordinates ;$X_w,Y_w,Z_w$ Represents world coordinates ;R Represents the orthogonal unit rotation matrix ,t Represents three-dimensional translation vector .
According to the rotation angle, the rotation matrix in three directions can be obtained , The rotation matrix is their product ：$R=R_x\ast R_y\ast R_z$
By the way, record the formula of three rotation matrices , Often forget .

Around the $X$ rotate $\theta$ degree

$$\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right]=\left[\begin{array}{ccc}1& 0& 0\\ 0& cos\theta & sin\theta \\ 0& -sin\theta & cos\theta \end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]=R_x\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]$$

Around the $Y$ Shaft rotation $\theta$ degree

$$\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right]=\left[\begin{array}{ccc}cos\theta & 0& -sin\theta \\ 0& 1& 0\\ sin\theta & 0& cos\theta \end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]=R_y\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]$$

Around the $Z$ Shaft rotation $\theta$ degree

$$\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right]=\left[\begin{array}{ccc}cos\theta & sin\theta & 0\\ -sin\theta & cos\theta & 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]=R_z\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]$$

Camera coordinates → Image coordinate system （ Center of the projection ）

The relationship between camera coordinate system and image coordinate system is perspective , Use similar triangles to calculate .

The matrix written in homogeneous coordinates is multiplied by
$$Zc\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\\ 1\end{array}\right]=\left[\begin{array}{c}K|0\end{array}\right]\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\\ 1\end{array}\right]$$
among f Representative focal length , That is, the camera coordinate system and image coordinate system are in Z The difference on the shaft . Now the projection point p The unit is still mm, Not at all pixel, It is not convenient for subsequent operations .

Image coordinate system → Pixel coordinate system （ discretization ）

The origin of the pixel coordinate system is in the upper left corner , And the unit is pixels . Both the pixel coordinate system and the image coordinate system are on the imaging plane , It's just that their origins and units of measurement are different . The origin of the image coordinate system is the intersection of the camera optical axis and the imaging plane , Usually it's the midpoint of the imaging plane, or principal point. The unit of the image coordinate system is mm, Belongs to a physical unit , And the unit of the pixel coordinate system is pixel, We usually describe a pixel as several rows and columns . So the conversion between the two is as follows ： among dx and dy Represents how much each column and row represents mm, namely 1pixel=dx mm

$$Zc\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}\frac{1}{d_x}& 0& u_0\\ 0& \frac{1}{d_y}& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ & \\ 0_{1\ast 3}& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]$$
among $\left[\begin{array}{ccc}\frac{1}{d_x}& 0& u_0\\ 0& \frac{1}{d_y}& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]$ Is the camera internal parameter matrix ,$\left[\begin{array}{cc}R& t\\ & \\ 0_{1\ast 3}& 1\end{array}\right]$ External parameter matrix . Camera calibration is to solve the parameters of these two matrices .