ORB_ VI ideological framework

According to the tutorial https://blog.csdn.net/wangshuailpp/article/details/78461171 From sorting

------ Tutorials are really often read and often new -------------------------------------------

Ideological framework

One .IMU part

1. Why IMU Preintegration ？
   answer ：IMU The real significance of the predicted score lies in that the result is a short period of time （tk～tk+1）IMU True rotation , Changes in displacement and velocity , and IMU The amount of change in a short time is very accurate . Using this accurate data and visual information for tight coupling can Improve VIO And eliminate monocular scale uncertainty .
2. IMU How to use the quantity calculated by pre integration , What does it do ？？
   answer ： In the paper Derived from a given integral form There are many unknowns in the pre integration , I can't use it directly , So the approach is ： Separate the position variables in the pre integration from the pre integration formula , Its size is not directly calculated in the optimization or filtering algorithm , But through Information matrix Attach weights to these unknown variables , and The information matrix is obtained from the covariance matrix .
   stay SLAM in , General probability models are modeled as Gaussian distribution , For example, the pose of the camera is a Gaussian distribution , The trajectory and characteristic points form a multivariate Gaussian distribution p(x1,x2,x3,pt1…), Then graph optimization or BA From a probability problem to a least square problem . To separate some variables from the multivariate Gaussian distribution , In fact, covariance matrix is well separated , The hard part is the information matrix （ Inverse of covariance ）, Because we are usually in SLAM What I know is H matrix , Not the covariance matrix . about H Separation of matrices , Need to use schur complement To segment . therefore , The correct way to remove a variable from the multivariate Gaussian distribution is to remove it from the multivariate Gaussian distribution marginalize out.（ source , Heyijia explains marginalization ：https://blog.csdn.net/heyijia0327/article/details/52822104）
   # summary ： Preintegration –》 Unknown variable –》 The information matrix gives weight to unknown variables –》 Information matrix –》 Covariance matrix
   So the key is how to calculate the covariance matrix
3. IMU What does the pre integral calculate ？
   answer ：IMU The function of pre integration is to calculate
   （1）IMU Observations of data （ Namely IMU The premixed value ）, Visual observations are used to calculate residuals （ That's the error ）
   （2） Covariance matrix of residuals , Assign corresponding weights to unknown variables through observed values （ Because there are many observations ）
   （3） Jacobian matrix , Jacobian matrix of residual error is the descending direction in optimization （ That's the gradient ） These three quantities
4. IMU What effect does the quantity calculated by pre integration play ？？
   answer ： For specific use , These three quantities are the following Union initialization provides initial values And back-end optimization Provide IMU The constraints of .

One .VIO Initialization part

1. The reason for our initialization is that the monocular inertial close coupled system is a highly nonlinear system , First of all, monocular is unable to obtain the absolute scale in space , and IMU And there must be bias , The gravitational acceleration is also needed in the later solution （ Including size and direction ）, Under speed sensitive conditions , Like drones , And accurate speed information , therefore , How to effectively calculate these quantities before processing the tightly coupled system , It is of great significance to the robustness of the whole tightly coupled system （ In fact, it can be understood as camera calibration , The internal parameters of the camera are not calibrated correctly , The camera must be inaccurate when positioning , And it's likely to hang up ）. So the initialization is actually very simple , Is to calculate the absolute scale s、 Gyroscope bias bg、 Acceleration offset ba、 Acceleration of gravity G And each IMU The speed of the moment v,VINS It is emphasized that the accelerometer offset value is generally coupled with the gravitational acceleration （ That is, it is absorbed by the acceleration of gravity ）, The magnitude of gravitational acceleration is much larger than its acceleration offset , And the accelerometer offset is relatively small in the initialization time , It's hard to really calculate , Therefore, the effect of accelerometer bias is ignored , No more calculations during initialization . The function of initialization is self-evident , It directly affects the robustness and positioning accuracy of the whole tightly coupled system , And initialization usually takes a long time ,VINS It will take about ten seconds ,ORB_SLAM2 combination IMU The time is set at 15 Seconds to complete initialization .

2. Pure visual initialization in the V Dot A part , First build a sliding window , Contains a set of data frames . The use of antipolar geometry model is mentioned in the paper 5 The point method is used to solve the relative transformation of monocular camera , Including relative rotation and displacement without scale information . In fact, each monocular model basically uses epipolar geometry to solve the relative transformation of two frames in initialization , It should be noted that rotation is scale invariant （ It's actually unit rotation , There will be no scale information , Think about it carefully ？）, As for why monocular has no scale information , I want to say a long word , But in fact, many people learn monocular vision SLAM None of them really understand , The source of monocular vision without scale is the polar geometry between the first two frames , To be more specific is to ask F/E perhaps H It needs to be lowered to a lower parameter ,F from 6 Dimension down to 5 dimension ,H from 9 Dimension down to 8 dimension , The dimension here is the scale , And it must be lowered . Then triangulate to get the corresponding 3d Point coordinates , There are these 3d Points and other frames in the sliding window 2d Click to proceed PNP Solve to obtain all the pose and feature points in the sliding window 3d coordinate , thus , Visual initialization is complete . Is it simple ？ Of course. , After all, it's just a simple visual initialization , What is really complicated is the visual inertia joint initialization , That is, the focus and difficulty of our initialization .

3. Visual inertia joint initialization

Visual inertia joint initialization at V Dot B part , The name of the definition given by the author here is Visual-Inertia Alignment, That is, visual inertia joint initialization （ And in the ORBSLAM2+IMU In my paper , The name defined by the author is IMU initialization, namely IMU initialization ）, Why define such a noun , I think it has two meanings , First, when initializing the gyro offset, it is necessary to use IMU Rotation of measurement and rotation of visual measurement , That is to combine visual and inertial data . Second, the scale obtained here S The value of is more than IMU Of , Or visual and IMU The scale of the whole system . During the specific explanation of each initialization process , It is necessary to make a general summary , The physical definition of initialization is actually the calibration of intrinsic parameters , The definition in the mathematical model is actually a formula （6） Solution of matrix equation , And the formula （6） In fact, it comes from the most primitive PVQ The integral formula , among Q The rotation corresponds to the gyroscope , and PV Corresponding to the accelerometer .