Halcon knowledge: binocular_ Discrimination knowledge

One 、 summary  

        binocular_disparity  function ： The binocular vision difference of two figures is calculated by correlation , Or for ：binocular_disparity A correlation technique is used to calculate the pixel level correspondence between two corrected images . And binocular_distance Different , Results are not converted to distance values .

Two 、 Operator interpretation

binocular_disparity(ImageRect1, ImageRect2 : Disparity, Score : Method, MaskWidth, MaskHeight, TextureThresh, MinDisparity, MaxDisparity, NumLevels, ScoreThresh, Filter, SubDisparity : )

input data ：

The input image ：

• ImageRect1    On the left
• ImageRect2    Right picture

Input control quantity ​：

•  Method,                There are three options   'ncc', 'sad', 'ssd'
• MaskWidth,          Calculate the relevant window size ：5, 7, 9, 11, 21
• MaskHeight,          Calculate the relevant window size ：5, 7, 9, 11, 21
• TextureThresh,      Variance threshold of texture image region .
• MinDisparity,        The minimum value of the expected difference .
• MaxDisparity,        The maximum expected difference .
• NumLevels,          Number of pyramid levels
• ScoreThresh,        Threshold of correlation function
• Filter,                    Downstream filter    'left_right_check', 'none'
• SubDisparity          Subpixel interpolation of parallax ：​'interpolation', 'none'

3、 ... and 、 Necessary condition

         The algorithm needs a reference image ImageRect1 And a search image ImageRect2, They must be corrected , That is, the corresponding kernel lines are parallel and located on the same image line (). If this assumption is violated , You can use the operator calibrate_cameras、gen_binocular_rectification_map and map_image Correct the image .

         therefore , Given the reference image ImageRect1 The pixel , Pass along ImageRect2 Search and match the corresponding rows in the with a size of MaskWidth and MaskHeight The local neighborhood within the rectangular window of ImageRect2 Homologous pixels in . The pixel correspondence is returned in the single channel parallax image , The image is the reference image according to the formula ImageRect1 Every pixel of (r1,c1) Appoint ImageRect2 Appropriate matching pixels (r2,c2). The quality measure of each parallax value is in Score Back in , Contains the matching function of the reference pixel S The best result of . For matching , Use the gray value of the original unprocessed image .

Four 、 Code example

* Set internal and external stereo parameters.
* Note that, typically, these values are the result of a prior
* calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006, \
                                622, 517, 1280, 1024, CamParam1)
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006, \
                                654, 519, 1280, 1024, CamParam2)
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89, \
             'Rp+T', 'gba', 'point', RelPose)

* Compute the mapping for rectified images.
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, RelPose, \
  1, 'viewing_direction', 'bilinear', CamParamRect1,CamParamRect2, Cam1PoseRect1, \
  Cam2PoseRect2,RelPoseRect)

* Compute the disparities in online images.
while (1)
  grab_image_async (Image1, AcqHandle1, -1)
  map_image (Image1, Map1, ImageRect1)

  grab_image_async (Image2, AcqHandle2, -1)
  map_image (Image2, Map2, ImageRect2)

  binocular_disparity(ImageRect1, ImageRect2, Disparity, Score, 'sad', \
    11, 11, 20, -40, 20, 2, 25, 'left_right_check','interpolation')
endwhile

5、 ... and 、 appendix ：3d Reconstructed operator

 binocular_disparity_mg 
function ： The binocular vision difference of two figures is calculated by multi lattice method . 
binocular_disparity_ms 
function ： The parallax of the corrected stereo image pair is calculated by using the multi scan line optimization method . 
binocular_distance 
function ： The binocular vision difference between two stereograms is calculated by correlation . 
binocular_distance_mg 
function ： The binocular vision difference of two stereograms is calculated by multi lattice method . 
binocular_distance_ms 
function ： Multi scan line optimization is used to calculate the distance value of the corrected stereo image pair . 
disparity_image_to_xyz 
function ： Convert a viewing distance into a corrected one 3D Solid point . 
disparity_to_distance 
function ： The video distance conversion is used to correct the interval value in the binocular stereo system . 
disparity_to_point_3d 
function ： Transform an image point and viewing distance into a three-dimensional point in a corrected stereoscopic system . 
distance_to_disparity 
function ： Convert an interval value into a viewing distance in a corrected stereoscopic system . 
essential_to_fundamental_matrix 
function ： Calculate a basic matrix derived from the original matrix . 
gen_binocular_proj_rectification 
function ： Calculate the projection correction value of the image of the weak binocular stereo system . 
gen_binocular_rectification_map_ 
function ： Create transfer graph , It describes the mapping from a binocular camera to an image of a common corrected image surface . 
intersect_lines_of_sight 
function ： Obtain a three-dimensional point from the intersection of two lines in the vision of a binocular camera system . 
match_essential_matrix_ransack 
function ： The original of stereo image pair is calculated by automatically mining the corresponding relationship between image points （ The essence ） matrix . 
match_fundamental_matrix_distortion_ransac 
function ： By automatically finding the corresponding relationship between image points , Calculate the basic matrix and radial distortion coefficient of a pair of stereo images . 
match_fundamental_matrix_ransack 
function ： Calculating the basic matrix of stereo image pair by automatically mining the corresponding relationship between image points . 
match_rel_pose_ransack 
function ： Calculate the relative orientation between two cameras by automatically mining the corresponding relationship between image points . 
reconst3d_from_fundamental_matrix 
function ： Calculate the 3D reconstruction of point projection based on the basic matrix . 
rel_pose_to_fundamental_matrix 
function ： Calculate the basic matrix obtained from the two camera correlation directions . 
vector_to_essential_matrix 
function ： Calculate the mapping between given image points and the original matrix of known camera matrix , Reconstruct 3D points . 
vector_to_fundamental_matrix 
function ： Calculate the basic matrix of the set of mappings between given image points , Reconstruct 3D points . 
vector_to_fundamental_matrix_distortion 
function ： Give a set of image point correspondence , Calculate the basic matrix and radial distortion coefficient , Reconstruct 3D points . 
vector_to_rel_pose 
function ： Calculate the relative orientation of two cameras with given image point correspondence and known camera parameters , Reconstruct 3D points . 

• Depth From Focus Focus depth  

depth_from_focus 
function ： Extracting height by multi focus gray level （ The thickness of the ）. 
select_grayvalues_from_channels 
function ： Use index image to select gray value of a multi-channel image . 

• Multi View Stereo Multi view stereo model  

clear_stereo_model 
function ： Clear the specified multi view stereo model , Free up memory . 
create_stereo_model 
function ： Create a multi view stereo model . 
get_stereo_model_image_pairs 
function ： Returns the list of image pairs set in the stereo model . 
get_stereo_model_object 
function ： Get the intermediate result of stereo reconstruction （ Icon ）. 
get_stereo_model_object_model_3d 
function ： Get the intermediate 3D object model of stereo reconstruction  
get_stereo_model_param 
function ： Extract the parameters of multi view stereo model . 
reconstruct_points_stereo 
function ： Reconstruction of multi view stereo 3D Reference point . 
reconstruct_surface_stereo 
function ： Rebuild multi perspective 3D Surfaces . 
set_stereo_model_image_pairs 
function ： Specify binocular （ Pair ） Surface stereo reconstruction of image . 
set_stereo_model_param 
function ： Set multi view stereo model parameters . 

• Photometric Stereo Estimate the light source  

estimate_al_am 
function ： Estimate the reflectivity of a plane and the number of reflected light . 
estimate_sl_al_lr 
function ： Estimate the inclination of a light source and the reflectivity of a plane . 
estimate_sl_al_zc 
function ： Estimate the inclination of a light source and the reflectivity of a plane . 
estimate_tilt_lr 
function ： Estimate the inclination of a light source . 
estimate_tilt_zc 
function ： Estimate the inclination of a light source . 
photometric_stereo 
function ： Reconstructing surfaces based on photometric stereo technology . 
reconstruct_height_field_from_gradient 
function ： Rebuild surfaces from surface gradients . 
sfs_mod_lr 
function ： Reconstruct a plane from a gray image . 
sfs_orig_lr 
function ： Reconstruct a plane from a gray image . 
sfs_pentland 
function ： Reconstruct a plane from a gray image . 
shade_height_field 
function ： Reconstruct an image of a gray bump on the surface . 
uncalibrated_photometric_stereo 
function ： Reconstructing a surface from several images with different illumination . 

• Sheet of Light Sheet light  

apply_sheet_of_light_calibration 
function ： Apply calibration transform to inconsistent images . 
calibrate_sheet_of_light 
function ： Use 3D Calibration object calibration sheet-of-light Setup table . 
clear_sheet_of_light_model 
function ： Clear the specified sheet light model , Free up memory . 
create_sheet_of_light_calib_object 
function ： Create calibration objects for the light calibration worksheet . 
create_sheet_of_light_model 
function ： Create one based on 3D Measured sheet light model . 
deserialize_sheet_of_light_model 
function ： Deserialization sheet-of-light model. 
get_sheet_of_light_param 
function ： Extract the parameters of the slice light model . 
get_sheet_of_light_result 
function ： Extract a measurement result obtained by the sheet light technique （ Icon ）. 
get_sheet_of_light_result_object_model_3d 
can ： Extract an image obtained by sheet light technique 3D Object model measurement results . 
measure_profile_sheet_of_light 
function ： The input and stored contour images are processed by sheet light technology . 
query_sheet_of_light_params 
function ： For a given sheet-of-light model, Get the names of common icons or control parameters that can be used for different light operators . 
read_sheet_of_light_model 
function ： Read from file sheet-of-light Model , Then create a new model . 
reset_sheet_of_light_model. 
function ： Reset sheet-of-light Model . 
serialize_sheet_of_light_model 
function ： serialize sheet-of-light Model . 
set_profile_sheet_of_light 
function ： By measuring the parallax setting sheet-of-light outline . 
set_sheet_of_light_param 
function ： Set the parameters corresponding to the slice light model . 
write_sheet_of_light_model 
function ： take sheet-of-light Model table write file .