realsense d455 semantic_ Slam implements semantic octree mapping

orb_slam2 + Semantic segmentation + Octree map ,
github Address ：https://github.com/floatlazer/semantic_slam

Environment configuration reference link

Now let's talk about using d455 To achieve real-time mapping , What needs to be modified .

First install intel realsense Of sdk,
To realsense-viewer The interface can be displayed successfully

Then install realsense Of ros wrapper, Give Way d455 It can be done by ros send out topic
Installation method
This step requires simultaneous installation ddynamic_reconfigure package , from github Up and down ,
and realsen_ros Put together catkin_ws/src Next make that will do

until roslaunch realsense_camera rs_rgbd.launch It can start

About which depth map to use topic The problem of ：
In some places, it is said that the depth map should be used /camera/depth/image_rect_raw
Some say yes /camera/aligned_depth_to_color/image_raw
（ What specific cameras have been released topic It can be used rostopic list Order confirmation ）
Use it carefully roslaunch realsense_camera rs_camera.launch Starting the camera is not /camera/aligned_depth_to_color/image_raw Of

stay rviz I observed it in the /camera/depth/image_rect_raw and /camera/aligned_depth_to_color/image_raw Two pictures ,
Find out /camera/aligned_depth_to_color/image_raw The depth map of is and rgb Image aligned ,
in other words /camera/depth/image_rect_raw The range of this depth map may be larger than rgb Wider images , If they are stacked together, they may not overlap .
So I chose /camera/aligned_depth_to_color/image_raw,
Of course, different cameras may be different .

To determine the rgb Images /camera/color/image_raw And depth map /camera/aligned_depth_to_color/image_raw after ,
To be modified orb_slam2 Under the ros_rgbd.cc, modify topic name .

Then you have to modify it semantic_slam/semantic_slam/params Under the semantic_cloud.yaml, modify topic
At the same time, we need to modify the semantic network path, namely model_path, To pth Where the papers are

Modify camera internal parameters ：
There are several methods for camera calibration , It can be used ros package ,realsense The order that comes with you , You can also take a few pictures matlab calibration ,
After calibration , modify semantic_slam/semantic_slam/params Under the semantic_cloud.yaml and xtion.yaml,
xtion.yaml The file is boot orb_slam2 Configuration file for .

At this time, the semantic segmentation image can be displayed , But it can't display the octuple map ！
What's the problem ,
You can see that the problem is semantic_cloud.py, The depth map is used directly , It doesn't work DepthMapFactor Conduct scale,
And we can see in orb_slam2 The medium depth map has gone through scale Of

imDepth.convertTo(imDepth,CV_32F,mDepthMapFactor);

We need to do the same thing , stay semantic_cloud.yaml In the definition of DepthMapFactor, And then put semantic_cloud.py The depth map in scale.

Then you can build a map ：