Depth Map-Based Object Detection

In the past few days, I have been in touch with the project of the car moving blocks. The visual algorithm is based on opencv. Although the method is simple, the effect is very nice. Today, I will make a record of the team's algorithm.

Method principle: frame difference detection, the depth map with the detection target and the background image without the target are compared to obtain the position of the target of interest.The specific steps are as follows,

1. Image reading

import cv2import numpy as npref_path = 'C:\Users/15989\Desktop\imgs/1.png'cur_path = 'C:\Users/15989\Desktop\imgs/201.png'cur_img = cv2.imread(cur_path, -1)ref_img = cv2.imread(ref_path, -1)

Because the image is a 16-bit depth map, the image is normalized for easy viewing. The operation is as follows:

def norm(img):img = (img - img.min()) / (img.max() - img.min())img *= 255return img

Read the image as follows (background image/target image):

2. Make frame difference

diff_frame = cv2.absdiff(cur_img, ref_img)

3. Select a region of interest

diff_frame_roi = diff_frame[200:, 200:540]

4. Convert the image into a binary image

frame_bin = cv2.inRange(diff_frame_roi, 20, 100)

5. Closure operation (expand first, then corrode) to eliminate small black holes

kernel = np.ones((3, 3), dtype=np.uint8)closing = cv2.morphologyEx(frame_bin, cv2.MORPH_CLOSE, kernel)

6. Find the target contour

contours, _ = cv2.findContours(closing, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

7. Find the minimum circumscribed circle containing the contour and find the corresponding radius, filter the contour radius, calculate the minimum circumscribed rectangle of the contour that meets the requirements, filter the width and height of the circumscribed rectangle and their ratio, and select the appropriate sizeThe goal of the (parameter tuning is mainly concentrated in this link)

x_obj, y_obj = 0, 0w, h = 0, 0xl, yl = [], []wl, hl = [], []for cont in contours:(x, y), radius = cv2.minEnclosingCircle(cont)if radius > 13 and radius < 28:rect = cv2.minAreaRect(cont)x_obj, y_obj = rect[0]w, h = rect[1]if w/h > 1.5 or w/h < 0.5 or w*h < 500:continueelse:xl.append(x_obj)yl.append(y_obj)wl.append(w)hl.append(h)

8. To get the final position of the target, since the picture is cropped in step 3, it is necessary to convert the coordinates obtained in the previous step into the coordinates of the original image and frame the target position in the picture

x_final = [xi + 200 for xi in xl]y_final = [yi + 200 for yi in yl]for x_f, y_f, w_f, h_f in zip(x_final, y_final, wl, hl):cv2.rectangle(cur_img, (int(x_f - w_f / 2), int(y_f - h_f / 2)), (int(x_f + w_f / 2), int(y_f + h_f / 2)), (0, 0, 0), thickness=2)