当前位置：网站首页>How did we solve the realsense color bias problem?

How did we solve the realsense color bias problem?

2022-06-22 03:32:00 【Oak China_ official】

edit ：OAK China
First episode ：oakchina.cn
Like it , Please do more ️

▌ Preface

Hello, Hello everyone , Here is OAK China , I'm your assistant .

I don't know if you've ever used it realsense Of D455, Friends who have used the camera will surely find that there is something wrong with the color of the camera .
Insert picture description here
What's the reason for this ？ Why doesn't Intel solve this problem ？ Share today OAK official Luxonis How does the company solve this problem .

▌ background

Luxonis Always keep close contact with users , Adhere to the principle of making friends with users . therefore , When users ask us questions that are important to them , We will listen . When users give us suggestions on how to make their robots work outdoors , And why their depth cameras can't work outdoors , We will listen . Although the depth camera used is not made by us . in fact , These cameras are made by incumbents in the market RealSense Made of , Customers are already phasing them out . As shown in the figure below ,RealSense The flagship product of the depth camera series D455 Unable to take useful images outdoors .
Insert picture description here

This is a huge problem , Because almost every robot system uses color data . In agriculture and many industries , This is a complete hindrance . So what did we do ？

Um. , We are very friendly to our customers , So we fixed it for our customers . We use the requested OV9782 The global shutter color makes a version of OAK-D, And solved this problem . How to solve it ？

Classical computer vision image sensor tuning problem , This is called incorrect “ Lens shadow correction ”(LSC), It will cause colors that do not exist in the scene , But this is an image sensor 、 The result of the interaction between the lens and the lens holder , And effectively “ make up ” A color that doesn't exist . Use appropriate LSC, The image will show a more correct color .

Insert picture description here

Before we get into the details of how to do it , Let's quickly review . All of us OAK-D-S2 and OAK-D-S2-PoE Standard configuration IMX378 As RGB camera , But because we want to give our customers maximum flexibility , We let them choose to use OV9782 Replace the camera , In order to FOV from 120 Add to 150. up to now , One disadvantage of this conversion is that it may change color when shooting video outdoors .

In the end, we found out , This question is related to OV9782 It is related to supersaturation caused by exposure to strong light . under these circumstances , Pink or purple hues will permeate the image .

▌ Intel RealSense D455 - Pink question

We are not the only ones dealing with this problem , We are only the one who has solved this problem （ More on that later ）. Intel is our closest competitor in the market , their D455 The camera —— Use with our OAK The camera is the same OV9782—— Also struggling with outdoor color creep , But Intel still hasn't found a solution .

Google for a few minutes , You will find out how deeply their customers are concerned , From the direct support forums and industry news, you can find ：

IntelRealSense/librealsense GitHub problem
come from DotProduct, Known problems ：RealSense D455 and L515 My pink / Purple color difference
From their own point of view ：D455 RGB Color issues community page , Someone said in the comments that :“ This is a production problem , Intel must recall and replace all affected devices as we do . Intel ; You cannot fix this problem by updating the firmware . I want to return my D455, I would advise my customers not to buy .”

Intel may have the right to let this go , But this is not what we are Luxonis How it works .

We even won a great new customer , He was tired of waiting for Intel to take action .Greenzie Decision to discard D455 There are many reasons , But this coloring problem is one of them .

notes ：Greenzie The company is a famous commercial weeding robot manufacturer .

▌ Move forward —— Solutions and the future

It's time to take it seriously ！ How can we solve the problem that a big company like Intel cannot （ Or unwilling ） Problem solved ？ Simply speaking , We are a team that never gives up . We know that correct color representation is crucial for our users , So I have been solving this problem , Until we found a special lens shadow correction （LSC） Great improvements have been achieved . You can think of it as putting a pair of sunglasses on the camera lens , Make it more consistent .

We have seen before / After the image , But now let's look at a test video ！ Use a very short shutter speed even in strong light , We can also see how the presence of invasive coloring is significantly reduced .

OAK Camera color correction test #OpenCV AI Kit

Even better, we All wide angle Cameras —— Current and future models —— This enhanced functionality is now available , At the same time, it is still allowed to IMX378（120 DFOV 12MP Rolling shutter color ） and OV9782（150 DFOV 1MP Global shutter color ） Choose between .

Be sure to check out our full range of wide-angle camera options , Find the one that suits you ：

OAK-D-W,USB Power supply
OAK-D-Pro-W, With infrared night vision and active stereo vision ,USB Power supply
OAK-D-W-PoE, Power over Ethernet
OAK-D-Pro-W-PoE, Have night vision and active stereo vision , Power over Ethernet