当前位置：网站首页>Jetson nano from introduction to practice (cases: opencv configuration, face detection, QR code detection)

Jetson Nano It's a small size 、 Powerful artificial intelligence embedded development board , On 2019 year 3 Launched by NVIDIA in October . Preloaded Ubuntu 18.04LTS System , Equipped with... Developed by NVIDIA 128 nucleus Maxwell GPU, Can quickly move AI The technology is applied to various intelligent devices . Compared with Jetson Previous products （Jetson TK1、Jetson TX1、Jetson TX2、Jetson Xavier）,Jetson Nano The price is just 99 dollar , Greatly reduce the R & D cost of artificial intelligence terminals . therefore , Once launched , Has received widespread attention . Its official website address is ：Jetson Nano Developer Kit for AI and Robotics | NVIDIA

Here are some details Jetson Nano The advantages of ：

(1) Small size , Powerful performance , Affordable , The overall hardware design is similar to raspberry pie , Support a range of popular AI frame , In addition, NVIDIA has invested a lot of research and development energy to create a matching Jetpack SDK Development kit , Through this development kit, you can learn and develop AI Products become simpler and more convenient .

(2) Specially designed for AI And Design , More powerful than raspberry pie , Carrying four cores Cortex-A57 processor ,128 nucleus Maxwell GPU And 4GB LPDDR Memory , It can be a robot terminal 、 Industrial visual terminals bring enough AI Calculate the force .

(3) Can provide 472 GFLOP, Support for high resolution sensors , Multiple sensors can be processed in parallel , Multiple modern neural networks can be run on each sensor stream .

(4) Support NVIDIA's NVIDIA JetPack The component package , This includes in-depth learning 、 Computer vision 、GPU Calculation 、 Board level support package for multimedia processing ,CUDA,cuDNN and TensorRT Software library .

(5) Support a range of popular AI Framework and algorithm , such as TensorFlow,PyTorch,Caffe / Caffe2,Keras,MXNet etc. , So that developers can simply and quickly AI Models and frameworks are integrated into the product , Easily realize image recognition , object detection , Postural estimation , Semantic segmentation , Powerful functions such as video enhancement and intelligent analysis .

At present, the artificial intelligence tuyere has gradually entered the stage of landing application , More products hope to apply AI computing power to actual terminals , That is to realize the so-called edge computing requirements . Fundamentally speaking, the core of promoting artificial intelligence in recent years is the deep learning algorithm , But the reasoning acceleration of deep learning is inseparable from high speed GPU Support for , And the general desktop PC Or server level graphics card （ Such as NVIDIA 1080Ti etc. ） It's very expensive , Not suitable for edge computing , And it's too big . therefore , NVIDIA launched this embedded artificial intelligence development board Jetson Nano It is very suitable for the current industry demand .

2. Jetson Nano Environment configuration

2.1 Introduction of unpacking accessories

Directly purchased Jetson Nano Only one bare metal machine is included , As shown in the figure below ：

The image below shows Jetson Nano Bare metal interfaces ：

Interface 1：SD Card slot , Insert... On the back , Just press it gently when pulling it out .SD The card is mainly used to store the whole system and related data , Similar to desktop PC The role of hard disk ;
Interface 2: 40 individual pin Angular GPIO Interface , It is mainly used to connect external equipment , Such as thermostat 、 Level, etc ;NVIDIA The official offers JetsonGPIO library （Python） It can be easily controlled GPIO,Jetson.GPIO Kurt used raspberry pie RPi.GPIO It's the same as the library API.
Interface 3：USB Oral 5V Input source , This interface is used as the power input by default ; The interface can also be used as a data transmission line , For example, high-speed serial port ;
Interface 4： Wired network port ; If it is not enough to buy a wireless network card, you can directly connect the network cable to the port for networking ;
Interface 5: 4 individual USB3.0 mouth , For access to USB equipment , for example USB Type camera ;
Interface 6：HDMI Output port , It can be used to connect the display screen ;
Interface 7：DP Display interface , It can be used to connect the display screen ;（ Interface 6 And interface 7 Are used to connect to the peripheral display screen , Just different interfaces , Only one can be used in actual use , Usually use HDMI Interface ）. If you want to answer VGA display , You can buy one HDMI turn VGA And then use the interface 6 that will do ;
Interface 8： Direct 5V Input power ; Be careful ,Jetson Nano There are two ways of power input , One is to use interfaces 3 The way , Just buy one 5V Plug in the Android phone charger and you can use it , This is also the default way . The other is to use interfaces 8, You need to be right at this time J48 Make a short circuit , After the short circuit is completed, the interface can be switched 8 Perform power input , As shown in the figure below ：

Interface 9： MIPI CSI camera , You can buy a camera for raspberry pie directly ;

In addition to bare metal , You also need to buy your own memory card 、 keyboard 、 mouse 、5V2A DC power supply 、 Monitor 、 wireless network adapter 、 camera 、 Fan 、 Shell and other accessories can be better developed , These accessories can be easily accessible on the Internet .

The selection and installation of some accessories are described below .

Memory card ： General memory cards have 16G、32G、64G、128G etc. , In order to facilitate the later in-depth learning and development , Recommended selection 32G And above memory cards .

Keyboard and mouse can be used directly to support USB3.0 that will do ;
Choose the power supply 5V2A or 5V3A Power Supply , As shown in the figure below . The main , If the power connection method shown in the figure below is adopted , Need to put J48 Only when connected , Otherwise, the default power input is through USB Mode access .

Monitor ： It is recommended to use HDMI Your monitor is directly connected to , If you want to connect to the computer VGA Monitor , You can buy one HDMI turn VGA The conversion module of completes the connection

wireless network adapter ： Wired and wireless networking can be adopted , Through interface 4 You can access the Internet directly by connecting to the Internet cable . If wireless is required wifi Internet access , You need to buy wireless separately wifi modular . Suggest to buy Jetson Nano Standard supporting wireless Wifi modular , As shown in the figure below , It mainly includes a pair of dual antennas and a processing chip , Its compatibility can be guaranteed .

The wireless WIFI The installation of is a little more troublesome , It is necessary to disassemble the... On the bare metal machine GPU parts .

First of all, will wifi Two antennas are connected to the processing chip , Press on the card slot to clamp the antenna , As shown in the figure below ：

Then, as shown in the following figure, the Jetson Nano On bare metal GPU Disassemble the components ：

It will be removed wifi The chip is installed in the card slot , As shown in the figure below ：

Then reinstall GPU Components , Be careful not to jam the connecting wires of the two antennas during installation .

camera ： The camera can be used in two ways , One uses raspberry pie directly CSI camera , There is also the use of USB camera . If you use a raspberry pie camera , Then it can be accessed as shown in the figure below （ Pay attention to the front and back ）：

Fan ： In general , No fan pair is required Jetson Nano Conduct heat dissipation treatment , But if you use deep learning techniques , And high-frequency reasoning operations , Well, it's better to be in GPU A cooling fan is installed on the module , As shown in the figure below ：

Shell ： Exposed development boards are prone to short circuits , From a security point of view , After installing the above accessories, it is better to install the whole Jetson Nano Install a housing . There are many kinds of shells , It can be installed according to the shell instructions actually purchased , The following figure is a common Jetson Nano Transparent shell ：

2.2 Burning system

NVIDIA is officially Jetson Nano Provides SD System image of card version , And has been updated and maintained , The image contains the corresponding Ubuntu System and configured cuda The environment and opencv Environmental Science , Therefore, you only need to download and install the image Jetson Nano Most of the environment configurations . You can download images directly from the official website , Download at ：Jetson Download Center | NVIDIA Developer, double-click Image Download the latest image , As shown in the figure below ：

This article downloads 2019 year 12 month 17 Updated on JP4.3 Version image , What's downloaded is a zip Compressed package , Will be zip Compress the package to decompress , You can get the suffix img The file of , This file is the image file we need . The image file accounts for about 12.5G Space , All this content needs to be stored in the SD In the card , therefore , It is recommended to select a large capacity SD Better card , Such as 64G or 128G. In general , If SD The card is new , You can write directly , But sometimes you need to do something about the old SD Re write the card , At this time, it is necessary to check in advance SD Format the card , Avoid errors in the mirroring process . If it has been burned before Jetson Nano mirrored SD card , Then you need to be right first SD Card for partition deletion and re merging , It's because of what happened Jetson Nano Burned SD Cards form 12 Subarea , Therefore, you need to delete and merge these partitions with the disk manager first , Then perform a new image burning （ If it's new SD Cards do not require these operations ）, As shown in the figure below ：

As shown in the figure above , old Jetson Nano Mirror card will form 12 Subarea , Compare the disk in the above figure 2 Of 12 Zones , In turn “ Delete volume ” Handle , Then for the disk 2 again “ New simple volume ”,

If it's new SD card , You only need to format the following . Then start to burn formally . There are many burning tools , This article recommends the use of Win32DiskImager, The tool can be downloaded and installed directly on the Internet . Double-click to open Win32DiskImager, Choose the img Mirror image , And configure the corresponding SD Card Charms , As shown in the figure below ：

single click “ write in ” You can complete image burning , The whole burning time is about 15 About minutes .

2.3 Power on and basic settings

After burning, you will SD Insert the card into Jetson Nano In the card slot on the back , And then turn it on .

Initial installation requires basic configuration , Including the account password 、Wifi password 、 typewriting 、 Time zone, etc , Just follow the prompts , After the configuration is completed, an update operation will be performed by default Applying Changes, This update requires networking , If you are not connected to the network, click cancel Cancel the manual update after networking . Long update time , Wait for . Finally, it will enter the desktop , As shown in the figure below ：

2.4 Development environment configuration

2.4.1 Update source and software

After installing the system, you should first update the source , Otherwise, subsequent updates and upgrades will be very slow . But because of Jetson Nano It's using aarch64 Architecturally Ubuntu 18.04.2 LTS System , And AMD Architecturally Ubuntu Different systems , Therefore, it needs to be replaced by aarch64 Source , It's important to be careful here , Do not replace with x86-64 It's the source of my life .（ Recently tried the latest Jetpack4.6, Personally, I don't think it's a problem to change the source , Therefore, this part of the source change content is only put here for reference ）

We choose the source of Tsinghua University to update . First back up the original source , change source.list Name of file , For a rainy day ：

sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak    
sudo gedit /etc/apt/sources.list

Then delete everything , Copy the following ：

deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic main multiverse restricted universe
deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-security main multiverse restricted universe
deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-updates main multiverse restricted universe
deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-backports main multiverse restricted universe
deb-src http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic main multiverse restricted universe
deb-src http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-security main multiverse restricted universe
deb-src http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-updates main multiverse restricted universe
deb-src http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-backports main multiverse restricted universe

This is the end of the source change .

Open the terminal , Enter the following command to update ：

sudo apt-get update
sudo apt-get full-upgrade

The above updates take a long time , The update may fail due to the network speed , Do not shut down and execute the command again , It will automatically resume transmission at breakpoints .

2.4.2 Screen closing time setting

Jetson Nano Default 5 No operation for minutes Jetson The screen closes , To restart the screen, you need to re-enter the boot password . Due to the development of Jetson Nano You often need to wait in the process of , So I don't want to open the screen frequently , Instead, I want the screen to stay open . open System Settings Enter the system setting interface , As shown in the figure below ：

single click Brightness & Lock, And then Turn screen off when inactive for Change it to Never that will do , As shown in the figure below ：

2.4.3 Install Chinese IME

In the development process, it is often necessary to use Chinese search and write necessary Chinese notes , Therefore, it is recommended to install Chinese input method for the system .Jetson Nano Bring their own ibus Chinese input method , However, Chinese input can only be carried out under simple configuration . Input the command directly in the terminal ibus The interface shown in the figure below will appear , explain Jetson Nano I've brought it with me ibus Input method environment .

The following is ibus Download Pinyin input method , Enter the command ：

sudo apt-get install ibus-pinyin

The above download and installation will take about ten minutes . After installation, enter system configuration System Settings Interface , Select language support options Language Support , As shown in the figure below :

And then choose “ Add or remove languages ” Interface , The system will select language support , As shown in the figure below ：

Choose... Here “ Chinese simplified ” And then click Apply that will do . This Apply A series of Chinese language packs will be installed , As shown in the figure below ：

After installation, adjust the Chinese language to the front in the language support interface , As shown in the figure below ：

And then click “ Applied to the whole system ”. The final will be “ Keyboard input system ” Change it to iBus that will do .

Restart the system （ Very important ！！！）, Then enter the following command in the terminal to enter ibus Configuration interface ：

 ibus-setup

Click... In the configuration interface “ add to ” Button , And then unfold “ chinese ” Options , choice Intelligent Pinyin. If you can't find it here “ chinese ” Option, you can shut down and restart first , Look again .

After adding, enter the following command to restart ibus that will do .

ibus restart

Last , Cut the input method into Pinyin input method in the task bar on the top of the desktop Pi, As shown in the figure below ：

At this point, you can use Chinese input . As shown below ：

2.4.4 install Code OSS

Visual Studio Code（VS Code） The development environment is free of charge （IDE）, Apply to Windows,Mac and Linux.VS Code In recent years, it has received more and more attention , Become the preferred compilation environment for the majority of programming developers . It is an open source project launched by Microsoft , It has attracted numerous third-party developers and end users , Become one of the top open source projects . It's powerful 、 Fast , It also achieves a simple and smooth user experience with a large number of plug-ins , It belongs to a very excellent IDE.

Native VS Code It doesn't apply to Jetson Nano, At present , It's not targeted yet Jetson Nano In this way ARM The equipment VS Code Official version . however , Because it's open source , So anyone can compile a version . among ,Code-OSS It is such an embedded environment “VS Code”.Code-OSS be based on VS Code, It's not just a code editor , It has built-in Explorer capabilities for managing entire project folders rather than individual scripts, as well as rich third-party plug-ins . actually Code-OSS Almost have VS Code All complete functions of , So use it as a code editor to edit code , for example python, It will make the whole development process more convenient . The specific installation methods are explained below .

open Chromiun browser , Enter url ：

Click on Packsges, Check the package names listed , Select suffix with arm64(aarch64) Of , As shown in the figure below ：

Click to enter the details page , Find the corresponding wget command , As shown in the figure below ：

This command demonstrates how to download the installation package , As follows ：

wget --content-disposition https://packagecloud.io/headmelted/codebuilds/packages/debian/stretch/code-oss_1.42.0-1575969886_arm64.deb/download.deb

Copy the command to the terminal to download the installation package . As shown in the figure below ：

here , The installation package has been downloaded to home The root directory , You can view the downloaded through the file explorer deb Installation package , As shown below ：

Enter the following commands in the terminal to complete the final installation ：

sudo dpkg -i code-oss_1.42.0-1575969886_arm64.deb

After installation, you can search in the search Code OSS, Will pop up Code OSS Applications , This is what we need Python Programming IDE. Click the application to open as shown in the following figure ：

The following is a simple demonstration of how to use Code OSS perform Python Script .

First, in the Code OSS Install in Python plug-in unit , The plug-in installation method is the same as that of the common VS Code Exactly the same , Not familiar with VS Code Our readers can start with the desktop PC Be familiar with VS Code The basic usage is switched to Jetson Nano From the environment . The plug-in installation is shown in the figure below , stay Extensions Search for python, Select the first pop-up plug-in to install ：

The next in home Create a new one in the directory code Folder , This folder is used to store Python Code script . And then in Code OSS Open the just created code Folder , Then create a new file , Press ctrl+s Key to save the file , Name the file main.py, Then enter the following code ：

a = 36
b = 64
print(a+b)

Then press ctrl+F5 Key to run the script , The effect is as follows ：

thus , Completed Python Installation and running of the editor .

2.4.5 install Qt5

In the actual product deployment phase , Considering the terminal equipment speed 、 stability 、 Memory occupation and other factors , Generally C++ To develop the final product , Only in the design phase of product model can python Algorithm development . therefore , Need a product that can be used in Jetson Nano In the development C++ The compiler is convenient for us to develop floor products .VS Code It can be developed by itself C++ application , however Code-OSS about C++ Our support is not good , therefore , You need to install another excellent C++ Compiler to complete C++ Development tasks . This article recommends the use of Qt.

Qt It's a cross platform C++ Development Library , Mainly used to develop graphical user interface （Graphical User Interface,GUI） Program , Of course, you can also develop a command line without an interface （Command User Interface,CUI） Program .Qt Is pure C++ Developed , So use it to develop C++ Applications have natural advantages .Qt There are many supported operating systems , For example, the general operating system Windows、Linux、Unix, Smart phone systems Android、iOS、WinPhone And embedded systems QNX、VxWorks wait . Of course ,QT I fully support Jetson Nano Of Ubuntu Environmental Science .

Jetson Nano Lower installation QT Relatively simple , Just type the command ：

sudo apt-get install qt5-default qtcreator -y

What is installed at this time is Qt5.9.5 edition .

After installation , Also search in the search menu Qt, Then it will appear Qt Creator, This is called Qt Of IDE, To open it . Next, a simple demonstration of how to create a simple C++ Console program .

open Qt Creator, As shown in the figure below ：

single click New Project Create a new project , Choose here Application Under the Qt COnsole Appliation application , Create a Qt Version of C++ Console program ：

Then the project is named QTtest：

Then click next by default to complete the project creation . You can see ,Qt We've created a C++ file main.cpp Used for compiling C++ Code , And there's one more QTtest.pro The configuration file is used to configure the entire project , The effect is shown below ：

At this point, you can directly press ctrl+r Key to run the project , But since we don't have any output code , So the terminal that pops up does not output any value . So let's revise that main.cpp Code for , Also, perform the addition of two integers and output the result , The code is as follows ：

#include <QCoreApplication>
#include <QtDebug>

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
    int c=64;
    int d=36;
    qDebug() << (c+d);
    return a.exec();
}

here , Press again ctrl+r Key to run the project and output the results shown in the figure below ：

Qt Creator Please learn relevant tutorials by yourself , There are many resources in this part and they are mature , For the development of actual embedded products, master Qt and C++ The use of is a necessary process . This article is no longer about Qt In especial Qt Interface programming for in-depth introduction .

Finally, let's take a look at the generation of a QTtest In contrast debug Executable project build-QTtest-unknown-Debug, Born in this folder debug Version of QTtest Executable program . Through terminal cd Command to enter the folder , Then input

./QTtest

Will directly execute the program , As shown in the figure below ：

That is to say, in essence, we have successfully deployed and developed an application , The function of this application is very simple , Only two fixed integers are added . Simple as it is , But it has combed a common form of our normal development of artificial intelligence products , In the first place VS Code of use python The script performs algorithm verification , Last but not least QT Write corresponding C++ application , Finally, the binary executable program is generated , The resulting binary executable is ours “ product ”, The executable code is encapsulated 、 Invisible 、 Can run directly .

thus , We're done Jetson Nano General development configuration for , Next, several small projects will be demonstrated , So that readers can learn more deeply Jetson Nano Development method of .

3. project Case study

3.1 Face detection

This section begins with Python To complete the face detection algorithm , It will explain Python Configure and use Opencv And some common methods python The installation of the library .

3.1.1 install pip

because Jetson Nano It's pre installed in Python3.6 edition , So it can be installed directly pip.

Enter the following commands in the terminal to install ：

sudo apt-get install python3-pip python3-dev

At this time after installation pip yes 9.01 edition , Need to be right pip Upgrade , Otherwise, others will be installed later Python There will be problems in the library . The upgrade command is as follows ：

python3 -m pip install --upgrade pip

The upgraded version is 19.0.3. Although the upgrade has been completed , But at this time pip3 There's a little bug It needs to be repaired manually .

First use the following command to open pip3 file ：

sudo vim /usr/bin/pip3

Enter characters on the keyboard a Enter insertion mode , Then you can start editing the file , take ：

from pip import main
if __name__ == '__main__':
    sys.exit(main())

It is amended as follows ：

from pip import __main__
if __name__ == '__main__':
    sys.exit(__main__._main())

Then press Esc Key to enter command mode . Finally, press English ”:” Key to enter the last line mode , Knock in wq Press enter to save the changes and exit the editor .

3.1.2 install Python Common machine learning packages

sudo apt-get install python3-scipy
sudo apt-get install python3-pandas
sudo apt-get install python3-sklearn

3.1.3 Configure for Python Of Opencv

There are two ways to install python Under the opencv. One is downloading Opencv Source code and recompile to generate the corresponding python package , Then copy the package to python In the installation package path of ; The other is to use commands directly sudo pip3 install python3-opencv. It should be noted that , The second method essentially installs the compiled opencv package , Its opencv The version of is fixed , If you want to use the latest opencv, such as opencv4, Then the second method is not appropriate . This section simply uses the first method to install .

The original image has been pre installed opencv4.1.1, You can use the following command to view the current Opencv Version number :

opencv_version

The output result is shown in the figure below ：

therefore , We don't need to recompile , It can be used directly .

3.1.4 be based on Opencv Face detection

（1）python Face detection

This section begins by writing a python Scripts are used to detect faces in images , Use Code OSS open 2.4.4 Section code Folder , Create a new python Script , be known as face_detect_test.py, The code is as follows ：

import cv2 
filepath = "test.jpg" 
img = cv2.imread(filepath) #  Read the picture  
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) #  Turn grey  
# OpenCV Face recognition classifier  
classifier = cv2.CascadeClassifier( "haarcascade_frontalface_default.xml" ) 
color = (0, 255, 0) #  Define the paint color  
#  Call face recognition  
faceRects = classifier.detectMultiScale( gray, scaleFactor=1.2, minNeighbors=3, minSize=(32, 32)) 
if len(faceRects): #  Greater than 0 A face is detected  
    for faceRect in faceRects: #  Frame each face individually  
        x, y, w, h = faceRect 
        #  Frame the face  
        cv2.rectangle(img, (x, y), (x + h, y + w), color, 2) 
cv2.imshow("image", img) #  Display images  
c = cv2.waitKey(10) 
cv2.waitKey(0) 
cv2.destroyAllWindows()

In the above code filepath Used to store the image path to be detected , Generally, it can be placed in the same directory as the source file . In the structure opencv Face detection classifier , The corresponding face detection profile is required , The file stores the relevant parameters for the face detection algorithm , This document can be obtained from opencv The installation directory is found ：/usr/share/opencv4/. Once found, copy it to the source file directory .

Press ctrl+F5 function , The effect is shown below ：

（2）C++ Face detection

This section writes a C++ application , It is used to detect the face in the image , Use Qt5 Development . Related implementation methods and python Same version . Mainly explain how to be in QT Lower integration Opencv Conduct C++ Project development .

C++ Lower development Opencv Some additional configuration is required , Have a look first opencv The location of .Jetson Nano Pre installed Opencv4.1.1 The location of the header file is shown in the following figure ：

The library file is placed in :

/usr/lib/aarch64-linux-gnu

therefore , Only need Qt Of pro Include the above two directories in the file .

use Qt Creator Reopen 2.4.5 Section created QTtest project , edit QTtest.pro The documents are as follows ：

QT -= gui

CONFIG += c++11 console
CONFIG -= app_bundle
CONFIG += C++11  #  Add pair C++11 Support for 

# The following define makes your compiler emit warnings if you use
# any feature of Qt which as been marked deprecated (the exact warnings
# depend on your compiler). Please consult the documentation of the
# deprecated API in order to know how to port your code away from it.
DEFINES += QT_DEPRECATED_WARNINGS

# You can also make your code fail to compile if you use deprecated APIs.
# In order to do so, uncomment the following line.
# You can also select to disable deprecated APIs only up to a certain version of Qt.
#DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000    # disables all the APIs deprecated before Qt 6.0.0

INCLUDEPATH += /usr/include/opencv4 # Add the header file path 

LIBS += -L/usr/lib/aarch64-linux-gnu -lopencv_core -lopencv_imgcodecs -lopencv_imgproc -lopencv_highgui -lopencv_objdetect  # Add libraries that need to be linked 


SOURCES += main.cpp

The key point is to pay attention to the header file and lib How to add files .

I'm going to modify main.cpp file , The code is as follows ：

#include <iostream>
#include <string>
#include <opencv4/opencv2/opencv.hpp>
#include <opencv4/opencv2/core.hpp>
#include <opencv4/opencv2/highgui.hpp>
#include <opencv4/opencv2/imgproc.hpp>
#include <opencv4/opencv2/objdetect.hpp>
#include <opencv4/opencv2/imgproc/types_c.h>


using namespace std;
using namespace cv;
int main( int argc, char** argv )
{
    std::string filepath( "test.jpeg" );
    Mat img,gray;
    img = imread( filepath, IMREAD_COLOR );
    cvtColor(img, gray, CV_BGR2GRAY);

    CascadeClassifier classifier;
    classifier.load("haarcascade_frontalface_default.xml");
    Scalar color=Scalar(0, 255, 255);

    vector<Rect> faceRects;
    classifier.detectMultiScale(gray,faceRects,1.2,3,0,Size(32,32));

    for (size_t i = 0; i < faceRects.size(); i++)
    {
        rectangle(img, faceRects[i], color);
    }

    namedWindow( "Display window", WINDOW_AUTOSIZE );
    imshow( "Display window", img);
    waitKey(0);
    return 0;
}

Rebuild the entire project , And then test.jpeg and haarcascade_frontalface_default.xml The file is placed in the compiled build-QTtest-unknown-Debug In the folder , The effect picture of the running project is as follows ：

3.2 QR code detection （ Make a code scanning gun ）

Now Alipay and wechat widely use QR code as a means of payment , In real life shopping, we often show the QR code to merchants through mobile phones for scanning , So can we build a code scanner by ourselves ？ With Jetson Nano This embedded artificial intelligence development board , We can make a code scanning gun by ourselves .

3.2.1 Read the camera

In this section, we hope to be able to read images through the camera , And the two-dimensional code in the image is analyzed in real time , That is to realize the function of a code scanning instrument . This section realizes the camera reading function . There are generally two options for cameras , One is relatively cheap csi camera （ Raspberry pie camera ）, The other is USB camera . It is worth noting that , If the USB camera , Then the image reading and rendering will use Jetson Nano Of GPU, If we are still doing some reasoning work of deep learning at this time , Then it will obviously take up some GPU resources . contrary ,Jetson Nano about csi Camera reading and rendering will use Gstreamer Pipe to handle , Will use specific hardware acceleration , The whole treatment effect will be better .

In this section, we will introduce the reading methods of the two cameras in detail . Either way , We all use Opencv This powerful image processing open source library is used as the basis to perform related operations .

（1） Read CSI camera

Use Gstreamer Read CSI The cameras are mainly divided into 3 A step ： establish Gstreamer The Conduit ; Bind the pipe opencv Video stream of ; Frame by frame extraction and display . The following is the first presentation based on Python Detailed code of ：

import cv2

#  Set up gstreamer Pipeline parameters 
def gstreamer_pipeline(
    capture_width=1280, # The width of the image pre captured by the camera 
    capture_height=720, # The height of the image pre captured by the camera 
    display_width=1280, # The width of the image displayed in the window 
    display_height=720, # The height of the image displayed in the window 
    framerate=60,       # Capture frame rate 
    flip_method=0,      # Whether to rotate the image 
):
    return (
        "nvarguscamerasrc ! "
        "video/x-raw(memory:NVMM), "
        "width=(int)%d, height=(int)%d, "
        "format=(string)NV12, framerate=(fraction)%d/1 ! "
        "nvvidconv flip-method=%d ! "
        "video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! "
        "videoconvert ! "
        "video/x-raw, format=(string)BGR ! appsink"
        % (
            capture_width,
            capture_height,
            framerate,
            flip_method,
            display_width,
            display_height,
        )
    )


if __name__ == "__main__":
    capture_width = 1280
    capture_height = 720
    display_width = 1280
    display_height = 720
    framerate = 60
    flip_method = 0

    #  Create pipes 
    print(gstreamer_pipeline(capture_width,capture_height,display_width,display_height,framerate,flip_method))

    # The pipeline is bound to the video stream 
    cap = cv2.VideoCapture(gstreamer_pipeline(flip_method=0), cv2.CAP_GSTREAMER)

    if cap.isOpened():
        window_handle = cv2.namedWindow("CSI Camera", cv2.WINDOW_AUTOSIZE)
        
        #  Frame by frame display 
        while cv2.getWindowProperty("CSI Camera", 0) >= 0:
            ret_val, img = cap.read()
            cv2.imshow("CSI Camera", img)

            keyCode = cv2.waitKey(30) & 0xFF         
            if keyCode == 27:# ESC Key to exit 
                break

        cap.release()
        cv2.destroyAllWindows()
    else:
        print(" Failed to turn on the camera ")

Then 3.1.4 Section , stay Code-OSS Create a new file named csi_camera_test.py, Then copy the above code into the file , Save and press ctrl+F5 Run script （ Premise ： Make sure you have correctly installed CSI Raspberry pie camera ）, The operation effect is as follows ：

You can see that the video stream image can be displayed normally , But because of the raspberry pie camera itself , There is a lot of noise in the image , There is also some distortion in the color （ It is recommended to buy better cameras in real industrial scenes ）. Let's synchronize the following C++ edition . Then 3.1.4 Section 2 , modify main.cpp The documents are as follows ：

#include <iostream>
#include <string>
#include <opencv4/opencv2/opencv.hpp>
#include <opencv4/opencv2/core.hpp>
#include <opencv4/opencv2/highgui.hpp>
#include <opencv4/opencv2/imgproc.hpp>
#include <opencv4/opencv2/objdetect.hpp>
#include <opencv4/opencv2/imgproc/types_c.h>
#include <opencv4/opencv2/videoio.hpp>

using namespace std;
using namespace cv;

string gstreamer_pipeline (int capture_width, int capture_height, int display_width, int display_height, int framerate, int flip_method)
{
    return "nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)" + to_string(capture_width) + ", height=(int)" +
           to_string(capture_height) + ", format=(string)NV12, framerate=(fraction)" + to_string(framerate) +
           "/1 ! nvvidconv flip-method=" + to_string(flip_method) + " ! video/x-raw, width=(int)" + to_string(display_width) + ", height=(int)" +
           to_string(display_height) + ", format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink";
}

int main( int argc, char** argv )
{
    int capture_width = 1280 ;
    int capture_height = 720 ;
    int display_width = 1280 ;
    int display_height = 720 ;
    int framerate = 60 ;
    int flip_method = 0 ;

    // Create pipes 
    string pipeline = gstreamer_pipeline(capture_width,
    capture_height,
    display_width,
    display_height,
    framerate,
    flip_method);
    std::cout << " Use gstreamer The Conduit : \n\t" << pipeline << "\n";

    // The pipeline is bound to the video stream 
    VideoCapture cap(pipeline, CAP_GSTREAMER);
    if(!cap.isOpened())
    {
        std::cout<<" Failed to turn on the camera ."<<std::endl;
        return (-1);
    }

    // Create a display window 
    namedWindow("CSI Camera", WINDOW_AUTOSIZE);
    Mat img;

    // Frame by frame display 
    while(true)
    {
        if (!cap.read(img))
        {
            std::cout<<" Capture failed "<<std::endl;
            break;
        }
        imshow("CSI Camera",img);

        int keycode = cv::waitKey(30) & 0xff ; //ESC Key to exit 
            if (keycode == 27) break ;
    }

    cap.release();
    destroyAllWindows() ;
}

Which requires additional additions opencv Header file for video processing #include <opencv4/opencv2/videoio.hpp>. in addition , You need to modify the pro file , Process the video to the corresponding opencv_videoio Library included , complete pro The documents are as follows ：

QT -= gui

CONFIG += c++11 console
CONFIG -= app_bundle
CONFIG += C++11  #  Add pair C++11 Support for 

# The following define makes your compiler emit warnings if you use
# any feature of Qt which as been marked deprecated (the exact warnings
# depend on your compiler). Please consult the documentation of the
# deprecated API in order to know how to port your code away from it.
DEFINES += QT_DEPRECATED_WARNINGS

# You can also make your code fail to compile if you use deprecated APIs.
# In order to do so, uncomment the following line.
# You can also select to disable deprecated APIs only up to a certain version of Qt.
#DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000    # disables all the APIs deprecated before Qt 6.0.0

INCLUDEPATH += /usr/include/opencv4 # Add the header file path 

LIBS += -L/usr/lib/aarch64-linux-gnu -lopencv_core -lopencv_imgcodecs -lopencv_imgproc -lopencv_highgui -lopencv_objdetect -lopencv_videoio  # Add libraries that need to be linked 


SOURCES += main.cpp

Save all the changes and rebuild the project and run it to get the same results , The video can be displayed correctly .

（2） Read USB camera

Compared to reading CSI camera , Read USB The camera is simpler , It only takes two steps ： Turn on the camera ; Frame by frame extraction . But here's the thing Jetson Nano Not all USB camera , It is recommended to try to choose Linux Drive free USB camera . This paper adopts a 4K HD camera .

Here's how Python Version of the complete code ：

import cv2
 
 # Create a camera capture module 
cap = cv2.VideoCapture(1)

# create a window 
window_handle = cv2.namedWindow("USB Camera", cv2.WINDOW_AUTOSIZE)

#  Frame by frame display 
while cv2.getWindowProperty("USB Camera", 0) >= 0:
    ret_val, img = cap.read()
    print(img.shape)
    
    #  The image is too large and needs to be adjusted 
    height, width = img.shape[0:2]
    if width>800:
        new_width=800
        new_height=int(new_width/width*height)
        img = cv2.resize(img, (new_width,new_height))
 
    cv2.imshow("USB Camera", img)

    keyCode = cv2.waitKey(30) & 0xFF         
    if keyCode == 27:# ESC Key to exit 
        break

# Release resources 
cap.release()
cv2.destroyAllWindows()

The above code used... When opening the camera cap = cv2.VideoCapture(1), Parameters here 1 Because of the current Jetson Nano Also connected CSI camera ,CSI The identification of the camera is 0, So the USB The identification of the camera is 1, This can be obtained by testing in actual use . in addition , The size of the image is limited in the above code , If the width exceeds 800, Then the proportionally scaled image is displayed again . The effect is shown below ：

You can see this USB 4K The display effect of the camera for the image is good , The color is more realistic , Less noise . We will continue to use this camera for QR code detection later .

Here's how C++ Version code , modify main.cpp file , The code is as follows ：

#include <iostream>
#include <string>
#include <opencv4/opencv2/opencv.hpp>
#include <opencv4/opencv2/core.hpp>
#include <opencv4/opencv2/highgui.hpp>
#include <opencv4/opencv2/imgproc.hpp>
#include <opencv4/opencv2/objdetect.hpp>
#include <opencv4/opencv2/imgproc/types_c.h>
#include <opencv4/opencv2/videoio.hpp>

using namespace std;
using namespace cv;

int main( int argc, char** argv )
{
    // Turn on the camera 
    VideoCapture cap(1);

    // Create a display window 
    namedWindow("USB Camera", WINDOW_AUTOSIZE);
    Mat img;

    // Frame by frame display 
    while(true)
    {
        if (!cap.read(img))
        {
            std::cout<<" Capture failed "<<std::endl;
            break;
        }
        int new_width,new_height,width,height,channel;
        width=img.cols;
        height=img.rows;
        channel=img.channels();
        cout<<width<<"  "<<height<<"  "<<channel<<endl;

        new_width=800;
        if(width>800)
        {
            new_height=int(new_width*1.0/width*height);
        }

        resize(img, img, cv::Size(new_width, new_height));
        imshow("USB Camera",img);

        int keycode = cv::waitKey(30) & 0xff ; //ESC Key to exit 
            if (keycode == 27) break ;
    }

    cap.release();
    destroyAllWindows() ;
}

The effect is as follows ：

3.2.2 QR code detection and reading

This section will use Opencv Realize the function of two-dimensional code detection and reading . stay opencv4.0 in the future , The QR code reading module has been integrated , therefore , We can use the latest opencv To achieve two-dimensional code detection and reading . QR code detection and recognition are mainly divided into 3 Step ： Use QRCodeDetector() Function to create a QR code detector ; Use detectAndDecode Function to detect and recognize the two-dimensional code of the image ; Output the test results .

Here, the main task is to read each frame of the video stream and then detect the image , For convenience , We only give information about USB A complete example of the camera , about CSI The camera can be based on 3.2.1 In this section, you can migrate the relevant QR code detection code . combination 3.2.1 Section to get USB Camera video code , Give a complete Python Version two-dimensional code detection and reading code ：

import cv2
import numpy as np
 
 # Create a camera capture module 
cap = cv2.VideoCapture(1)

# create a window 
window_handle = cv2.namedWindow("USB Camera", cv2.WINDOW_AUTOSIZE)

# Create a QR code detector 
qrDecoder = cv2.QRCodeDetector()

#  Frame by frame display 
while cv2.getWindowProperty("USB Camera", 0) >= 0:
    ret_val, img = cap.read()
    #print(img.shape)
    
    #  The image is too large and needs to be adjusted 
    height, width = img.shape[0:2]
    if width>800:
        new_width=800
        new_height=int(new_width/width*height)
        img = cv2.resize(img, (new_width,new_height))

    #  QR code detection and recognition 
    data,bbox,rectifiedImage = qrDecoder.detectAndDecode(img)
    if len(data)>0:
        print(" Decode data  : {}".format(data))
        n = len(bbox)
        for j in range(n):
            cv2.line(img, tuple(bbox[j][0]), tuple(bbox[ (j+1) % n][0]), (255,0,0), 3)
    else:
        print(" QR code not detected ")

    # Display images 
    cv2.imshow("USB Camera", img)

    keyCode = cv2.waitKey(30) & 0xFF         
    if keyCode == 27:# ESC Key to exit 
        break

# Release resources 
cap.release()
cv2.destroyAllWindows()

The renderings are as follows ：

C++ The complete code of the version is as follows ：

#include <iostream>
#include <string>
#include <opencv4/opencv2/opencv.hpp>
#include <opencv4/opencv2/core.hpp>
#include <opencv4/opencv2/highgui.hpp>
#include <opencv4/opencv2/imgproc.hpp>
#include <opencv4/opencv2/objdetect.hpp>
#include <opencv4/opencv2/imgproc/types_c.h>
#include <opencv4/opencv2/videoio.hpp>
#include <opencv4/opencv2/imgcodecs.hpp>

using namespace std;
using namespace cv;

int main( int argc, char** argv )
{
    // Turn on the camera 
    VideoCapture cap(1);

    // Create a display window 
    namedWindow("USB Camera", WINDOW_AUTOSIZE);
    Mat img;

    // Create a QR code detector 
    QRCodeDetector qrDecoder = QRCodeDetector();

    // Frame by frame display 
    while(true)
    {
        if (!cap.read(img))
        {
            std::cout<<" Capture failed "<<std::endl;
            break;
        }
        int new_width,new_height,width,height,channel;
        width=img.cols;
        height=img.rows;
        channel=img.channels();
        //cout<<width<<"  "<<height<<"  "<<channel<<endl;

        // Resize image 
        new_width=800;
        if(width>800)
        {
            new_height=int(new_width*1.0/width*height);
        }
        resize(img, img, cv::Size(new_width, new_height));

        // QR code detection and reading 
        Mat bbox, rectifiedImage;
        std::string data = qrDecoder.detectAndDecode(img, bbox, rectifiedImage);
        if(data.length()>0)
        {
            cout << " Decode data : " << data << endl;

            int n = bbox.rows;
            for(int i = 0 ; i < n ; i++)
            {
                line(img, Point2i(bbox.at<float>(i,0),bbox.at<float>(i,1)), Point2i(bbox.at<float>((i+1) % n,0), bbox.at<float>((i+1) % n,1)), Scalar(255,0,0), 3);
            }
        }
        else
            cout << " QR code not detected " << endl;

        imshow("USB Camera",img);

        int keycode = cv::waitKey(30) & 0xff ; //ESC Key to exit 
            if (keycode == 27) break ;
    }

    cap.release();
    destroyAllWindows() ;
}

The effect is shown below ：

3.3 Two colors LED Light control (GPIO)

3.3.1 Python Realization

Jetson Nano A very important function module is the hardware control , And then many DIY The Internet of things . in front Jetson Nano On the physical map, we can see 40 A neatly arranged GPIO Pin , These are the Jetson Nano Used for signal control of hardware equipment . So how to use these GPIO Interface ？

here Jetson Nano Provide ready-made Python library ：Jetson.GPIO.Jetson Nano The initial system has been pre installed by default Jetson.GPIO library , If you accidentally uninstall or delete it, you can install it through the following command ：

sudo pip3 install Jetson.GPIO

Then set permissions ：

sudo groupadd -f -r gpio
sudo usermod -a -G gpio <your_user_name>

Restart after setting ：

sudo reboot

In specific use GPIO Let's first understand the function distribution of these pins ：

The above figure has marked the functions of each pin , Middle two columns Pin Corresponding 40 One pin , We mainly focus on specific GPIO Mouth and GND mouth .GPIO It can be understood as information input port , Its high and low levels can be determined by python Code control ,GND It means grounding （ In high school physics , there GPIO It's easy to understand , once GPIO If the high level is set, it is equivalent to connecting the positive pole of the power supply , and GND Equivalent to negative pole of power supply , Add a small lamp and a resistor in the middle to form a basic circuit structure ）. In this section we will learn how to use GPIO To light up a LED Small lights . The lamp is shown in the figure below （ This kind of small lamp can be bought on a treasure , Very cheap ）：

The module in the figure has three pins , The left side is marked with ‘-’ The pins of GND, The middle pin is connected to GPIO, Right side callout ”S” The pin of is also connected to GPIO.

When the intermediate pin is high , be LED The light is one color .

When S The pin is high level , be LED The light is another color .

Two colors LED The basic schematic diagram of the lamp is as follows ：

Refer to the above schematic diagram , What we have to do is very simple . Here we choose GPIO 13（ Corresponding pin22） and GPIO15（ Corresponding pin18） As the control signal of two colors ,GND choice pin30 that will do . So we just need to put this LED The three interfaces of can be connected respectively .

#  Import GPIO library 
import Jetson.GPIO as GPIO
 
#  Import time base 
import time 
 
#  Defining signals Pin
pin1 = 22
pin2 = 18
 
#  Set up GPIO Pattern 
GPIO.setmode(GPIO.BOARD)

#  Set the initial pin Initial value of signal 
GPIO.setup(pin1, GPIO.OUT, initial=GPIO.LOW) 
GPIO.setup(pin2, GPIO.OUT, initial=GPIO.LOW) 
 
#  Switch every two seconds LED Light status 
print(' Press ctrl+c Key to exit ')
try:
    while True: 
      time.sleep(2) 
      GPIO.output(pin1, GPIO.HIGH) 
      GPIO.output(pin2, GPIO.LOW) 
      time.sleep(2) 
      GPIO.output(pin2, GPIO.HIGH) 
      GPIO.output(pin1, GPIO.LOW) 
except KeyboardInterrupt:
    pass

#  Turn off the lights after stopping 
GPIO.setup(pin1, GPIO.OUT, initial=GPIO.LOW) 
GPIO.setup(pin2, GPIO.OUT, initial=GPIO.LOW) 

#  Clear resources 
GPIO.cleanup()

The final effect is shown in the figure below ：

3.3.2 C++ Realization

First you need to compile and install C++ Version of Jetson GPIO library . The specific command is as follows ：

git clone https://github.com/pjueon/JetsonGPIO
cd JetsonGPIO/build
make all
sudo make install

After the installation is completed, the generated libJetsonGPIO.a Copy the library file to /usr/local/lib/ Under the table of contents , take JetsonGPIO.h Copy the header file to /usr/local/include/ Below directory , Later we will write C++ The program can be used directly .

Restart the system after installation ：

sudo reboot

After restarting, we can use the following command to see C++ Version of GPIO The library installation is abnormal ：

find /home -name JetsonGPIO.h

The corresponding header file path will be output normally , As shown below ：

/home/qb/code/JetsonGPIO/include/JetsonGPIO.h

Next, create a file named CGpioDemo Folder , Then create... Respectively in this folder CMakelists.txt and CGpioDemo.cpp Two documents , among CMakelists.txt The contents are as follows ：

cmake_minimum_required (VERSION 3.8)

project ("CGpioDemo")

# Add the header file path 
include_directories(/home/qb/code/JetsonGPIO/include)

# Add library file path 
link_directories(/home/qb/code/JetsonGPIO/build)

#  Add source code to this project's executable .
add_executable (CGpioDemo "CGpioDemo.cpp")

# TODO:  If necessary , Please add test and install target .
target_link_libraries( CGpioDemo -lJetsonGPIO -lpthread)

Note that in the above code /home/qb/code/JetsonGPIO/include and /home/qb/code/JetsonGPIO/build You need to modify it in combination with your own path . These two paths are what we just compiled JetsonGPIO The library path .

CGpioDemo.cpp The contents are as follows ：

// Import files 
#include <iostream>
#include <chrono> 
#include <thread>
#include <signal.h>

// Import JetsonGPIO The library files 
#include <JetsonGPIO.h>

using namespace std;


//  Define the program exit flag 
bool done = false;

//  Define key interrupt signal callback function 
void signalHandler(int s) {
    done = true;
}

int main() {

    //  Definition Pin
    int pin1 = 22;
    int pin2 = 18;

    // Define callback function signals , When pressed Ctrl+C Key to exit the program 
    signal(SIGINT, signalHandler);

    // Set the mode  
    GPIO::setmode(GPIO::BOARD);

    //LED pin Initial settings 
    GPIO::setup(pin1, GPIO::OUT, GPIO::LOW);
    GPIO::setup(pin2, GPIO::OUT, GPIO::LOW);

    std::cout << " Press  CTRL+C  Exit procedure " << std::endl;

    int curr_value = GPIO::LOW;

    // every other 2 One second jump 
    while (!done) {

        std::this_thread::sleep_for(std::chrono::milliseconds(2000));
        GPIO::output(pin1, GPIO::HIGH);
        GPIO::output(pin2, GPIO::LOW);

        std::this_thread::sleep_for(std::chrono::milliseconds(2000));
        GPIO::output(pin2, GPIO::HIGH);
        GPIO::output(pin1, GPIO::LOW);
    }

    // Turn off the lights 
    GPIO::output(pin2, GPIO::LOW);
    GPIO::output(pin1, GPIO::LOW);

    // Clear resources 
    GPIO::cleanup();
    return 0;
}

And then according to C++ By cmake Recompile, .

4. Summary

This blog is from Jetson Nano Start to explain the installation of , Until the development of face detection 、 QR code scanner gun 、 Two colors LED The lamp and other cases are over . This tutorial is more practical , Teach readers how to build artificial intelligence products step by step from the concept of embedded products , Each case includes python and c++ Two versions . In order to adapt to new readers , The selected cases are relatively simple . Proceed from the actual situation , At present, artificial intelligence uses deep learning for high-precision reasoning , Compared with traditional algorithms, deep learning can greatly improve image detection 、 Recognition and semantic segmentation accuracy , However, how to effectively implement the deep learning algorithm has become the most popular vent at present , That is, how to efficiently implement the so-called edge computing .Jetson Nano As the artificial intelligence development board launched by NVIDIA , It is very suitable to use it as the embedded landing platform for deep learning , Especially in combination with the TensorRT Development kit , The models trained by various in-depth learning frameworks can further accelerate reasoning . From the data released by NVIDIA , Use TensorRT Can speed up reasoning 3 More than times the time . therefore , Future use Jetson Nano A key direction of is the landing application of deep learning .

This tutorial will continue to update later Jetson Nano Deep learning related content .

Publisher ： Full stack programmer stack length , Reprint please indicate the source ：https://javaforall.cn/151747.html Link to the original text ：https://javaforall.cn

原网站

版权声明
本文为[Full stack programmer webmaster]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/176/202206242246017566.html