[CUDA study notes] What is GPU computing

Series Article Directory

Chapter 1 Getting to Know CUDA

---

Foreword

This article is my study notes for the CUDA course, mainly for myself to review the past and learn new things in the future.It's an honor to be of any help to you.If you are a beginner in CUDA, please correct me if I am wrong.If there is any infringement, please contact the author to delete it.

---

I. CPU architecture vs GPU architecture

CPU Architecture

GPU Architecture

Here is an example of three GPUs.In general, GPU is divided into three layers: GPU-SM-SP
sm: (full name: Streaming Multiprocessor, stream multiprocessor)
sp: (full name: Streaming Processor, stream processor)

1. Early GPU (C2050):

2, Fermi Architecture:

3, newer version GPU:

Second, what is GPU computing:

• NVIDIA released CUDA, a general parallel computing platform and programming model built on NVIDIA's CPUs. Based on CUDA programming, the parallel computing engine of GPUs can be used to solve more complex computing problems more efficiently.
• GPU is not an independent computing platform, but needs to work in conjunction with the CPU, which can be regarded as a co-processor of the CPU. Therefore, when we say GPU parallel computing, we actually refer to the CPU+GPU-based heterogeneous computing.Construct the computing architecture.The cpu controls the architecture and logic of the entire program, and the gpu acts as a collaboration and a computing module, thereby accelerating the running speed of the entire program.
• In the Tesco computing architecture, the GPU and CPU are linked together through the PCIe bus to work together.The speed of PCIe can reach 16G/s, 32G/s
• The location of the CPU is called the host, and the location of the GPU is called the device.

Third, why use GPU computing:

• The powerful parallel computing engine of GPUs can greatly speed up the calculation, such as about 15 times.
• Supercomputers use accelerators, such as Tianhe, Summit.
• Machine learning and artificial intelligence need to train models and require a lot of calculations, especially dense matrix and vector calculations, GPU can be more than ten times faster
• One of the most successful applications of GPU is in the field of deep learning. Parallel computing based on GPU has become the standard for training deep learning models.

Fourth, the division of labor and cooperation between CPU and GPU:

• GPU includes more computing cores, which are especially suitable for data-parallel computing-intensive tasks, such as large-scale matrix operations.
• CPU has fewer computing cores, but it can implement complex logic operations, so it is suitable for control-intensive tasks.
• The off-the-shelf on the CPU is heavyweight, and the context switching overhead is large
• GPUs are lightweight due to the presence of many cores
• Heterogeneous computing platforms based on CPU+GPU can complement each other's advantages. The CPU is responsible for processing the serial programs responsible for the logic, while the GPU focuses on processing data-intensive parallel computing programs, so as to maximize the efficiency.

V. GPU computing architecture:

Six. Program structure:

7. Language selection:

• CUDA is a GPU programming model developed by NVIDIA. It provides a simple interface for GPU programming. Based on CUDA programming, applications based on GPU computing can be built.
• CUDA provides support for other programming languages, such as C/C++, Python, Fortran and other languages. Here we choose the CUDA C/C++ interface to explain CUDA programming.

8. Compiler:

• CUDA: NVIDIA, latest CUDA nvcc
• OS: Linux Ubuntu
• Linux Advantages:
  1. Easy to write compilation scripts, Makefile;
  2. Many command lines to try;
  3. Lightweight operating environment;
  4. Free

9. CUDA tools:

• Compiler: nvcc (C/C++)
• Debugger: nvcc-gdb
• Performance analysis: nsight, nvprof
• Libraries: cublas, nvblas, cusolver, cufftw, cuspares, nvgraph

X. Sample program:

• https://github.com/huiscliu/tutorials
• Env: Linux, Mac OSX