[literature notes] pointmlp

Write it at the front , When I read this document , I found that there were no good reading notes of this document on the Internet at that time
So I wrote this book ...

ICLR 2022 ;Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework

Address of thesis ：Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework (arxiv.org)

Code address ：PointMLP-Pytorch

The whole author is spreading ： My method PointMLP Pure fresh and refined , Change numerous for brief

Current status

The author sums up ：

• Everyone is crowding out their heads to make an article on local feature extraction , Cause extractor （local geometric extractors） More and more complex , The performance is more and more stretched （ Although the accuracy has been improved ）
• Research work on local geometric feature extraction , Nearly saturated .（ The current extractor can extract features very well ）
• Current 3D point cloud applications（ Production application ） in , Prefer to use PointNet++ etc. （ Simple and lightweight ）, Complex extractors are rarely used
• And the author believes that the depth task of point cloud lives in the shadow of the image .（ It means that others will copy the new method of copying the image next door , Such as transformer）

The author thinks that ,

• Studying more detailed local geometric information may not be the key to point cloud analysis . We should re-examine the concise design concept in point cloud analysis .
• In this paper, the PointMLP follow PointNet++ Design concept of , But it has explored a simpler but deeper network architecture .
• There is no carefully designed local geometry extraction operation in this paper , Pure depth layering MLP Architecture can also show satisfactory performance and even better results .
• This article does not deliberately follow the trend of visual community , Instead, we pursue a simple and experienced point cloud analysis architecture

show Show ：

There is one saying. , It really performs in speed and accuracy SOTA

Network structure

This paper uses a simple feedforward residual MLP The Internet （ be called PointMLP） Let's learn the point cloud expression , The network is hierarchically aggregated MLP Extracted local features , And abandon the use of fine local geometry extractors .

The detailed framework is shown in the figure below ：

chart ：PointMLP Overview of a phase in . Given input point cloud ,PointMLP Use residual points MLP block Residual Point Block Gradually extract local features . At each stage , The author first uses the geometric affine module Geometric Affine Module Transform local points , Then extract them before and after the aggregation operation （ Geometric features ）. PointMLP Gradually expand the receptive field and simulate the complete geometric information of the point cloud by repeating multiple stages .

Simply review the existing networks

The author briefly introduces three representative network structures （ Don't focus ）

• PointNet++： The author believes that it is the most groundbreaking , Its kernel operator can be expressed as ：

among ：A Represents the aggregation process （PointNet++ Use in MaxPooling）;$\Phi $ Represents the feature calculation process （PointNet++ Use in MLP);f_ij It means the first one i The number of sampling points j Characteristics of neighbor points

• RSCNN …

• Point Transformer …

This article uses MLP

PointMLP One stage of （ namely ResP Block） It can be expressed as ：

In style ：

• Φpre (·) and Φpos (·) Is the residual point MLP block ： Shared Φpre (·) It aims to learn shared weights from local areas , and Φpos (·) Used to extract deep aggregation features .

Its mapping function can be written as a series of homogeneous residuals MLP block ：MLP(x) + x, among MLP from FC、 The normalization and activation layers are combined （ Repeat twice ）.

• A(·) Feature aggregation uses maximum pooling .

In layered and deep Networks , The above stages need to be repeated （ The stack ） s Time . Even though PointMLP The framework of is very simple , however ：

1. because PointMLP Only used MLPs, Nature is invariant to displacement , It perfectly fits the characteristics of point cloud
2. Connect by combining residuals ,PointMLP It can be easily extended to dozens of layers , Thus, the depth feature representation
3. Since there is no complex extractor and the main operation is only highly optimized feedforward MLP, Even if more layers are introduced ,PointMLP It can still be implemented effectively

In the experiment , repeat 4 Time （s=4） Stage , Use KNN（k=24） Find domain points

Geometric affine module

geometric affine module

** The module solves the problem ：** Due to the uneven density and geometric structure of the point cloud , For different local areas , need Use different extractors , To improve performance （ precision ）

Expressed as ：

In style ：

• α ∈ Rd and β ∈ Rd It's a learnable parameter
• ⊙ Express Hadamaji (Hadamard product)
• $\varepsilon $ = 1e−5 Is a decimal of numerical stability
• σ It's a scalar , Describes the characteristic deviations of all local groups and channels

By doing so , author Convert local points to Normal distribution , meanwhile Keep the original geometric attributes

Lightweight version PointMLP-elite

To further improve efficiency （ Faster , More lightweight ）, The author introduces elite edition , Greatly improve the speed of training and testing , Reduced memory requirements , Although the accuracy ratio PointMLP A little close , But better than most

How is it implemented ？

• In the middle FC The number of channels in the layer is reduced r times . This method will residual MLP Parameters from 2d² Reduce to (2d²)/r, default setting r = 4
• Slightly adjusted the network architecture
  • Reduce MLP block
  • Reduce embedding dimension

Besides , The author also tried grouping FC operation , But the experiment found that it did not improve the performance , Gave up .

experimental analysis

1. Conduct comparative tests on multiple benchmark data sets

• Object classification The experiment performed well , achieve SOTA, Detailed comparative data are listed in the paper
• Component segmentation The article only gives effect pictures , The comparative data is not listed in detail , I guess it's average
• Semantic segmentation The article didn't mention at all

2. Ablation Experiment

• Network depth （24、40、56 layer ）
• Geometric affine module
• Components （Φpre、Φpos、Affine）
• Res., Residual block

The details of this experiment are suggested to look directly at the original picture in the article .

summary

In this paper , The author proposes a simple and powerful architecture , be known as PointMLP, For point cloud analysis . PointMLP The key insight behind this is that complex local geometry extractors may not be important for performance .

The author first uses simple residuals MLP Indicates a local point , Because they are permutation invariant and simple . Then the author introduces a lightweight geometric affine module to improve the performance .

In order to further improve efficiency , The author also introduces a lightweight network PointMLP-elite.

Experimental results show that ,PointMLP It is superior to the related work on different benchmarks in terms of simplicity and efficiency .

The author hopes that this novel idea can inspire the community to rethink the network design and local geometry in the point cloud .

>>> End <<<

In fact, the author has a lot of literature notes , Since there are better notes on the Internet （ The same document ）, Don't make Internet garbage