Summary of statistical learning methods

Common statistical learning consists of the following ways :

One , Supervised learning

1. Generation method ： Learn from data the joint probability distribution p(y|x), And then find it as a prediction model . The model represents a given input x Generate input y The generative relationship of .eg,nb,hmm.
2. Discrimination method ： This method learns the decision function directly from the data f(x) Or conditional probability distribution as a prediction model , The discriminant method is concerned with the given input x, What kind of output should be predicted y.eg,k Nearest neighbor algorithm , perceptron , Decision tree ,LR, Maximum entropy model ,SVM,CRF.

  1, classification

1. perceptron ( Binary linear model )----> neural network ( A hidden layer is added to the former , Output layer neurons have more than one output )--> Deep neural network ( A neural network with many hidden layers , In essence, multilayer neural network and deep neural network DNN It's all one thing )
2. KNN
3. probability ( Naive Bayes （NB）,Logistic Regression（LR）, Maximum entropy MEM（ And LR Both belong to log linear classification model ）)
4. Support vector machine (SVM)
5. Decision tree (ID3、CART、C4.5)

6.     assembly learning{
           Boosting{
               Gradient Boosting{
                   GBDT
                   xgboost（ Tradition GBDT With CART As a base classifier ,xgboost It also supports linear classifiers , This is the time xgboost It's equivalent to taking L1 and L2 Logistic regression of regularized terms （ Classification problem ） Or linear regression （ The return question ）;xgboost yes Gradient Boosting An efficient system implementation of , Not a single algorithm .）
               }
               AdaBoost
           }   
           Bagging{
                Random forests 
           }
           Stacking
       }

    

  2, Regression method

1. Linear regression
2. Trees return
3. Ridge Ridge return
4. Lasso Return to

  3, Tagging

     Probability graph model ：HMM,MEMM（ Maximum Entropy Markov ）,CRF( Annotation is essentially a generalization of the classification problem , It is also a simple form of a more complex structure prediction problem , Its input is an observation sequence , The output is a sequence of tags or states . The goal is to learn a model , So that he can give a marker sequence to the observation sequence as a prediction . The commonly used statistical learning methods are ： hidden Markov model , Conditional random field （ Applied to part of speech tagging ）)

Two , Unsupervised learning

  1, clustering

1. Basic clustering （K—mean, Two points k-mean,K Median clustering ,GMM clustering ）      
2. Hierarchical clustering
3. Density clustering
4. Spectral clustering

  2, Theme model

1. pLSA
2. LDA Implicit Dirichlet analysis （ Applied to extract the subject features of the document ）      

  3, Correlation analysis

1. Apriori Algorithm
2. FP-growth Algorithm       

  4, Dimension reduction

1. PCA Algorithm
2. SVD Algorithm
3. LDA Linear discriminant analysis
4. LLE Local linear embedding       

  5, Anomaly detection

       Anomaly detection

3、 ... and , Semi-supervised learning

      

 namely ,semi-supervised, The learning process does not depend on external consultation , A method of automatically utilizing the distribution information contained in unlabeled samples , namely , The training set contains both labeled sample data and unlabeled sample data . for example , Semi supervision SVM, Semi supervised clustering, etc 

Four , Reinforcement learning

      

 namely ,reinforcement. Its essence is to solve decision making  problem , That is, making decisions automatically , And can make continuous decisions . for example , unmanned ,AlphaGo, Play games, etc .