Based on the matrix calculation in the linear regression equation of the coefficient estimates

I. Description of the problem:

Regression analysis is to use known data to determine the coefficients in the regression equation.
Univariate linear regression analysis is a very simple and very basic regression theory, which can be used to describe the change trend of the linear relationship between two variables, and then predict the data at the unknown point.
Univariate linear regression analysis is to use known data to estimate the coefficients k and b in the equation y=kx+b. The common methods are: calculation mathematics method - least square method, statistical method - maximum likelihood estimation method, machine learning methods - perceptrons, etc., and can also be directly solved based on matrix operations, because the linear equation system with invertible coefficient matrix can be constructed using known data.
This paper takes the data of y = 2x + 1 for fitting as an example, and gives the method of estimating coefficients based on matrix operations and the matlab implementation.

II. Mathematical derivation

Problem description
Assuming known data points (xi,yi), i=1…n, where xi is not all 1, a linear trend can be preliminarily analyzed according to the scatter plot.
 Therefore, y=kx+b can be used to fit this set of data. At this time, y=kx+b can be called either the fitting function of the set of data (computational mathematics) or the regression equation (statisticalstudy).
The formula is derived as follows:
Substitute (xi,yi) into the linear function y=kx+b to get:

Write the matrix form as follows:

Remember

Then equation (2) can be written as

and the two column vectors of matrix X are linearly independent, then X'X is an invertible matrix, which gives:

SoThere are:

This will get the linear regression equationcoefficient.

Third, Matlab program

1. Draw a scatter plot

trainX = linspace( 0, 2, 50 );trainY = 2 * trainX + 1 + randn( size( trainX ) )*0.3;plot( trainX, trainY, 'b.', 'markersize', 20 )

The result is shown below:

From the imageIt can be seen that the known data points basically show a linear trend, so a univariate linear regression model can be used to fit this set of data.
2. Data Fitting

I = ones( size(trainX) );X = [ trainX', I' ];Y = trainY';B = inv( X' * X ) * X' * Y % regression coefficient, B(1) = k, B(2) = b

The regression coefficients are: k = B(1) = 2.0660, b = B(2) = 0.9925
3. Complete code

clear allclctrainX = linspace( 0, 2, 50 );trainY = 2 * trainX + 1 + randn( size( trainX ) )*0.3;% draw a scatter plotplot( trainX, trainY, 'b.', 'markersize', 20 )I = ones( size(trainX) );X = [ trainX', I' ];Y = trainY';B = inv( X' * X ) * X' * Y % regression coefficient, B(1) = k, B(2) = b% draw the regression function curve (straight line)x = [ -1 : 3 ];y = B(1) * x + B(2);hold onplot( x, y, 'r', 'LineWidth', 2 );title( 'y = 2x + 1' )

The result is as follows:

Modify the sentence "trainY= 2 * trainX + 1 + randn( size( trainX ) )*0.3;" is "trainY = -2 * trainX + 5 + randn( size( trainX ) )*0.3;", the following fitting image can be obtained:

Fourth, Supplementary Instructions

This method is only suitable for regression coefficient estimation in univariate or multiple linear regression analysis. For nonlinear regression, it can be considered to transform the linear regression analysis by logarithmic transformation and other methods before solving.