Understanding of "the eigenvectors corresponding to different eigenvalues cannot be orthogonalized"

The following mainly focuses on the following issues ：

1. The physical meaning of Schmidt orthogonalization
2. The meaning of characteristic subspace
3. How to understand “ The real symmetric matrix can only orthogonalize the eigenvectors corresponding to the same eigenvalue ”（ a key ！）

The physical meaning of Schmidt orthogonalization

The process of Schmidt orthogonalization ：

Yes ${\alpha }_1$ and ${\alpha }_2$ Schmidt orthogonalization （ Hereinafter referred to as ” Orthogonalization “）

1. ${\beta }_1={\alpha }_1$
2. ${\beta }_2={\alpha }_2-\frac{({\alpha }_2,{\beta }_1)}{({\beta }_1,{\beta }_1)}{\beta }_1$

Understand it as a whole ,$({\alpha }_2,{\beta }_1)$ Express ${\alpha }_2$ and ${\beta }_1$ Inner product , Is a number ; similarly ,$({\beta }_1,{\beta }_1)$ It's also a number ;${\beta }_1$ Namely ${\alpha }_1$.

therefore ${\beta }_2$ It's essentially ${\alpha }_1$ and ${\alpha }_2$ The linear combination of .

Simplify ${\beta }_2={\alpha }_2-\frac{({\alpha }_2,{\beta }_1)}{({\beta }_1,{\beta }_1)}{\beta }_1$ ：

$$\begin{gathered} {\beta }_2={\alpha }_2-\frac{({\alpha }_2,{\beta }_1)}{({\beta }_1,{\beta }_1)}{\beta }_1 \\ ={\alpha }_2-\frac{|{\alpha }_2||{\beta }_1|cos\theta }{|{\beta }_1{|}^2}{\beta }_1 \\ ={\alpha }_2-\frac{|{\alpha }_2|cos\theta }{|{\beta }_1|}{\beta }_1 \\ ={\alpha }_2-|{\alpha }_2|\frac{{\beta }_1}{|{\beta }_1|}cos\theta \end{gathered}$$
among $\theta$ by ${\alpha }_1$ and ${\alpha }_2$ The angle between .

After Visualization ：

The meaning of characteristic subspace

Simple understanding ： For a matrix $A$ A characteristic value of $\lambda$ , The linear independent eigenvector corresponding to the eigenvalue is ${\alpha }_1$ 、 ${\alpha }_2$ 、 …… 、${\alpha }_s$, this $s$ The set of all vectors and zero vectors that can be formed by a linear combination of linearly independent eigenvectors is called $A$ The eigenvalues of the $\lambda$ The characteristic subspace of .

Well defined ：

Given $n$ Order matrix $A$ , Regulations $V_{\lambda }=\{\alpha \in C^n|A\alpha =\lambda \alpha \}$

（1）$V_{\lambda }$ Non empty ：$0\in V_{\lambda }$

（2）$V_{\lambda }$ Close to addition ：
$$\begin{gathered} \alpha ,\beta \in V_{\lambda } \\ \Rightarrow A\alpha =\lambda \alpha ,A\beta =\lambda \beta \\ \Rightarrow A(\alpha +\beta )=\lambda (\alpha +\beta ) \\ \Rightarrow \alpha +\beta \in V_{\lambda } \end{gathered}$$
（3）$V_{\lambda }$ Logarithmic multiplication closed ：
$$\begin{gathered} \alpha \in V_{\lambda } \\ \Rightarrow A\alpha =\lambda \alpha \\ \Rightarrow A(k\alpha )=k(A\alpha )=k(\lambda \alpha )=\lambda (k\alpha ) \\ \Rightarrow k\alpha \in V_{\lambda } \end{gathered}$$

（4）${\alpha }_1,{\alpha }_2,...,{\alpha }_s\in V_{\lambda }\Rightarrow k_1{\alpha }_1+k_2{\alpha }_2+...+k_s{\alpha }_s\in V_{\lambda },k_i\in C$

（5）$V_{\lambda }$ yes $n$ Subspace of dimensional complex vector space

$\lambda$ yes $A$ The eigenvalues of the $\iff$$V_{\lambda }\ne 0$

$V_{\lambda }$ be called $A$ The eigenvalues of the $\lambda$ The characteristic subspace of

How to understand “ The real symmetric matrix can only orthogonalize the eigenvectors corresponding to the same eigenvalue ”

The reason for this thinking comes from a class of linear algebra problems ：

Known matrix $A=?$（ Real symmetric matrix ）
（1） Find invertible matrix $P$, bring $P^{-1}AP$ It's a diagonal matrix
（2） Find the orthogonal matrix $Q$, bring $Q^{T}AQ$ It's a diagonal matrix

The students who have reviewed everything must be able to calculate it skillfully .
First question ： All linearly independent eigenvectors corresponding to each eigenvalue are arranged together ;
Second questions ： take The same eigenvalue After Schmidt orthogonalization of all corresponding linear independent eigenvectors , Normalize all eigenvectors , Then put all the linearly independent eigenvectors together .
  \space  
We all know that the eigenvectors corresponding to different eigenvalues of a real symmetric matrix are not only linearly independent , But also orthogonal （ This is easier to prove ）; So we only need to orthogonalize all eigenvectors corresponding to the same eigenvalue .
But you may have never seen a class of problems ,“ Prescribed matrix $A$ Not a real symmetric matrix , It is required to find the orthogonal matrix $Q$, bring $Q^{T}AQ$ It's a diagonal matrix ”, because In the process of similar diagonalization of non real symmetric matrices, the eigenvectors cannot be orthogonally normalized .
  \space  
Why is that ？ The reason will be explained below .

“ In the process of similar diagonalization of non real symmetric matrices, the eigenvectors cannot be orthogonally normalized ” The fundamental reason is that Schmidt orthogonalization of eigenvectors corresponding to different eigenvalues will change the eigenvalues corresponding to the new vector .

For example, two-dimensional vector , hypothesis $A=\left(\begin{array}{cc}1& -1\\ 0& 2\end{array}\right)$, Its eigenvalue ${\lambda }_1=1、{\lambda }_2=2$, The corresponding eigenvectors are ${\alpha }_1=\left(\begin{array}{c}1\\ 0\end{array}\right)、{\alpha }_2=\left(\begin{array}{c}1\\ -1\end{array}\right)$, By orthogonalizing them, we get ${\beta }_1=\left(\begin{array}{c}1\\ 0\end{array}\right)、{\beta }_2=\left(\begin{array}{c}0\\ 1\end{array}\right)$. But clearly ,$A{\beta }_2\ne {\lambda }_2{\beta }_2$, That is to say, after orthogonalization ${\beta }_2$ The corresponding characteristic value is no longer ${\lambda }_2$ 了 .

Why? “ Schmidt orthogonalization of eigenvectors corresponding to different eigenvalues will change the eigenvalues corresponding to the new vector ”

Visualize the above example ：

After Schmidt orthogonalization, the two vectors are indeed orthogonal , But according to the following properties of eigenvalues and eigenvectors , because ${\beta }_2$ Not by ${\alpha }_2$ Linear representation , therefore ${\beta }_2$ Not eigenvalues ${\lambda }_2$ Eigenvector of , That is to say ${\beta }_2$ be not in ${\lambda }_2$ In the characteristic subspace of , therefore ${\beta }_2$ The corresponding characteristic value is not ${\lambda }_2$. In this way, orthogonalization is meaningless .

nature ： if ${\alpha }_1,{\alpha }_2,...,{\alpha }_t$ All are matrices. $A$ Of are characteristic values $\lambda$ Eigenvector of , Then when $k_1{\alpha }_{+}{k}_2{\alpha }_2+...+k_t{\alpha }_t$ When it's not zero ,$k_1{\alpha }_{+}{k}_2{\alpha }_2+...+k_t{\alpha }_t$ Still matrix $A$ Belong to characteristic value $\lambda$ Eigenvector of .

---

To show more clearly , We expand into three-dimensional space .

Hypothetical matrix $A=\left(\begin{array}{ccc}\frac{3}{2}& 0& \frac{1}{2}\\ 0& 2& 0\\ \frac{1}{2}& 0& \frac{3}{2}\end{array}\right)$ Is a third-order real symmetric matrix , The three eigenvalues are ${\lambda }_1=1、{\lambda }_2=2、{\lambda }_3=2$, among ${\lambda }_2={\lambda }_3$, Because the number of linearly independent eigenvectors of a real symmetric matrix must be the same as the order , therefore ${\lambda }_1$ The corresponding eigenvector is ${\alpha }_1=(-1,0,1{)}^T$,${\lambda }_2$ and ${\lambda }_3$ The corresponding eigenvectors are ${\alpha }_2=(0,1,0{)}^T$ and ${\alpha }_3=(1,2,1{)}^T$, And the eigenvectors corresponding to different eigenvalues are orthogonal , namely $({\alpha }_1,{\alpha }_2)=0$,$({\alpha }_1,{\alpha }_3)=0$,${\alpha }_2$ and ${\alpha }_3$ Not orthogonal .

hypothesis ${\alpha }_1$、${\alpha }_2$ and ${\alpha }_3$ In 3D space, the following ：

Yes ${\alpha }_2$ and ${\alpha }_3$ After Schmidt orthogonalization ：

It can be found that the three vectors are orthogonal , And ${\beta }_2$ and ${\beta }_3$ The corresponding characteristic value is still $2$.

---

If we deal with the matrix $A$ Not a real symmetric matrix , Just a general matrix , That is to say, even the eigenvectors corresponding to different eigenvalues are not necessarily orthogonal , What happens if we orthogonalize two eigenvectors corresponding to different eigenvalues ？

First of all, we need to be clear about , If $\lambda$ The characteristic subspace of is determined by two linearly independent vectors , Then the eigensubspace of the eigenvalue should be composed of all the vectors in the plane determined by the two linearly independent vectors , That is to say, the eigenvector satisfying the eigenvalue should be in the plane , The vectors that do not satisfy the eigenvalue are all out of plane .

Suppose for a non real symmetric third-order matrix $A$ for , Its characteristic value is recorded as ${\lambda }_1、{\lambda }_2、{\lambda }_3$, among ${\lambda }_2={\lambda }_3$, Suppose the eigenvectors corresponding to the three eigenvalues ${\alpha }_1$、${\alpha }_2$ and ${\alpha }_3$ as follows ：

Obviously, none of the three is orthogonal .

If the ${\alpha }_2$ and ${\alpha }_3$ Schmidt orthogonalization , Because orthogonalization is only the addition and multiplication of two vectors , So it will not affect the plane formed by the two , in other words ${\alpha }_2$ and ${\alpha }_3$ After Schmidt orthogonalization ${\beta }_2$ and ${\beta }_3$ Still in ${\alpha }_2$ and ${\alpha }_3$ In the plane formed by , namely ${\beta }_2$ and ${\beta }_3$ In the eigensubspace of the same eigenvalue , That is to say, the eigenvalues of Schmidt orthogonality have not changed .

If the ${\alpha }_1$ and ${\alpha }_2$ Schmidt orthogonalization , take ${\alpha }_2$ As ${\beta }_1$, take ${\alpha }_1$ Convert to ${\beta }_2$, You can get ${\beta }_1={\alpha }_2=(1,1,0{)}^T$,${\beta }_2=(\frac{1}{2},1,\frac{1}{2}{)}^T$, The drawing is as follows ：

The light blue dotted line indicates ${\lambda }_1$ The characteristic subspace of , One dimensional space . After Schmidt orthogonalization ${\lambda }_1$ The corresponding eigenvectors ${\alpha }_1$ Turned into $(\frac{1}{2},1,\frac{1}{2}{)}^T$, Obviously the vector is no longer ${\lambda }_1$ In the characteristic subspace of , This explanation ${\beta }_2=(\frac{1}{2},1,\frac{1}{2}{)}^T$ The corresponding characteristic value is not ${\lambda }_1$;

Similarly , Can also be ${\alpha }_1$ As ${\beta }_1$, take ${\alpha }_2$ Convert to ${\beta }_2$, After Schmidt orthogonalization, we will find that ${\beta }_2$ No longer by ${\alpha }_2$ and ${\alpha }_3$ In the plane of , explain ${\beta }_2$ The characteristic value of is changed .

---

Through the example above , It can be concluded that the Schmidt orthogonalization of eigenvectors corresponding to different eigenvalues will lead to the change of eigenvalues corresponding to new vectors , Because the essence of Schmidt orthogonalization is the addition and multiplication of vectors , After Schmidt orthogonalization, the vector in the same feature subspace is still in the original feature subspace , But it is hard to say that the vectors of different feature subspaces are orthogonalized .

Tang Jiafeng's teaching plan mentioned a property ：
set up $A$ by $n$ Order matrix ,${\lambda }_1$,${\lambda }_2$ by $A$ Two different eigenvalues of , also $A\alpha ={\lambda }_1\alpha$,$A\beta ={\lambda }_2\beta$（$\alpha$ and $\beta$ It's a non-zero vector ）, For any $a\ne 0$,$b\ne 0$, vector $a\alpha +b\beta$ It must not be an eigenvector .
  \space  
Schmidt orthogonalization is a process of linear combination , According to this property, it can also be explained that the eigenvectors corresponding to different eigenvalues are not eigenvectors obtained by orthogonalization .

Schmidt orthogonalization when doing problems

If you want to compute an orthogonal matrix $Q$, bring $Q^{T}AQ=\Lambda$, be $A$ It must be a real symmetric matrix , And it is necessary to Schmidt orthogonalize the eigenvector corresponding to the same eigenvalue , Then normalize all eigenvectors , The normalized eigenvectors are arranged to obtain $Q$;

If you want to calculate a reversible matrix $P$, bring $P^{-1}AP=\Lambda$, be $A$ The requirements are not so strict . If $A$ Is a non real symmetric matrix , that $P$ It must not be transformed into an orthogonal matrix .