Paper notes: multi label learning dm2l

Abstract : Share your understanding of the paper . See the original Ma, Z.-C., & Chen, S.-C. (2021). Expand globally, shrink locally: Discrimi-nant multi-label learning with missing labels. Pattern Recognition, 111, 107675.

1. Contribution of thesis

• Optimize both globally and locally ;
• Support nonlinear transformation with kernel function ;
• Theoretical analysis in place .

2. Basic symbols

Symbol	meaning	explain
$\mathbf{X}\in {\mathbb{R}}^{n\times d}$	Attribute matrix
${\mathbf{X}}_k\in {\mathbb{R}}^{n_k\times d}$	With $k$ Attribute submatrix of labels
$\mathbf{Y}\in \{-1,+1{\}}^{n\times c}$	Label matrix
$\tilde{\mathbf{Y}}\in \{-1,+1{\}}^{n\times l}$	Observed label matrix
$\mathbf{\Omega }=\{1,\dots ,n\}\times \{1,\dots ,c\}$	Observation tag location set
$\mathbf{W}\in {\mathbb{R}}^{m\times l}$	coefficient matrix	It's still a linear model
${\mathbf{w}}_i\in {\mathbb{R}}^m$	The coefficient vector of a label
$\mathbf{C}\in {\mathbb{R}}^{l\times l}$	Label correlation matrix	Pairwise correlation , Does not satisfy symmetry

3. Algorithm

Basic optimization objectives :
$$\min \frac{1}{2}\Vert R_{\Omega }(\mathbf{X}\mathbf{W})-\tilde{\mathbf{Y}}{\Vert }_F^2+{\lambda }_d\Vert \mathbf{X}\mathbf{W}{\Vert }_{\ast }\text{\hspace{1em}(1)}$$
among ,

• The loss function section does not consider missing values , This is a normal operation .
• Kernel regularity (nuclear norm) The prediction matrix is partially considered , Not just $\mathbf{X}\mathbf{W}$, It's a little strange. .

The optimization objective after considering the label structure :
$$\min \frac{1}{2}\Vert R_{\Omega }(\mathbf{X}\mathbf{W})-\tilde{\mathbf{Y}}{\Vert }_F^2+{\lambda }_d\left(\sum _{k=1}^c\Vert {\mathbf{X}}_k\mathbf{W}{\Vert }_{\ast }-\Vert \mathbf{X}\mathbf{W}{\Vert }_{\ast }\right),\text{\hspace{1em}(2)}$$
among ,

• $\Vert {\mathbf{X}}_k\mathbf{W}{\Vert }_{\ast }$ It expresses the local label structure , The smaller the better ;
• $\Vert \mathbf{X}\mathbf{W}{\Vert }_{\ast }$ It expresses the global label structure , The bigger the better ( More separable , The higher the amount of information ).
• These two points are the source of the topic .

Add nonlinear optimization objective :
$$\min \frac{1}{2}\Vert R_{\Omega }(\mathbf{X}\mathbf{W})-\tilde{\mathbf{Y}}{\Vert }_F^2+{\lambda }_d\left(\sum _{k=1}^c\Vert f({\mathbf{X}}_k)\mathbf{W}{\Vert }_{\ast }-\Vert f(\mathbf{X})\mathbf{W}{\Vert }_{\ast }\right),\text{\hspace{1em}(5)}$$
among $f(\cdot )$ Nonlinear transformation caused by kernel function .

4. Summary

• Another pile of theoretical proofs .