【GCN-RS】Towards Representation Alignment and Uniformity in Collaborative Filtering (KDD‘22)

Towards Representation Alignment and Uniformity in Collaborative Filtering (KDD’22)

This paper mainly studies the characterization of collaborative filtering methods . The existing research mainly focuses on designing more powerful encoder To learn a better representation of . And lack of right CF The expected attributes of the representation in , This is important for understanding the existing CF The basic principle of the method and the design of new learning objectives are very important . This article defines Alignment and Uniformity Two indicators to measure the quality of characterization .

• Theoretically revealed BPR Loss and these two attributes （ Alignment and uniformity ） The connection between .
• from Alignment and Uniformity From the perspective of classic CF Methods analyze the learning process , better Alignment or Uniformity Will help improve the recommended performance
• According to the analysis results, a learning goal to directly optimize these two indicators is proposed DirectAU

Alignment and Uniformity

stay CF in , Use encoder $f()$ Map users and items to low dimensional representations $f(u),f(i)\in {\mathbb{R}}^{\mathrm{d}}$. For example, the matrix decomposition model is a embedding surface （LightGCN On this basis, neighborhood information is used ）.

The quality of representation is highly correlated with two key attributes ：Alignment and Uniformity. Given the data distribution $p_{data}(\cdot )$ And the distribution of positive sample pairs $p_{pos}(\cdot ,\cdot )$,Alignment For the definition of Standardization of positive sample pairs embedding Expectation of distance between , use $\tilde{f}()$ Express L2 Standardized characterization ：
$$l_{\text{align }}\triangleq \underset{\left(x,x^{+}\right)\sim p_{\text{pos }}}{\mathbb{E}}{\Vert f(x)-f\left(\widetilde{x^{+}}\right)\Vert }^2$$
Uniformity Defined as the logarithm of the mean of the paired Gaussian function ：
$$l_{\text{uniform }}\triangleq \log \underset{x,y\sim p_{\text{data }}}{\mathbb{E}}{e}^{-2\Vert f(x)-f(\tilde{y}){\Vert }^2}$$
These two indicators are very consistent with the goal of representational learning ： Positive samples should be close to each other , The random samples should be distributed on the hypersphere as evenly as possible .

The author first analyzes theoretically ,Perfect Alignment Refer to encoder f Encode all samples into the same representation ：$f(u)=\widetilde{f(}i)$ a.s. over $(u,i)\sim p_{\text{pos }}$;Perfect Uniformity Refer to encoder f The characterization of all samples is evenly distributed on a hypersphere .

Theoretical proof

The author put BPR loss Simplification , Prove if it exists Perfect Alignment and Perfect Uniformity Of encoder, yes BPR loss Lower bound of ：

The formula （4） The condition satisfied is if and only if f yes perfectly aligned; The formula （5） The condition satisfied is if and only if f yes perfectly uniform Of . therefore ,$L_{BPR}$ Greater than or equal to one independent of f The constant , If and only if f yes perfectly aligned and uniform The equal sign is established .

experimental analysis

In order to verify BPR And other things loss In the process of optimization, alignment and uniformity will be optimized , The author carried out experiments on different methods , As the training progresses , Alignment and uniformity will be optimized and improved accordingly . This also shows that CF The quality of user and product representation in depends on these two attributes . Better alignment or uniformity can help improve recommendation performance , It may be beneficial to optimize them at the same time .

After random initialization ,Uniformity very good ,Alignment Is very poor , The early learning process is mainly optimization Alignment, The performance improvement in the later stage mainly comes from Uniformity. It's easy to understand , It is easy to realize that positive samples are close , But on the basis of uniform sample distribution, it needs to spend effort Of .

DirectAU

After analysis, I found Alignment and Uniformity It is very important for the quality of user and item representation , So I designed a new learning goal , Directly optimize these two attributes ：
$$\begin{array}{rl}& l_{\text{align }}=\underset{(u,i)\sim p_{\text{pos }}}{\mathbb{E}}\Vert f(\widetilde{(}u)-f\widetilde{(}i){\Vert }^2\\ & l_{\text{uniform }}=\log \underset{u,u^{\prime }\sim p_{\text{user }}}{\mathbb{E}}{e}^{-2{\Vert f\widetilde{(u)}-f\left(\widetilde{u^{\prime }}\right)\Vert }^2}/2+{\log }_{i,i^{\prime }\sim p_{\text{item }}}^{\mathbb{E}}{e}^{-2{\Vert f(i)-f\left(i^{\prime }\right)\Vert }^2}/2.\\ & {\mathcal{L}}_{\text{DirectAU }}=l_{\text{align }}+\gamma l_{\text{uniform }}\end{array}$$
One advantage here is that negative samples are not needed to construct negative samples , Just need positive sample optimization $l_{align}$, then batch Sample optimization $l_{uniform}$,in-batch Sample of yes yes user and user Calculation ,item and item Calculation .

experimental result

Only MF As encoder Good results , If you use LightGCN As encoder, The improvement is also relatively large ：

This loss It can make embedding Of Uniformity Maintain the state of random initialization , That is, the state of uniform distribution .