Computing trends in three times of machine learning

link ：https://arxiv.org/pdf/2202.05924.pdf

 

Calculation 、 Advances in data and algorithms are leading to modern machine learning (ML) The three basic factors of progress . This paper studies the trend of the most easily quantifiable factors
– Calculation .

The article shows that , stay 2010 Years ago , The growth of training calculation conforms to Moore's law , About every 20 Double in months . since 2010 Since the advent of deep learning in the early S , The expansion speed of training calculation is accelerated , About every 6 Double in months . 2015 end of the year , As the company develops large-scale ML Model , The requirements for training calculation have increased 10 To 100 times , A new trend has emerged . Based on these observations , This article will ML The history of computing is divided into three eras ： The era before deep learning 、 deep The times of learning and large-scale . Overall speaking , The work of this paper emphasizes the training of advanced ML The rapidly growing computing needs of the system .