Zhiyuan community weekly 86: Gary Marcus talks about three linguistic factors that can be used for reference in large model research; Google puts forward the Wensheng graph model parti which is compar

Must see every week AI Point of view 、 Research and various resources , Don't miss an important piece of information ！ Welcome to click here , Subscribe to Zhiyuan community AI weekly .

Point of view

“ If we want to focus on the missing elements of the pre training big model , There are three key factors that should be considered ：

1. Reference resources （Reference）: Words and sentences do not exist in isolation . Language is a word / The connection between sentences and the outside world , The word sequences in large language models and their lack of connection with the outside world .

2. Cognitive models （Cognitive models）： The ultimate goal of the language system is to update the world , Continuous but dynamic perception . Large models do not produce such cognitive models , At least there is no such recognition that people can reliably use it .

3. form （Compositionality）： A complex whole , in the majority of cases , Be able to systematically explain the part where it is , And how these parts are organized together . image DALL-E When it comes to the composition of such a system , Face significant challenges . for example ,GPT...... Can not produce a reflection of the structural relationship between sentences 、 An interpretable representation .”

—— In recent days, , When it comes to the defects of the pre training large model , Professor, New York University Gary Marcus I think we can learn three important factors from linguistics .（ Extended reading ）

“（ In this paper ） I propose a general model called agent （Common Model of the Intelligent Agent） The concept of , Such decision makers （Decision Maker） It can be substantially and widely applied to psychology 、 Artificial intelligence 、 economics 、 Control theory and neuroscience ...... This generic model includes many aspects ： Decision makers interact directly with them , You need input 、 Output and goals , And the system composition within the decision-maker , For perception 、 Decision making 、 Internal assessment , And a world model . I noticed that they have different names in different disciplines , But essentially the same concept ...... Now is the time to endorse and build a substantial generic agent model , It can span and integrate multiple fields .”

—— In a new paper this year , The father of reinforcement learning Richard Sutton A general model of intelligent decision maker is proposed , Think it can unify the research of multiple disciplines .（ Extended reading ）

Scientist trends

1. Professor at the University of Texas at Austin Scott Aaronson induction OpenAI

6 month 20 Japan , Professor at the University of Texas at Austin Scott Aaronson Announced in OpenAI Work for a year , Its main responsibility is to think about AI security and alignment （AI Safety and Alignment） Theoretical basis .Scott Araonson Professor of computer science at the University of Texas at Austin , Director of quantum information center , His research areas include the performance and limitations of quantum computers , More generalized computational complexity theory, etc .2020 In, he was awarded a prize for his contribution in the field of quantum computing ACM Calculation Award .

2. AI2 CEO Oren Etzioni Leaving office , The board of directors has started recruiting successors

Oren Etzioni He is an honorary professor at the University of Washington , He was a professor in the Department of computer science and engineering . At present, he will continue to serve as CEO until this year 9 month 30 Japan , Then he served as a member of the board of directors and a consultant .AI2 By the late co-founder of Microsoft Paul Allen On 2014 An artificial intelligence research institute established in the United States in , The development includes NLP And other artificial intelligence research and engineering projects , Well known projects include academic search engines Semantic Scholar etc. .

3. Take stock of these years AI Start up companies that scientists join

In the past two years , Scientists who have resigned from large domestic and foreign technology companies , There are two main development paths ： One is to return from industry to academia , The second is to leave the big factory 、 Start your own business . This article takes stock of AI Domestic start-up companies joined by scientists , Like little ice 、 Innovative wisdom 、 Circular intelligence, etc , And the development of these scientists .

Research frontier

1. Google's proposal is based on Pathways Autoregressive Wensheng graph model Parti, The effect is comparable to Imgen
2. The father of reinforcement learning Richard Sutton writing ： A general model for pursuing intelligent decision makers
3. Tsinghua tianjixin X Chip boarding Science Robotics
4. OpenAI Propose a video pre training model VPT, You can play Minecraft game

Mechanism dynamics

1. OpenAI Three products （GPT-3、Copilot、DALL-E） The number of registered users has exceeded one million ,DALL-E This goal has been achieved in less than three months
2. CIFAR Announce the second phase of Pan Canada AI strategic , Will provide more than... Within ten years 4.43 Billion dollars in funding
3. Cohere、OpenAI、AI21 Three best practice guidelines for jointly publishing deployment models

Activities

1. Video playback | 2022 The video of the opening ceremony of Zhiyuan conference and sub forum was online
2. Event registration | China Artificial Intelligence Society ：2022 China International Intelligent Driving forum - Intelligent driving of technological change （6 month 25 Japan ）
3. Event registration | MIT、 Wisconsin 、UMass、 Researchers such as the University of Utah ：MLNLP The eighth academic seminar （6 month 26 Japan ）
4. Event registration | University of Illinois at urbana - Champagne （UIUC） Li Bo ： The combination of machine learning and knowledge reasoning in trusted machine learning （6 month 30 Japan ）

resources

1. FlagAI Feizhi ：AI Basic model open source project , Support one click call OPT Wait for the model
2. NATO group study report ： Knowledge representation and reasoning - Overview of current technology and future opportunities
3. 2021 China deep learning software framework Market Research Report
4. CVPR2022 Microsoft 《 Progress of visual language pre training 》 course

---

View Pre Workout 、 Weekly content in areas such as intensive learning , Welcome to click here .

Weekly clue collection and cooperation , Please contact the ：[email protected]