Basics of reptile B1 - scrapy (learning notes of station B)

Catalog

1. Basic concepts

2. Pychram Medium Scrapy

 3. A basic description of the parsing page （CCS Use of selectors ）

 4. utilize Items.py receive data

  5. stay Setting.py Set crawler camouflage in

 6. Run the crawler project and save it as SCV file

7. Paging crawling Douban data

---

Study address

1. Basic concepts

 

  

2. Pychram Medium Scrapy

 

 

 3. A basic description of the parsing page （CCS Use of selectors ）

 

 4. utilize Items.py receive data

 items.py In file ：

Instantiation MovieItem(), obtain movie_item object , And will css The page data extracted by the selector is stored in movie_item In the object ： 

 

 

 5. stay Setting.py Set crawler camouflage in

 6. Run the crawler project and save it as SCV file

 

7. Paging crawling Douban data