Requests crawl page source code data

requests

• This is a web request based module in crawler
• effect ： Simulate the browser to initiate a request .
• Coding process ：
  • 1. Appoint url
  • 2. Initiate request
  • 3. Get response data （ Crawled page source code data ）
  • 4. Persistent storage

1 Crawl the page source code data of Sogou homepage

import requests

#  Appoint url
url = 'https://www.sogou.com/'
#  Initiate request  get The return value of the method is the corresponding object 
response = requests.get(url=url)
#  Obtain corresponding data 
# .text： What is returned is the response data of string type 
page_text = response.text
#  Persistent storage 
with open('./sougou.html', 'w', encoding='utf-8') as fp:
    fp.write(page_text)

The effect after running

Follow the column for more details