Get the short video! Batch download of Kwai video (with source code)

Hello everyone , I'm brother Chen ~

I believe everyone has contacted the short video platform , Like a certain sound 、 A hand waiting for a platform , Everyone is familiar with , So the technology shared by brother Chen today is ： Search for a video on a hand , And download ！

01 Get search links

Small partners who have written interfaces or developed websites know , When requesting a resource on a server , By visiting the link ( Interface ), The server responds and returns data .

1. Search request link

therefore , The first step is to get the search request link , Here, brother Chen obtains data by grabbing data packets .

Through here mitmproxy Grab a hand applet , If you don't know the little partner of this technical operation , You can refer to my previous article （ With 【 A journey 】 For example , About small program crawler technology ）, The article from 0 To 1 Explained how to use mitmproxy Collection applet .

For example, search ： ballad , View packets on the packet capture page , Find the following packet

Click on the packet

You can see that the request to search for a link is post The way , And the request header headers And request parameters , In request parameters keyword It's the keyword of the search , By modifying the keyword You can get different content .

2. Analyze packets

By viewing the returned data , You can find all the video content in the field feeds in

Extract fields ： Video address 、 user name 、 Cover image 、 Video name

mp4_url = i['mainMvUrls'][0]['url']
userName = i['userName']
pic_url = i['coverUrls'][0]['url']
caption = i['caption']

02 Request data

The request mode and parameters of the data packet are clear , And the data returned , And then we started to go through Python To construct request and process response data .

Request header and request parameters

headers = {
    'content-type':'application/json',
    'cookie':' Their own cookie'
}
s = json.dumps({
    "keyword": " ballad ",
    "pcursor": "",
    "ussid": ""
})

Request address ：

url = 'https://wxmini-api.uyouqu.com/rest/wd/search/feed'
r = requests.post(url, data=s,headers=headers).json()

printout ：

03 Save the data

We download and save the video and the corresponding cover image to the local , Here are two new functions , One is to download videos , One is to download the cover picture .

Download Video

# Download Video 
def download_mp4(mp4_name,mp4_url):
    dir = str(time.strftime('%y%m%d', time.localtime()))
    dir_path = "/"+dir
    #  Determine whether the folder exists 
    if not os.path.exists(dir_path):
        os.mkdir(dir_path)
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36'
    }
    r = requests.get(mp4_url, headers=headers, stream=True)
    if r.status_code == 200:
        #  Intercept image file name 
        with open(dir_path+"/"+mp4_name+".mp4", 'wb') as f:
            f.write(r.content)

Download the cover image

# Download the pictures 
def download_img(img_name,img_url):
    dir = str(time.strftime('%y%m%d', time.localtime()))
    dir_path = "/"+dir
    #  Determine whether the folder exists 
    if not os.path.exists(dir_path):
        os.mkdir(dir_path)
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36'
    }
    r = requests.get(img_url, headers=headers, stream=True)
    if r.status_code == 200:
        #  Intercept image file name 
        with open(dir_path+"/"+img_name+".jpg", 'wb') as f:
            f.write(r.content)
 
 

Both the video and the cover image are saved in a folder named after the date of the day , If there is no such folder, it will be created automatically .

Call these two functions

 # Start downloading pictures 
download_img(caption,pic_url)
# Start downloading videos 
download_mp4(caption,mp4_url)

After execution , Save results ：

You can see that the cover image and video are saved successfully ！ Its name is to name both with the video name .

04 Summary

This article explains the technology of a hand search video download , It is also a good little crawler for novices to learn , Little partner who wants to learn , must do Try it **！ must do Try it **！ must do Try it ！

The source address of this article ：https://gitee.com/lyc96/kuaishoushipinxiazai