当前位置:网站首页>一个小工具可以更快的写爬虫
一个小工具可以更快的写爬虫
2022-06-28 07:08:00 【somenzz】
你好,我是 somenzz,我想你应该写过爬虫吧,就是那个 requests 库,它太好用了,应该没有人不知道 requests。
通常我们使用谷歌浏览器的开发者工具(F12)查找网络请求,然后将网络请求里面的 url、headers、cookie、user-agent、请求的参数等等,记录下来,然后,用 requests 模拟浏览器发出的请求。
这些简单无脑复制操作,让我觉得写爬虫真的累,有没有一个工具,可以直接把这个请求转化为 Python 代码?
还真有!
那就是 curlconverter[1]

用法很简单:

以谷歌浏览器为例:
1、在「开发者工具」中打开「Network」选项卡
2、右键单击(或按住 Ctrl 键单击)请求
3、点击 "Copy" → "Copy as cURL"
4、将其粘贴到上面的 curl 命令框中
选择需要转换语言(默认就是 Python),就可以复制代码了,非常方便:

最后的话
这个工具真是相见恨晚,可以大大节省你写爬虫花费的时间,请记住网址是:https://curlconverter.com/,如果觉得有帮助,请点在看扩散,如果有好的工具,也可以留言分享。
[1]
curlconverter: https://curlconverter.com/
边栏推荐
- [rust translation] implement rust asynchronous actuator from scratch
- 实时数据库 - 笔记
- Speech enhancement - spectrum mapping
- FPM tool installation
- [rust daily] May 24, 2020 rush, rocket, Mun, caspin
- Tryout title code
- Evolution of vivo push platform architecture
- A small code editor can also run programs -- a summary of sublime Text3 running programs in various languages
- An important term in MySQL -- CRUD
- Techo day Tencent technology open day, June 28 online waiting for you!
猜你喜欢
[c #] [reprint]furion frame address and tutorial address
Students who do not understand the code can also send their own token. The current universal dividend model can be divided into BSC and any generation B
Batch import of pictures into WPS table by date
The practice of traffic and data isolation in vivo Reviews
Compilation principles final review
最后的二十九天
Jinshan cloud team shared | 5000 words to understand how Presto matches with alluxio
MySQL master-slave replication, detailed configuration, create unable to connect processing prompt
LeetCode+ 51 - 55 回溯、动态规划专题
我的MVVM开源项目《出行防疫App》已发布
随机推荐
Puge -- singleton mode
ice - 资源
Kubernetes cluster lossless upgrade practice
Hungry for a long time, the brain will enter the "power saving mode"! Your senses are weakened and you can't see clearly
编译原理期末复习
Floating and positioning
What is a consistent hash? What scenarios can it be applied to?
Is it safe to open a stock trading account on your mobile phone?
Comprehensive analysis of real enterprise software testing process
Vivo browser rapid development platform practice - Overview
Leetcode+ 51 - 55 retrospective and dynamic planning topics
Triode driven brushless motor
"Jay bear" plummeted by 96.6%. Why is NFT with star goods cold?
OPC 协议认识
XML序列化向后兼容
An important term in MySQL -- CRUD
How bacnet/ip gateway collects data of building centralized control system
Top 25 most popular articles on vivo Internet technology in 2021
C language tutorial
[C#][转载]furion框架地址和教程地址