当前位置:网站首页>scrapy. Transfer of meta parameter data of request()
scrapy. Transfer of meta parameter data of request()
2022-06-22 10:26:00 【Houxiaojoo】
scrapy.Request() Of meta Parameters Data transfer
You need to parse the list page , When it is necessary to parse the details page ,
With The following template code is an example , Because originally only in parse Function defines ABC Class item,
So this requires item from parse() Function passed to parse_detail() Function , In order to continue to collect data item in , Then they pass it on together .
It needs to use scrapy.Request() Of meta Parameters ,
stay scrapy.Request() In the middle of , It needs to be written in the form of a dictionary , The key name of the dictionary can be chosen casually .
Such as meta={‘abcdefg’: item}
stay parse_detail() Take it out , The key name at the time of retrieval only needs to be consistent with the one above . Take out the code example :
item = response.meta.get(‘abcdefg’)
class RecruitnameSpider(scrapy.Spider):
*************************
# Parse list page
def parse(self, response):
result = response.*****************
for r in result:
item = ABC()
item['a'] = ***
item['b'] = ***
item['c'] = ***
******
yield scrapy.Request(
url=new_detail_url,
callback=self.parse_detail,
# Data transfer
meta={
'itemes': item}
)
# Analysis details page
def parse_detail(self, response):
item = response.meta.get('itemes')
item['d'] = ***
item['e'] = ***
item['f'] = ***
among ABC Class is items.py The class defined in the file .
边栏推荐
- Qt编写物联网管理平台36-通信协议
- JG_fx_20220620
- MySQL中from_unixtime和unix_timestamp处理数据库时间戳转换问题-案例
- 2022年深入推进IPv6部署和应用,该如何全面实现安全升级改造?
- 【Shell】常用指令集锦
- Catch up with this big guy
- Byte 2: why is the key of the redis master node expired, but the expired data is still read from the secondary node? How to solve it?
- [deep learning] tensorflow, danger! Google itself is the one who abandoned it
- 普乐蛙VR台风逃生体验VR灾害自救模拟系统VR科普知识设备
- 传iPhone 14将全系涨价;TikTok美国用户数据转移到甲骨文,字节无法访问;SeaTunnel 2.1.2发布|极客头条...
猜你喜欢
随机推荐
符合我公司GIS开源解决方案的探讨
【无标题】#修复日志#
Encryption market plummeted, Seth triggered a new round of concern
蓝牙、wifi、zigbee和lora、NB-lot,通话信号,网络信号4G
加密市场暴跌,stETH引发新一轮担忧
Who says PostgreSQL has no reliable high availability (2)
scrapy.Request() 的 meta参数 数据的传递
【科普】一文弄懂监督式学习、非监督式学习以及强化式学习
数据智能基础设施升级窗口将至?看九章云极 DingoDB 如何击破数据痛点
Cobalt Strike 从入门到入狱(三)
Cobalt Strike 從入門到入獄(三)
Cobalt strike from starting to Imprisonment (3)
Analysis of thinkphp5.0.24 deserialization vulnerability
Force buckle 1108 IP address invalidation
Catch up with this big guy
LeetCode Algorithm 剑指 Offer 18. 删除链表的节点
thinkphp3.2.3日志包含分析
C language to write a two-way linked list
无线802.1x认证简介及配置方法
三个月让软件项目成功“翻身”!









