当前位置:网站首页>scrapy. Transfer of meta parameter data of request()

scrapy. Transfer of meta parameter data of request()

2022-06-22 10:26:00 Houxiaojoo

scrapy.Request() Of meta Parameters Data transfer


You need to parse the list page , When it is necessary to parse the details page ,
With The following template code is an example , Because originally only in parse Function defines ABC Class item,
So this requires item from parse() Function passed to parse_detail() Function , In order to continue to collect data item in , Then they pass it on together .

It needs to use scrapy.Request() Of meta Parameters ,
stay scrapy.Request() In the middle of , It needs to be written in the form of a dictionary , The key name of the dictionary can be chosen casually .

Such as meta={‘abcdefg’: item}

stay parse_detail() Take it out , The key name at the time of retrieval only needs to be consistent with the one above . Take out the code example :

item = response.meta.get(‘abcdefg’)

class RecruitnameSpider(scrapy.Spider):
	*************************

	#  Parse list page 
	def parse(self, response):
		result = response.*****************
		for r in result:
            item = ABC()
            item['a'] = ***
            item['b'] = ***
            item['c'] = ***
 			******
 			
			yield scrapy.Request(
		                url=new_detail_url,
		                callback=self.parse_detail,
		                #  Data transfer 
		                meta={
    'itemes': item}
		            )

	#  Analysis details page 
	def parse_detail(self, response):
		item = response.meta.get('itemes')
		item['d'] = ***
		item['e'] = ***
		item['f'] = ***
		

among ABC Class is items.py The class defined in the file .

原网站

版权声明
本文为[Houxiaojoo]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/173/202206221018065630.html