MCPcopy
hub / github.com/scrapy/scrapy / page_count

Method page_count

scrapy/extensions/closespider.py:92–102  ·  view source on GitHub ↗
(self, response: Response, request: Request, spider: Spider)

Source from the content-addressed store, hash-verified

90 self._close_spider("closespider_errorcount")
91
92 def page_count(self, response: Response, request: Request, spider: Spider) -> None:
93 self.counter["pagecount"] += 1
94 self.counter["pagecount_since_last_item"] += 1
95 if self.counter["pagecount"] == self.close_on["pagecount"]:
96 self._close_spider("closespider_pagecount")
97 return
98 if self.close_on["pagecount_no_item"] and (
99 self.counter["pagecount_since_last_item"]
100 >= self.close_on["pagecount_no_item"]
101 ):
102 self._close_spider("closespider_pagecount_no_item")
103
104 def spider_opened(self, spider: Spider) -> None:
105 assert self.crawler.engine

Callers

nothing calls this directly

Calls 1

_close_spiderMethod · 0.95

Tested by

no test coverage detected