MCPcopy
hub / github.com/scrapy/scrapy / _process_spidermw_output_async

Method _process_spidermw_output_async

scrapy/core/scraper.py:450–463  ·  view source on GitHub ↗

Process each Request/Item (given in the output parameter) returned from the given spider. Items are sent to the item pipelines, requests are scheduled.

(
        self, output: Any, response: Response | Failure
    )

Source from the content-addressed store, hash-verified

448 return deferred_from_coro(self._process_spidermw_output_async(output, response))
449
450 async def _process_spidermw_output_async(
451 self, output: Any, response: Response | Failure
452 ) -> None:
453 """Process each Request/Item (given in the output parameter) returned
454 from the given spider.
455
456 Items are sent to the item pipelines, requests are scheduled.
457 """
458 if isinstance(output, Request):
459 assert self.crawler.engine is not None # typing
460 self.crawler.engine.crawl(request=output)
461 return
462 if output is not None:
463 await self.start_itemproc_async(output, response=response)
464
465 def start_itemproc(
466 self, item: Any, *, response: Response | Failure | None

Callers 1

Calls 2

start_itemproc_asyncMethod · 0.95
crawlMethod · 0.45

Tested by

no test coverage detected