MCPcopy
hub / github.com/scrapy/scrapy / create_crawler

Method create_crawler

scrapy/crawler.py:360–380  ·  view source on GitHub ↗

Return a :class:`~scrapy.crawler.Crawler` object. * If ``crawler_or_spidercls`` is a Crawler, it is returned as-is. * If ``crawler_or_spidercls`` is a Spider subclass, a new Crawler is constructed for it. * If ``crawler_or_spidercls`` is a string, this fun

(
        self, crawler_or_spidercls: type[Spider] | str | Crawler
    )

Source from the content-addressed store, hash-verified

358 return self._crawlers
359
360 def create_crawler(
361 self, crawler_or_spidercls: type[Spider] | str | Crawler
362 ) -> Crawler:
363 """
364 Return a :class:`~scrapy.crawler.Crawler` object.
365
366 * If ``crawler_or_spidercls`` is a Crawler, it is returned as-is.
367 * If ``crawler_or_spidercls`` is a Spider subclass, a new Crawler
368 is constructed for it.
369 * If ``crawler_or_spidercls`` is a string, this function finds
370 a spider with this name in a Scrapy project (using spider loader),
371 then creates a Crawler instance for it.
372 """
373 if isinstance(crawler_or_spidercls, Spider):
374 raise ValueError(
375 "The crawler_or_spidercls argument cannot be a spider object, "
376 "it must be a spider class (or a Crawler object)"
377 )
378 if isinstance(crawler_or_spidercls, Crawler):
379 return crawler_or_spidercls
380 return self._create_crawler(crawler_or_spidercls)
381
382 def _create_crawler(self, spidercls: str | type[Spider]) -> Crawler:
383 if isinstance(spidercls, str):

Callers 7

crawlMethod · 0.80
crawlMethod · 0.80
get_crawlerFunction · 0.80

Calls 1

_create_crawlerMethod · 0.95

Tested by 5

get_crawlerFunction · 0.64