MCPcopy
hub / github.com/scrapy/scrapy / _test_spider

Method _test_spider

tests/test_spider_start.py:23–37  ·  view source on GitHub ↗
(
        self, spider: type[Spider], expected_items: list[Any] | None = None
    )

Source from the content-addressed store, hash-verified

21
22class TestMain:
23 async def _test_spider(
24 self, spider: type[Spider], expected_items: list[Any] | None = None
25 ) -> None:
26 actual_items = []
27 expected_items = [] if expected_items is None else expected_items
28
29 def track_item(item, response, spider):
30 actual_items.append(item)
31
32 crawler = get_crawler(spider)
33 crawler.signals.connect(track_item, signals.item_scraped)
34 await crawler.crawl_async()
35 assert crawler.stats
36 assert crawler.stats.get_value("finish_reason") == "finished"
37 assert actual_items == expected_items
38
39 @coroutine_test
40 async def test_start_urls(self):

Callers 4

test_start_urlsMethod · 0.95
test_startMethod · 0.95
test_start_subclassMethod · 0.95
_test_startMethod · 0.95

Calls 4

get_crawlerFunction · 0.90
crawl_asyncMethod · 0.80
connectMethod · 0.45
get_valueMethod · 0.45

Tested by

no test coverage detected