MCPcopy
hub / github.com/scrapy/scrapy / test_crawl_multiple

Method test_crawl_multiple

tests/test_crawl.py:407–430  ·  view source on GitHub ↗
(self, caplog: pytest.LogCaptureFixture)

Source from the content-addressed store, hash-verified

405
406 @coroutine_test
407 async def test_crawl_multiple(self, caplog: pytest.LogCaptureFixture) -> None:
408 settings_dict = get_reactor_settings()
409 runner_cls = (
410 CrawlerRunner
411 if settings_dict.get("TWISTED_REACTOR_ENABLED", True)
412 else AsyncCrawlerRunner
413 )
414 runner = runner_cls(settings_dict)
415 runner.crawl(
416 SimpleSpider,
417 self.mockserver.url("/status?n=200"),
418 mockserver=self.mockserver,
419 )
420 runner.crawl(
421 SimpleSpider,
422 self.mockserver.url("/status?n=503"),
423 mockserver=self.mockserver,
424 )
425
426 with caplog.at_level(logging.DEBUG):
427 await ensure_awaitable(runner.join())
428
429 self._assert_retried(caplog.text)
430 assert "Got response 200" in caplog.text
431
432 @coroutine_test
433 async def test_unknown_url_scheme(self, caplog: pytest.LogCaptureFixture) -> None:

Callers

nothing calls this directly

Calls 7

_assert_retriedMethod · 0.95
get_reactor_settingsFunction · 0.90
ensure_awaitableFunction · 0.90
getMethod · 0.45
crawlMethod · 0.45
urlMethod · 0.45
joinMethod · 0.45

Tested by

no test coverage detected