This method starts a :mod:`~twisted.internet.reactor`, adjusts its pool size to :setting:`REACTOR_THREADPOOL_MAXSIZE`, and installs a DNS resolver based on :setting:`DNSCACHE_ENABLED`. If ``stop_after_crawl`` is True, the reactor will be stopped after all cr
(
self, stop_after_crawl: bool = True, install_signal_handlers: bool = True
)
| 764 | return self.stop() |
| 765 | |
| 766 | def start( |
| 767 | self, stop_after_crawl: bool = True, install_signal_handlers: bool = True |
| 768 | ) -> None: |
| 769 | """ |
| 770 | This method starts a :mod:`~twisted.internet.reactor`, adjusts its pool |
| 771 | size to :setting:`REACTOR_THREADPOOL_MAXSIZE`, and installs a DNS |
| 772 | resolver based on :setting:`DNSCACHE_ENABLED`. |
| 773 | |
| 774 | If ``stop_after_crawl`` is True, the reactor will be stopped after all |
| 775 | crawlers have finished, using :meth:`join`. |
| 776 | |
| 777 | :param bool stop_after_crawl: stop or not the reactor when all |
| 778 | crawlers have finished |
| 779 | |
| 780 | :param bool install_signal_handlers: whether to install the OS signal |
| 781 | handlers from Twisted and Scrapy (default: True) |
| 782 | """ |
| 783 | from twisted.internet import reactor |
| 784 | |
| 785 | if stop_after_crawl: |
| 786 | d = self.join() |
| 787 | # Don't start the reactor if the deferreds are already fired |
| 788 | if d.called: |
| 789 | return |
| 790 | d.addBoth(self._stop_reactor) |
| 791 | |
| 792 | self._setup_reactor(install_signal_handlers) |
| 793 | reactor.run(installSignalHandlers=install_signal_handlers) # blocking call |
| 794 | |
| 795 | |
| 796 | class AsyncCrawlerProcess(CrawlerProcessBase, AsyncCrawlerRunner): |
nothing calls this directly
no test coverage detected