MCPcopy
hub / github.com/scrapy/scrapy / url_is_from_spider

Function url_is_from_spider

scrapy/utils/url.py:36–38  ·  view source on GitHub ↗

Return True if the url belongs to the given spider

(url: UrlT, spider: type[Spider])

Source from the content-addressed store, hash-verified

34
35
36def url_is_from_spider(url: UrlT, spider: type[Spider]) -> bool:
37 """Return True if the url belongs to the given spider"""
38 return url_is_from_any_domain(url, _spider_domains(spider))
39
40
41def url_has_any_extension(url: UrlT, extensions: Iterable[str]) -> bool:

Calls 2

url_is_from_any_domainFunction · 0.85
_spider_domainsFunction · 0.85