MCPcopy
hub / github.com/scrapy/scrapy / _process_links

Method _process_links

scrapy/linkextractors/lxmlhtml.py:251–256  ·  view source on GitHub ↗
(self, links: list[Link])

Source from the content-addressed store, hash-verified

249 return any(allowed) and not any(denied)
250
251 def _process_links(self, links: list[Link]) -> list[Link]:
252 links = [x for x in links if self._link_allowed(x)]
253 if self.canonicalize:
254 for link in links:
255 link.url = canonicalize_url(link.url)
256 return self.link_extractor._process_links(links)
257
258 def _extract_links(self, *args: Any, **kwargs: Any) -> list[Link]:
259 return self.link_extractor._extract_links(*args, **kwargs)

Callers 1

extract_linksMethod · 0.95

Calls 1

_link_allowedMethod · 0.95

Tested by

no test coverage detected