MCPcopy
hub / github.com/scrapy/scrapy

github.com/scrapy/scrapy @2.16.0 sqlite

repository ↗ · DeepWiki ↗ · release 2.16.0 ↗
6,284 symbols 26,787 edges 439 files 857 documented · 14%

Core symbols most depended-on inside this repo

get
called by 367
scrapy/http/headers.py
url
called by 155
tests/mockserver/ftp.py
items
called by 137
scrapy/http/headers.py
get_value
called by 95
scrapy/statscollectors.py
maybe_deferred_to_future
called by 81
scrapy/utils/defer.py
xpath
called by 81
scrapy/http/response/text.py
pop
called by 77
scrapy/pqueues.py
info
called by 73
scrapy/http/cookies.py

Shape

Method 4,413
Class 1,260
Function 582
Route 29

Languages

Python100%

Modules by API surface

tests/test_feedexport.py112 symbols
tests/spiders.py106 symbols
tests/test_downloader_handlers_http_base.py101 symbols
tests/test_spidermiddleware.py98 symbols
tests/test_exporters.py98 symbols
tests/test_crawler.py94 symbols
tests/test_engine.py86 symbols
tests/test_crawl.py76 symbols
tests/test_settings/__init__.py73 symbols
tests/test_crawler_subprocess.py72 symbols
tests/test_http_request_form.py71 symbols
tests/test_loader.py70 symbols

Used by 2 indexed graphs manifest dependencies, hub-wide

github.com/rmax/scrapy-redis
github.com/run-llama/llama_index

Dependencies from manifests, versioned

CPython1×
PyPy1×
Twisted21.7.0 · 1×
alabaster1.0.0 · 1×
annotated-types0.7.0 · 1×
attrs26.1.0 · 1×
automat25.4.16 · 1×
babel2.18.0 · 1×
certifi2026.2.25 · 1×
cffi2.0.0 · 1×
charset-normalizer3.4.6 · 1×
constantly23.10.4 · 1×

For agents

$ claude mcp add scrapy \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact