Return True if more than half the query terms have a whole-word match in the document. Uses \b word boundaries so that a query term like "exa" do

Unigram tokens of a document's title, URL and extract (no bigrams).

Crawl the parent URL, score its outbound links, and collect the best for final ranking.

Fetch HaGeZi blacklist with caching.

Derive a psycopg2 DSN from Django's DATABASES['default'] setting. Supports both a pre-built URL (via dj_database_url) and a dict of individua

Get or create Redis connection for robots.txt caching.

Get or create Redis connection for robots.txt caching.

Index everything Super Search found against the query's unigrams/bigrams. Runs the blocking index write off the event loop and never lets an inde

Inline pydantic ``$defs`` so the schema is self-contained. ``model_json_schema`` emits nested models (e.g. ``ResultItem``) as ``$ref``s into

L2-normalised bag-of-words over title + extract, so cosine is a plain dot product.

Register the v2 search endpoint: returns SearchResponse with optional quota info.

Schedule periodic background tasks if they are not already queued. Uses django-background-tasks; requires `manage.py process_tasks` t

Cheap human-readable proxy used to score outbound links with the LTR model.

Count how many query terms appear as whole words anywhere in the URL. Uses \b word boundaries so that "exa" does not score a URL containing "

Serialise the data using JSON, compress it and store it at index i. If the data is too big, it will store the first items in the list

Resolve the requesting user from either an X-API-Key header or a JWT. Returns the user on success; raises ``HttpError(401)`` otherwise. Database

Select v2 source candidates: recommended + has on-site search + non-obscure.

Converts a `html` containing an HTML page into Unicode. Tries to guess character encoding from meta tag.

Lower-case all words in stoplist and create frozen set.

Extract a clean plain-text snippet from an HTML fragment using the justext pipeline.

Extract hrefs from all <a> elements in the DOM. justext drops paragraphs with no text nodes (e.g. icon-only anchors like <a href="/discord"><

Return (og:title, og:description) from Open Graph meta tags, or empty strings.