Continuously process batches with error handling.

Kept for migration compatibility (0010_apikey references this by name).

Lazy initialization of Redis connection.

Run indexing (backward compatibility).

Continuously run indexing with error handling.

Create sample FoundURL objects for testing

Sample vote data for testing

Run the LTR model over the given documents and return raw per-doc scores. Sync — call from a thread when used inside an async context.

A search-scoped ApiKey for verified_user. Has a .raw_key attribute for use in headers.

Set scopes=["crawl"] on all existing ApiKey rows that have no scopes.

Create a temporary directory for test data

Test batch queue operations in Redis

Test that built-in rules allow other legitimate domains.

Test that built-in rules allow legitimate domains.

Test that built-in rules blacklist bad patterns.

Create a static blacklist provider for tests to avoid network calls.

Test that canceling an already canceled subscription returns 409.

Test successful subscription cancellation at period end.

Test that canceling without authentication returns 401.

Test that canceling without a subscription returns 404.

Test that changing to an invalid plan returns 400.

Test that changing plan without a subscription returns 404.

Test successful plan change.

Test that changing plan without authentication returns 401.

When multiple docs share a URL, whichever wins carries the merged user_ids.

Test CombinedBlacklistProvider with is_domain_blacklisted method.

Test CombinedBlacklistProvider continues even if one provider fails.

Test URL crawling error handling

Test individual URL crawling functionality

End-to-end: a just-crawled URL is not put back on the queue, but a newly discovered link is.

A first-time crawled URL should come back with last_crawled set to now.

A user who has only accepted the API TOS cannot create a crawl key.