Send a CDP command, wait for response.

Clear stale rope_deltas state on Qwen3VL model. Qwen3VLModel.compute_3d_position_ids stores self.rope_deltas after processing image batches.

Distribute tile_infos dynamically across workers and collect results. Splits tile_infos into small work chunks and puts them into the shared

Emit a per-rank debug line when tracing distributed hangs.

Split a batch of inputs into chunks along batch dimension. Only includes tensor values in chunks (non-tensors are dropped) so that RandContex

Extract URL from example metadata. Args: example: Example dict with 'metadata' field. Returns: Extracted URL or None.

Get navigate URL for an article. Supports both file:// and http://.

Write log line directly to file descriptor (wandb-proof).

InfoNCE contrastive loss with hard negative support (from contrastors/loss.py). Inspired by: https://github.com/mlfoundations/open_clip/blob/main

Launch Chrome via subprocess, connect via websocket.

Get EVQA query image path. Dispatches by dataset_name: inaturalist | landmarks.

Wrap queries in chat template with retrieval instruction.

Launch Chrome via Playwright, get CDP session.

Return the path to the editorial photo for a QA row, or None.

Convert raw bytes, base64 string, dict with 'bytes' key, or PIL Image to a PIL Image. Returns None on failure.

Swap model params with EMA params. Call before eval.

Wrap document images in chat template with representation instruction. Returns batch-major pixel_values (B, max_patches, dim) so that chunk_i

Send a CDP command and wait for its response.

Filter out tiles with extreme aspect ratios. Args: tile_paths: List of tile image paths. max_aspect_ratio: Maximum allowed aspect

Format MC options as 'A. text1\nB. text2\n...

Create an OpenAI-compatible client for local or hosted endpoints.

All-gather tensors across GPUs while preserving gradients.

Generate response from messages with retry on timeout/connection errors. Args: max_retries: Retry count for timeout errors.

GradCache: large effective batch with constant memory (from contrastors/loss.py). 1. Forward all chunks WITHOUT grad → cache embeddings 2. Co

Run mini-v6/v8/... eval on every loaded test dataset and log per-set metrics.

Split an image into fixed-size tiles. Args: image_path: Path to the source image. output_dir: Directory to save tiles. ti

Quick HTTP check to see if kiwix-serve is responding.

Resolve chunk file path from article_id, tile_index, chunk_index.

Save query image from any task to disk. Returns path or None. Images saved to {base_dir}/{task_name}_images/{example_id}.png Works with PIL im

Build OpenAI-compatible messages for a LiveVQA MCQ call. Supports mixed content: images first, then text chunks, then the MCQ prompt.

Capture screenshot for a single example. Args: example: Example dict with metadata containing URL. screenshot_dir: Directory to s

Return raw per-hit dicts (title/text/url/score/...) for this example. Used by wrappers that need per-chunk granularity (e.g. RenderedTextWrap

SQuAD-style normalization: lowercase, strip punctuation, articles, whitespace.

Last-token pooling + L2 normalization. Matches the production embedding pipeline (embed_tiles.py direct_gpu backend).

Render a list of URLs to tiled JPEG images. Args: urls: URLs to capture. output_dir: Directory to write tile subdirectories into.

Retrieve relevant content for the query. Args: query: The question/query text. example: The full example dict (may co

Force lazy images to load by promoting data-src and setting loading='eager'.

Extract physical hostname from machine_id (``hostname-PID``). Machine IDs look like ``ip-172-31-2-101-4188035`` (AWS) or ``129-80-136-19-1450

Initialize screenshot utilities (lazy import).

Parse newly appended lines from a JSONL results file into state/cycle.

Wait for all document images to finish loading (load or error).

Find the best available Chrome binary. Auto-installs if none found. Search order: 1. CHROME_PATH env var 2. pixelrag-installed headless_s

Forward pass on chunks without grad, caching embeddings + RNG states.

Load model on one GPU, embed all assigned tiles, write partial .npz. This function runs in a child process with CUDA_VISIBLE_DEVICES set. Ar

Pre-compute chat template strings once to avoid per-batch overhead.

Collate: process queries as text, images as visual prompts. With hard negatives, document images are interleaved: [pos1, neg1a, neg1b, po

Average gradients across ranks for modules outside DDP's reducer path.

Render a PDF file to tiled JPEG images. Args: path: Path to the PDF file. output_dir: Directory to write the tile subdirectory in

Render a URL to tiled JPEG images. Args: url: URL to capture (http:// or https:// or file://). output_dir: Directory to write til

Launch one non-daemonic process per GPU and collect partial .npz paths. When reuse_workers=True, uses dynamic work distribution via a shared queu

Save LoRA weights + optimizer/scheduler state.

Save npz with optional compression.

Wait for a CDP event with the given method name. Args: method: CDP event method, e.g. "Page.frameStoppedLoading".

Build the embedding prompt using the model's chat template. Uses the official Qwen3-VL-Embedding prompt format: system: <instruction>

Embed tile infos with a pre-initialized engine function + prompt. Uses a pipeline: background threads read files, hash, and decode only uniqu

Extract source URLs from tile paths in results, preserving retrieval order.