Context carries the per-request information needed for rate limit evaluation.

How a model is deployed on a set of GPUs. Attributes ---------- tp : tensor parallelism degree (GPUs per instance)

Validation error.

Find minimum-cost (n_s, n_l, gamma) fleet meeting a P99 TTFT SLO. Parameters ---------- gpu_short : GPU type for short-context poo

Raised when an import source cannot be converted safely.

Static specification for one pool in the fleet. Parameters ---------- pool_id : unique name (e.g. "short", "long") gpu

Sample request lengths from an empirical CDF. Supports a JSON file containing either: {"cdf": [[token_threshold, cumulative_frac], ...]}

Dataset metadata and configuration information. Attributes: name: Dataset name (e.g., "GPQA-Main", "ARC-Challenge") descript

Canonical inventory for one routing signal family.

Generate a Poisson arrival stream of requests. Parameters ---------- lam : arrival rate (requests per second) length_gen : Cdf

Event-driven fleet simulation. Usage ----- :: cfg = FleetConfig(pools=[PoolConfig(...)], router_type="LengthRouter",

Full fleet configuration. Parameters ---------- pools : ordered list of pool specs (router sees them in this order) router_typ

Standardized question representation for multi-choice reasoning tasks. Attributes: question_id: Unique identifier for the question

TextContent represents text content

Algorithm configuration for multi-model decisions. Specifies how multiple models in a decision should be orchestrated. Supports three catego

Dataset for dual-task learning with category classification and PII detection.

Entry holds a single lookup value together with provenance metadata.

Configuration parsing error.

Self-contained model architecture description used by fleet-sim.

Find the optimal xP + yD disaggregated fleet configuration. Parameters ---------- prefill_profile : GpuProfile for prefill (context) work

MemoryType represents the category of a memory in the agentic memory system.

Compute W, H, and KV-cache capacity from hardware + model + serving config. Usage:: from fleet_sim.hardware import H100_SXM from

Configuration for RAG (Retrieval-Augmented Generation) plugin. The RAG plugin retrieves relevant context from external knowledge bases and in

Physical constants for one GPU type. All bandwidth and capacity values are raw hardware specs; the ``effective_mem_bw`` property applies the

DeploymentBackend implementation for local Docker workflows.

A dual-purpose classifier using DistilBERT for both category classification and PII detection. This model uses a shared DistilBERT backbone

Trainer for the dual-purpose classifier.

Configuration for production labeling.

Configuration for Elo rating-based model selection. Uses Bradley-Terry model for pairwise comparisons, learning from user feedback.

Configuration for session-aware agentic model selection.

Generate synthetic data for testing dual-task training. Creates both category classification and PII detection labels.

Tool defines a tool function

Result of ensuring bootstrap workspace state.

Configuration for GMTRouter (Graph-based) model selection. Uses heterogeneous GNN for personalized routing decisions.

GPU performance profile with directly specified constants. Parameters ---------- name : human-readable label W

Analytical threshold optimization using cached traces (no live router). Constructor args: severity_fn: (item) -> int, severity weight for

QueryModality declares which modality of incoming request payload the embedding rule's query is computed from. The candidates remain text in every cas

Combined loss function for dual-task learning.

PyTorch Dataset for hallucination detection.

Descriptor for an LLM in the routing pool.

Configuration for ReMoM (Reasoning for Mixture of Models) algorithm. This algorithm performs multi-round parallel reasoning with intelligent synt

One inference request flowing through the fleet. Attributes ---------- req_id : globally unique request identifier arrival_time

Configuration for RouterDC (Dual-Contrastive) model selection. Matches queries to models using embedding similarity based on model descriptions.

SelectionMethod defines the type of model selection algorithm

Configuration for AutoMix (POMDP-based) model selection. Optimizes cost-quality tradeoff using Partially Observable MDP.

DependencyType classifies external dependencies

A single sample for hallucination detection evaluation.

Configuration for Hybrid model selection. Combines multiple selection methods with configurable weights.

Utility class for formatting prompts consistently across datasets.

Configuration for the canonical rl_driven selection algorithm. Matches decision.algorithm.rl_driven in the router config contract.

Server represents a gRPC server for the Envoy ExtProc

Trainer for hallucination detection with per-epoch benchmark evaluation.

A training sample for model selection.

AlgorithmTier classifies algorithms by production readiness

CacheBackendType defines the available cache backend implementations

Dataset that returns PAIRS of (clean, typo) versions for consistency training.

Data processing configuration.

Result from feedback classification.

Aggregated metrics from a completed fleet simulation.

Dataset for user-LLM interactions.

Configuration for a single model.

Configuration for the canonical multi_factor selector.

Dataset class for multitask learning.

Trainer for multitask BERT model.

================================================================================================ HALLUCINATION DETECTION GO BINDINGS (Token-level Dete

Complete pipeline configuration.

Plugin configuration with type validation. Configuration schema validation is performed in the validator module to ensure proper plugin-speci

Thread-safe progress reporter that queues SSE events.

Metrics for a single reasoning mode evaluation.

Response contains the output from looper execution

Configuration for router_replay plugin. The router_replay plugin captures routing decisions and payload snippets for later debugging and repl

Dataset for router training.