Initialize the request batcher Args: max_batch_size: Maximum number of requests per batch max_wait_m

Initialize the DeepConf processor. Args: model: The language model tokenizer: The tokenizer

Initialize the confidence calculator. Args: window_size: Size of sliding window for group confidence

Initialize the threshold calibrator. Args: variant: "low" (aggressive, top 10%) or "high" (conservative, top 90%

Initialize the JSON generator with a specific model.

Initialize with optional custom config path

Initialize the complexity classifier. Args: model_name: HuggingFace model name or path for the classifier

Initialize the steering vector manager. Args: dataset_name: Name of the HuggingFace dataset containing steering

Initialize the steering hook. Args: manager: The steering vector manager layer_num: The layer number

Initialize the AutoThink processor. Args: config: Configuration dictionary tokenizer: Model tokenize

Initialize the AutoThink processor. Args: model: Language model tokenizer: Model tokenizer

Initialize default values for mutable fields

Perform batch generation using MLX model Args: input_ids: Batched input token IDs attention_mask: At

Tokenize a batch of prompts with padding Args: prompts: List of text prompts Returns:

Call the model to perform aggregation (synchronous for executor)

Main health check loop

Use LLM to classify problem type

Strategy 4: Clean common formatting issues and parse.

Return the next response in sequence

Strategy 1: Find first complete JSON object with balanced braces.

Strategy 2: Use regex with non-greedy matching.

Strategy 3: Extract between ```json``` code blocks.

Extract reasoning strategy using LLM analysis

Generate solution enhanced with peer strategies

Parse content from boxed answers

Aggregate a single group of solutions

Get number of available slots, None if unlimited.

Get average latency

Background thread that forms and processes batches