Class MLXModelConfig

optillm/inference.py:292–299

Configuration for MLX models

@dataclass
class MLXModelConfig:
    """Configuration for MLX models"""
    model_id: str
    max_new_tokens: int = 4096
    temperature: float = 0.7
    top_p: float = 0.9
    repetition_penalty: float = 1.0
    enable_prompt_caching: bool = True

class MLXInferencePipeline:
    """MLX-based inference pipeline that mirrors PyTorch pipeline interface"""