Simple wrapper around `nn.Conv1d` that accepts dimension last.

Dataclass that stores completion parameters for a given chat format and create_chat_completion request. prompt contains the formatted prompt

桥接线程与 asyncio 的简易队列，确保跨线程 put 安全

The rotary position embeddings from RoFormer_ (Su et. al). A crucial insight from the method is that the query and keys are transformed b

OpenAI style error response

Batch cross-thread queue puts into the target event loop.

r""" Hedgehog feature map as introduced in `The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry <https://arxiv.org

High-level Python wrapper that manages the lifecycle and configuration of the llama.cpp sampler chain.

r""" Simple linear mapping feature map as in `Finetuning Pretrained Transformers into RNNs <https://arxiv.org/abs/2103.13076>`_

Represents a text generation task with associated configuration and state. This class encapsulates all necessary information for managing a singl

Listen for external interruption signals to trigger a stop condition. When an external thread calls `llama.abort()`, a loop interrupt is gene

核心引擎类，负责处理核心功能，包括： - Worker 管理 - Task 管理 - 线程与 asyncio 间的安全通信

Python wrapper for llama.cpp custom sampler. apply_func: Callable receiving llama_token_data_array and modifying logits in-p

r""" Deterministic Parameter-Free Projection (DPFP) feature map in `Linear Transformers Are Secretly Fast Weight Programmers <https://arxiv.or

Loader for IndexedDataset

A node in the prefix tree (Trie).

异步生成任务的控制器，管理单个生成请求的生命周期