Class SampleRequest

tensorrt_llm/serve/scripts/benchmark_dataset.py:204–212

Represents a single inference request for benchmarking.

@dataclass
class SampleRequest:
    """
    Represents a single inference request for benchmarking.
    """

    prompt: Union[str, Any]
    prompt_len: int
    expected_output_len: int
    multi_modal_data: Optional[dict] = None


# -----------------------------------------------------------------------------