Represents a single inference request for benchmarking.
| 35 | |
| 36 | @dataclass |
| 37 | class SampleRequest: |
| 38 | """ |
| 39 | Represents a single inference request for benchmarking. |
| 40 | """ |
| 41 | |
| 42 | no: int |
| 43 | prompt: Union[str, Any] |
| 44 | history_QA: Union[str, Any] |
| 45 | json_data: Optional[dict] |
| 46 | prompt_len: int |
| 47 | expected_output_len: int |
| 48 | response_format: Optional[dict] = None |
| 49 | random_flag: bool = False |
| 50 | |
| 51 | |
| 52 | class BenchmarkDataset(ABC): |