MCPcopy Create free account
hub / github.com/NVIDIA/TensorRT-LLM / benchmark_serving.py

File benchmark_serving.py

tensorrt_llm/serve/scripts/benchmark_serving.py:None–None  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

1# Adopted from
2# https://github.com/vllm-project/vllm/blob/200bbf92e8861e2458a6f90bca73f40cc3b1ad1f/benchmarks/benchmark_serving.py
3# SPDX-License-Identifier: Apache-2.0
4r"""Benchmark online serving throughput.

Callers

nothing calls this directly

Calls 2

mainFunction · 0.70
keysMethod · 0.45

Tested by

no test coverage detected