MCPcopy Create free account
hub / github.com/NVIDIA/TensorRT-LLM / _set_profiler

Method _set_profiler

tensorrt_llm/runtime/generation.py:243–256  ·  view source on GitHub ↗
(self)

Source from the content-addressed store, hash-verified

241 return context
242
243 def _set_profiler(self):
244 if self.profiler is not None:
245 return
246 assert self.context_0 is not None
247 assert self.context_1 is not None
248 self.profiler = _Profiler()
249 self.context_0.profiler = self.profiler
250 self.context_0.enqueue_emits_profile = False
251 self.context_1.profiler = self.profiler
252 self.context_1.enqueue_emits_profile = False
253 if self.engine.num_optimization_profiles == 2:
254 assert self.ctx_context is not None
255 self.ctx_context.profiler = self.profiler
256 self.ctx_context.enqueue_emits_profile = False
257
258 def __prepare(self, mapping: Mapping, engine_buffer):
259 self.runtime_rank = mapping.rank

Callers 1

decode_regularMethod · 0.80

Calls 1

_ProfilerClass · 0.85

Tested by

no test coverage detected