MCPcopy Create free account
hub / github.com/NVIDIA/TensorRT-LLM / has_int8_kv_cache

Method has_int8_kv_cache

tensorrt_llm/quantization/mode.py:164–165  ·  view source on GitHub ↗
(self)

Source from the content-addressed store, hash-verified

162 return self._any(self.PER_GROUP)
163
164 def has_int8_kv_cache(self):
165 return self._any(self.INT8_KV_CACHE)
166
167 def has_fp8_kv_cache(self):
168 return self._any(self.FP8_KV_CACHE)

Callers 11

has_kv_cache_quantMethod · 0.95
to_dictMethod · 0.95
test_int8_kv_cacheMethod · 0.80
buildFunction · 0.80
gpt_attentionFunction · 0.80
get_kv_cache_dtype_byte_sizeMethod · 0.80
forwardMethod · 0.80
forwardMethod · 0.80
kv_dtypeMethod · 0.80
dummy_weights_awqFunction · 0.80
load_gemma_weightsFunction · 0.80

Calls 1

_anyMethod · 0.95

Tested by 1

test_int8_kv_cacheMethod · 0.64