MCPcopy Create free account
hub / github.com/NVIDIA/TensorRT-LLM / GenerationSequence

Class GenerationSequence

tensorrt_llm/runtime/kv_cache_manager.py:40–63  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

38
39
40class GenerationSequence(object):
41
42 def __init__(self, seq_idx, batch_idx):
43 self.seq_idx = seq_idx
44 self.batch_idx = batch_idx
45
46 def get_batch_idx(self) -> int:
47 """
48 Returns idx of sequence in batch
49 """
50 return self.batch_idx
51
52 def get_seq_idx(self) -> int:
53 """
54 Returns sequence idx
55 """
56 return self.seq_idx
57
58 def __eq__(self, another):
59 return hasattr(another, 'seq_idx') and self.seq_idx == another.seq_idx and \
60 hasattr(another, 'batch_idx') and self.batch_idx == another.batch_idx
61
62 def __hash__(self):
63 return self.seq_idx
64
65
66class BlocksManager(object):

Callers 9

test_gpt_attentionMethod · 0.90
allcloseMethod · 0.90
test_gpt_pluginMethod · 0.90
test_sequenceMethod · 0.90
test_kv_cache_managerMethod · 0.90
decodeMethod · 0.85

Calls

no outgoing calls

Tested by 8

test_gpt_attentionMethod · 0.72
allcloseMethod · 0.72
test_gpt_pluginMethod · 0.72
test_sequenceMethod · 0.72
test_kv_cache_managerMethod · 0.72