MCPcopy
hub / github.com/llmware-ai/llmware / encode

Method encode

llmware/util.py:1832–1836  ·  view source on GitHub ↗

Encode the sequence and return the token ids in a list.

(self, seq)

Source from the content-addressed store, hash-verified

1830 return True
1831
1832 def encode(self, seq):
1833
1834 """ Encode the sequence and return the token ids in a list. """
1835
1836 return self.tokenizer.encode(seq, add_special_tokens=False).ids
1837
1838 def decode(self, seq, strip_bos_token=True):
1839

Callers 15

gpu_availableMethod · 0.80
count_tokensMethod · 0.80
token_counterMethod · 0.80
inferenceMethod · 0.80
function_callMethod · 0.80
streamMethod · 0.80
ov_token_counterMethod · 0.80
ov_token_counterMethod · 0.80
token_counterMethod · 0.80
token_counterMethod · 0.80
token_counterMethod · 0.80
token_counterMethod · 0.80

Calls

no outgoing calls

Tested by

no test coverage detected