MCPcopy Create free account
hub / github.com/ml-explore/mlx-examples / eos_token

Method eos_token

clip/tokenizer.py:36–37  ·  view source on GitHub ↗
(self)

Source from the content-addressed store, hash-verified

34
35 @property
36 def eos_token(self):
37 return self.vocab[self.eos]
38
39 def bpe(self, text):
40 if text in self._cache:

Callers

nothing calls this directly

Calls

no outgoing calls

Tested by

no test coverage detected