Read a tokenizer.json file from a Huggingface tokenizer and attempt to extract the symbols and merges for use with :clas::`mlx.data.core.BPETokeni

Read a sentencepiece file and decompose it to a symbol trie and BPE merges for use with :class:`mlx.data.core.BPETokenizer`. Because it isn't

Read an :class:`mlx.data.core.CharTrie` from a file with one token per line. Args: vocab_file (path or file like): The text file containi

Simple convenience function to utilize readinto if it is present in the python "file".

Seek the underlying python "file" to the requested position.

callback from the AVCodecContext to select the pixelFormat