load_streaming_state(path, metadata_path) Loads a streaming state from a safetensors file and its associated metadata json file. Parame

Deactivate CUDA Graphing for all the calls in this context manager.

Number of active codebooks.

Total number of codebooks.

Active number of codebooks used by the quantizer.

Pad for a convolution to make sure that the last window is full. Extra padding is added at the end. This is required to ensure that we can rebuild

Reset the state, meaning the next call we get CUDA Graphed again. Useful if some shapes have changed, or external state (e.g. KVCache) has cha

Save the streaming state, including that of sub-modules, to the given paths. Parameters ---------- save_path : str

This returns the quantizer that models the first level of the hierarchy (typically semantic).

This returns the quantizer that models the first level of the hierarchy (typically semantic). In this case, it's the quantizer itself.

Deactivates or changes the context span (in time steps) in a model. Args: model (nn.Module): model over which to look for attentions.

Set the number of active codebooks.

Set the number of active codebooks.

Set the active number of codebooks used by the quantizer.

Set the streaming state, including that of sub-modules.

Set the streaming state in-place, including that of sub-modules using a flattened-state dict.

Custom implementation of checkpointing in PyTorch as the builtin implementation is broken when using torch compile. Only supports wrapping a `nn.M

Token id for the start of sequence (text).

Token id for text padding.

torch.compile creates a huge pool of processes, even when not using the function at all, e.g. with Dora. This can polute stderr when doing CTRL+C.

Total number of codebooks.

Total number of codebooks.

Total number of quantizer codebooks available.

Special value that can be provided in the prompt to indicate that this specific value should be predicted and sampled. This allows for partial

Special value in the input tokens, indicating that no sampling should happen for that value, and no input should be given to the model.