Make repetition penalty processor. Paper: https://arxiv.org/abs/1909.05858 Args: penalty (float): The repetition penalty factor

Implements mixed quantization predicates with similar choices to, for example, llama.cpp's Q4_K_M. Ref: https://github.com/ggerganov/llama.cpp

A function that multiplies two numbers Args: a: The first number to multiply b: The second number to multiply

Return the size of this cache in bytes

Generate the next batch of tokens. Returns: List of Response objects for each sequence in the batch.

Called to report training loss at specified intervals.

Called to report validation loss at specified intervals or the beginning.

Parse a GLM 4.7 tool call string into a name and arguments dict.