Generate a response to a prompt and send it to the client in a single batch. Args: prompt (List[int]): The tokenized pro

Handle a GET request for the /health endpoint.

Handle a GET request for the /v1/models endpoint.

Compute full log-likelihood of a string, with no truncation, for perplexity computation - We will use the full max context length of the model

Custom Metal kernel that performs matrix multiplication directly on packed weights and scales the output. This eliminates the need to sto

See https://github.com/openai/gpt-2/blob/master/src/encoder.py for the rationale.

This function is taken from the original tf repo. It ensures that all layers have a channel number that is divisible by the divisor It ca

Make a frequency penalty processor. Corresponds to the OpenAI option with the same name. Namely, subtracts ``penalty`` from a logit for

Make a presence penalty processor. Corresponds to the OpenAI option with the same name. Namely, subtracts ``penalty`` from a logit if th

Make a single streaming API request and return - whether the request succeeded - the request start time - the time of ev

Splits the weights into smaller shards. Args: weights (dict): Model weights. max_file_size_gb (int): Maximum size of each sh

MLX equivalent of torch.topk

Convert message content to a format suitable for `apply_chat_template`. The function operates on messages in place. It converts the 'content

Recurrence per (b, h): h_t = h_{t-1} * exp(g_t) h_t = h_t + k_t^T @ v_t y_t = (q_t @ h_t) * scale Returns y with shap

Save model weights and metadata index into specified directory.

Set up and return the argument parser.

Set up and return the argument parser.

Set up and return the argument parser.

A generator producing token ids based on the given prompt from the model. Args: prompt (mx.array): The input prompt. model (