Take a single document and the LM results and evaluates, returning a dict where keys are the names of submetrics and values are the values of

Take a single document and the LM results and evaluates, returning a dict where keys are the names of submetrics and values are the values of

Take a single document and the LM results and evaluates, returning a dict where keys are the names of submetrics and values are the values of

Take a single document and the LM results and evaluates, returning a dict where keys are the names of submetrics and values are the values of

Take a single document and the LM results and evaluates, returning a dict where keys are the names of submetrics and values are the values of

Take a single document and the LM results and evaluates, returning a dict where keys are the names of submetrics and values are the values of

Upload the model to the Hugging Face Hub. Parameters: repo_id (`str`): The name of the repository you wa

Computes: C = silu(A * B1) * (A * B2) A is of shape (M, K) float16 B is of shape (K//8, N) int32 C is of shape (M, N)

Compute the matrix multiplication C = A x B. A is of shape (M, K) float16 B is of shape (K//8, N) int32 C is of shape (M, N) float16

Query OpenAI chat model and save the results to output_path. `instances` is a list of dictionaries, each dictionary contains a key "prompt" a

Register a string as contamination to be removed, e.g. a test set This breaks the dirt_string into ngrams to store for future cleaning

Register a string as contamination to be removed, e.g. a test set This breaks the dirt_string into ngrams to store for future cleaning

Resizes input token embeddings matrix of the model if `new_num_tokens != config.embedding_size`. Takes care of tying weights embeddi

Restores a checkpoint to the model and optimizer. Returns the remaining trainer state.

Run an experiment on Beaker. Example: $ gantry run --name 'hello-world' -- python -c 'print("Hello, World!")'

Saves a HF dataset to disk under the `datasets_dir`. It can be used to add a HF dataset to `olmo_data` as follows: ``` import datase

alias of save_quantized

Set or update Gantry's GitHub token for the workspace. You can create a suitable GitHub token by going to https://github.com/settings/tokens

Whether this task supports decontamination against model training set.