Given a `doc`, flatten it out so that each JSON blob contains exactly one question and one answer. Logic taken from the reference impl

Given a `doc`, flatten it out so that each JSON blob contains exactly one question and one answer. Logic taken from the reference impl

:returns: {str: [metric_score] -> float} A dictionary where keys are the names of submetrics and values are functions

:returns: {str: [metric_score] -> float} A dictionary where keys are the names of submetrics and values are functions

From catwalk. https://github.com/allenai/catwalk/blob/main/catwalk/model.py

Clears the entire sequence

Clips gradients for every group that has the field `max_grad_norm`. At the same time collect metrics for each parameter and its gradi

Uses RequestFactory to construct Requests and returns an iterable of Requests which will be sent to the LM. :param doc: T

Uses RequestFactory to construct Requests and returns an iterable of Requests which will be sent to the LM. :param doc: T

Dispatches requests to OpenAI chat completion API asynchronously. Args: messages_list: List of messages to be sent to OpenAI chat complet

Dispatches requests to OpenAI text completion API asynchronously. Args: prompt_list: List of prompts to be sent to OpenAI text completion

Downloads the content from the directory path to the local FS destination folder.

Modify ``x`` in place to replace ``float("-inf")`` with the minimum value of the dtype when ``check_neg_inf`` is ``True`` and to replace ``fl

Returns a fewshot context string that is made up of a prepended description (if provided), the `num_fewshot` number of examples, and an append

Returns a fewshot context string that is made up of a prepended description (if provided), the `num_fewshot` number of examples, and an append

Function to find the end position of the first consecutive sequence of True in an array.

Generate token IDs using beam search. Note that by default ``beam_size`` is set to 1, which is greedy decoding. :param inpu

Sample `Gumbel(phi)` conditioned on the maximum value being equal to `T`. `phi` should have shape `(batch_size, num_classes)` and `T

Lists all paths in an S3 bucket with given prefix and extension, along with their sizes. Args: bucket_name (str): Name of the S3 buc

Lists all paths in an S3 bucket with given prefix and extension, along with their sizes. Args: bucket_name (str): Name of the S3 buc

Loads a HuggingFace dataset. The dataset is assumed to be saved using `save_hf_dataset_to_disk` and located in `olmo_data/hf_datasets`.

Loads an oe-eval request file from `olmo_data/oe_eval_tasks`. TODO: Add support from loading from S3 instead?

Compute log-likelihood of generating a continuation from a context. Downstream tasks should attempt to use loglikelihood instead of other

Compute log-likelihood of generating a continuation from a context. Downstream tasks should attempt to use loglikelihood instead of other