A task that loads data in the MetaICL fewshot setting. This uses the same set of ICL demonstrations for all test instances.

A task that loads data in the MetaICL fewshot setting. This uses the same set of ICL demonstrations for all test instances.

An error with a configuration file.

OLMo (model) configuration.

Implements the beam search algorithm for decoding the most likely sequences. :param end_index: The index of the "stop" or "end" token in the

BuilderConfig for GPT3 Arithmetic Test Dataset.

BuilderConfig for GPT3 Arithmetic Test Dataset.

Cache for attention biases and other things that would normally be stored as buffers. We avoid using buffers because we've run into various i

Adapted from PyTorch's DistributedSampler, this wraps a Dataset or arbitrary sequence as an IterableDataset that can be deterministically res

Produces a sequence that applies a function to every element of another sequence. This is similar to Python's :func:`map`, but it returns a

Produces a sequence that applies a function to every element of another sequence. This is similar to Python's :func:`map`, but it returns a

A PyTorch :class:`~torch.utils.data.Dataset` backed by one or more numpy memory-mapped arrays of token IDs. Token IDs are chunked together in

BuilderConfig for Hendrycks ETHICS.

BuilderConfig for Hendrycks ETHICS.

A :class:`Checkpointer` that saves a single full model and optimizer state dictionary.

A :class:`FinalSequenceScorer` which scores the sequences by the average log probability of the tokens in the sequence. It optionally include

A base class for transformer block implementations.

An error from incorrect environment variables.

This is a transformer block where the output is computed as ``MLP(LN(x + Attention(LN(x))))`` (plus another skip connection). This block is s

An error with a network request.

This is a typical transformer block where the output is computed as ``MLP(LN(x + Attention(LN(x))))`` (plus another skip connection). To comp

A `Sampler` which redistributes the probability mass function for nodes among the top choices with a cumulative probability of at least `p`,

OLMo training configuration.

BuilderConfig for GLUE.

BuilderConfig for GLUE.

An error from incorrect CLI usage.

A :class:`~torch.distributed.checkpoint.StorageReader` based on :class:`~torch.distributed.checkpoint.FileSystemReader` that can read data di

A `Sampler` which redistributes the probability mass function for nodes among the top `k` choices, then samples from that subset after re-nor

A `Sampler` which uses the Gumbel-Top-K trick to sample without replacement. See [*Stochastic Beams and Where to Find Them: The Gumbel-Top-k

The default :class:`LayerNorm` implementation which can optionally run in low precision.

A sharded :class:`Checkpointer` that directly saves the local FSDP flat params data. The optimizer state is saved directly with `torch.save()

An error occurred reading or writing from a checkpoint.

Thread class with a stop() method. The thread itself has to check regularly for the stopped() condition.

Thread class with a stop() method. The thread itself has to check regularly for the stopped() condition.

Produces a sequence that's a slice into another sequence, without copying the elements. This assumes that the inner sequence never changes.

Produces a sequence that's a slice into another sequence, without copying the elements. This assumes that the inner sequence never changes.

Pointwise product of cosine schedule and linear decay; useful during annealing.

A `Sampler` that just deterministically returns the `k` nodes or beams with highest log probability.

This task is different because EAI relabels the datasets.

This task is different because EAI relabels the datasets.