MCPcopy Create free account

hub / github.com/allenai/OLMo / functions

Functions3,821 in github.com/allenai/OLMo

⨍ Functions 3,821◇ Types & classes 799↳ Endpoints 9
↓ 3 callersMethod_is_dir
(self, bucket_name: str, key: str)
scripts/storage_cleaner.py:359
↓ 3 callersMethod_is_dir
(self, bucket_name: str, key: str)
scripts/storage_cleaner.py:552
↓ 3 callersMethod_patch_sharded_tensor_load
Monkeypatch for torch's ShardedTensor, so we can unpickle without having torch.distributed set up.
olmo/checkpoint.py:1225
↓ 3 callersFunction_sacreformat
Format refs and preds for sacrebleu corpus calculation. It is very particular
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/metrics.py:161
↓ 3 callersFunction_sacreformat
Format refs and preds for sacrebleu corpus calculation. It is very particular
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/metrics.py:161
↓ 3 callersMethod_save_checkpoint
( self, checkpointer: Checkpointer, checkpoint_type: CheckpointType )
olmo/train.py:435
↓ 3 callersFunction_unarchive_if_archive
(dir_or_archive: str, storage: StorageAdapter)
scripts/storage_cleaner.py:692
↓ 3 callersFunction_wait_before_retry
(attempt: int)
olmo/util.py:578
↓ 3 callersFunctionactivation_checkpoint_function
(cfg: ModelConfig)
olmo/model.py:79
↓ 3 callersFunctionadd_cached_path_clients
()
olmo/util.py:843
↓ 3 callersMethodapply
( self, state: ConstraintStateType, class_log_probabilities: torch.Tensor, )
olmo/beam_search.py:543
↓ 3 callersMethodbase_dir
(self, base_dir: str)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/tasks/efficiency_benchmark.py:47
↓ 3 callersMethodbase_dir
(self, base_dir: str)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/tasks/efficiency_benchmark.py:47
↓ 3 callersFunctionbuild_scheduler
(cfg: TrainConfig, sched_cfg: Optional[SchedulerConfig] = None)
olmo/optim.py:968
↓ 3 callersMethodclose
(self, exit_code: int = 0)
olmo/train.py:1366
↓ 3 callersMethodconvert_choice
(choice)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/tasks/superglue.py:221
↓ 3 callersMethodconvert_choice
(choice)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/tasks/superglue.py:221
↓ 3 callersFunctioncreate_raw_dataset
(dataset: Dataset, id_key: str, prompt_key: str, answer_key: str, save_to: str)
scripts/add_code_eval.py:10
↓ 3 callersMethodcreate_storage_adapter
(cls, storage_type: StorageType)
scripts/storage_cleaner.py:98
↓ 3 callersMethodcreate_temp_dir
(self, directory: Optional[str] = None, suffix: Optional[str] = None)
scripts/storage_cleaner.py:148
↓ 3 callersMethoddelete_path
Deletes the entry at the given path and, if the path is a directory, delete all entries within its subdirectories.
scripts/storage_cleaner.py:70
↓ 3 callersMethoddetokenize
(self, text)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/tasks/mutual.py:70
↓ 3 callersMethoddetokenize
(self, text)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/tasks/mutual.py:70
↓ 3 callersMethoddevice
(self)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/models/gpt2.py:97
↓ 3 callersMethoddevice
(self)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/models/gpt2.py:97
↓ 3 callersFunctionfilter
(example)
scripts/prepare_tulu_data.py:71
↓ 3 callersMethodfind_params
(self, x, weight=False)
inference/compression/dependencies/AutoGPTQ/auto_gptq/quantization/quantizer.py:36
↓ 3 callersMethodformat_answer
(answer, label)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/tasks/superglue.py:254
↓ 3 callersMethodformat_answer
(answer, label)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/tasks/superglue.py:254
↓ 3 callersMethodfrom_checkpoint
Load an OLMo model from a checkpoint.
olmo/model.py:1725
↓ 3 callersFunctiongen_prompt
(train_df, subject, k=-1)
inference/eval/mmlu/run_eval.py:38
↓ 3 callersMethodget_answer_option
(cls, problem)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/tasks/race.py:95
↓ 3 callersMethodget_answer_option
(cls, problem)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/tasks/race.py:95
↓ 3 callersFunctionget_bytes_range
(source: PathOrStr, bytes_start: int, num_bytes: int)
olmo/util.py:368
↓ 3 callersFunctionget_gptq_peft_model
( model: BaseGPTQForCausalLM, peft_config: PeftConfig = None, model_id: str = None, adapter_na
inference/compression/dependencies/AutoGPTQ/auto_gptq/utils/peft_utils.py:350
↓ 3 callersMethodget_instances
(self, split: str, num_instances: Optional[int] = None)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/tasks/efficiency_benchmark.py:67
↓ 3 callersMethodget_instances
(self, split: str, num_instances: Optional[int] = None)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/tasks/efficiency_benchmark.py:67
↓ 3 callersFunctionget_local_world_size
()
olmo/torch_util.py:42
↓ 3 callersFunctionget_module_by_name_prefix
(model, module_name: str)
inference/compression/dependencies/AutoGPTQ/auto_gptq/modeling/_utils.py:40
↓ 3 callersMethodhas_training_docs
Whether the task has a training set
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/base.py:401
↓ 3 callersMethodhas_training_docs
Whether the task has a training set
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/base.py:401
↓ 3 callersFunctionhash_args
(attr, args)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/base.py:688
↓ 3 callersFunctionhash_args
(attr, args)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/base.py:688
↓ 3 callersMethodinject_to_model
(cls, *args, **kwargs)
inference/compression/dependencies/AutoGPTQ/auto_gptq/nn_modules/_fused_base.py:14
↓ 3 callersMethodinsert
(self, i: int, value: Any)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/tango_utils/sequences.py:347
↓ 3 callersMethodinstance_as_eleuther_doc
(self, instance: Dict[str, Any])
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/tasks/eleuther.py:94
↓ 3 callersMethodinstance_as_eleuther_doc
(self, instance: Dict[str, Any])
inference/efficiency/dependencies/previous_version/efficiency_benchmark/tasks/eleuther.py:94
↓ 3 callersMethodlist_dirs
Lists all the directories within the given directory. Returns only top-level entries (i.e. not entries in subdirectories).
scripts/storage_cleaner.py:64
↓ 3 callersFunctionload_fsdp_optim_state
(fsdp_model: FSDP, optim: Optimizer, optim_state: Dict[str, Any])
olmo/checkpoint.py:218
↓ 3 callersMethodlog_metrics_to_console
(self, prefix: str, metrics: Dict[str, float])
olmo/train.py:952
↓ 3 callersMethodmerge
(self)
inference/compression/dependencies/AutoGPTQ/auto_gptq/utils/peft_utils.py:211
↓ 3 callersMethodnormalize_string
(self, s)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/decontamination/janitor.py:199
↓ 3 callersMethodnormalize_string
(self, s)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/decontamination/janitor.py:199
↓ 3 callersFunctionpad_block
(block, pads)
inference/compression/dependencies/AutoGPTQ/auto_gptq/utils/data_utils.py:146
↓ 3 callersFunctionpeak_gpu_memory
Get the peak GPU memory usage in MB across all ranks. Only rank 0 will get the final result.
olmo/torch_util.py:104
↓ 3 callersFunctionquant_matmul_248
(input, qweight, scales, qzeros, g_idx, bits, maxq)
inference/compression/dependencies/AutoGPTQ/auto_gptq/nn_modules/triton_utils/kernels.py:286
↓ 3 callersMethodreset_parameters
(self)
olmo/model.py:701
↓ 3 callersMethodreset_parameters
(self)
olmo/model.py:859
↓ 3 callersMethodrestore_checkpoint
( self, load_path: PathOrStr, *, checkpoint_type: Optional[CheckpointType] = N
olmo/train.py:615
↓ 3 callersMethodscore
Score the final predictions found by beam search. Returns a tensor of the final sequence scores of shape `(batch_size, beam_size)`.
olmo/beam_search.py:435
↓ 3 callersMethodset_activation_checkpointing
( self, strategy: Optional[ActivationCheckpointingStrategy], checkpoint_func: Optional[Callable] = Non
olmo/model.py:1148
↓ 3 callersMethodstart
(self)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/stdio_wrapper.py:140
↓ 3 callersFunctionstate_dict_to_safetensors_file
(state_dict: Dict, filename: PathOrStr)
olmo/safetensors_util.py:70
↓ 3 callersMethodtest_docs
:return: Iterable[obj] A iterable of any object, that doc_to_text can handle
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/base.py:429
↓ 3 callersMethodtest_docs
:return: Iterable[obj] A iterable of any object, that doc_to_text can handle
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/base.py:429
↓ 3 callersMethodtie_weights
This function is intentionally left as a no-op. Weight tying is handled as follows: - When the model is initialized, the `ff
hf_olmo/modeling_olmo.py:189
↓ 3 callersMethodtraining_docs
:return: Iterable[obj] A iterable of any object, that doc_to_text can handle
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/base.py:415
↓ 3 callersMethodtraining_docs
:return: Iterable[obj] A iterable of any object, that doc_to_text can handle
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/base.py:415
↓ 3 callersMethodunshard_checkpoint
Unshard a checkpoint. Note this is not marked abstract because child classes are not required to implemented this.
olmo/checkpoint.py:533
↓ 3 callersMethodupdate_state
( self, state: ConstraintStateType, last_prediction: torch.Tensor, last_backpo
olmo/beam_search.py:574
↓ 3 callersMethodvalidation_docs
:return: Iterable[obj] A iterable of any object, that doc_to_text can handle
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/base.py:422
↓ 3 callersMethodvalidation_docs
:return: Iterable[obj] A iterable of any object, that doc_to_text can handle
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/base.py:422
↓ 2 callersMethod__init__
( self, dataset_path: str, dataset_name: Optional[str] = None, *, version_override: Optional[str] = No
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/tasks/efficiency_benchmark.py:36
↓ 2 callersMethod__init__
( self, dataset_path: str, dataset_name: Optional[str] = None, *, version_override: Optional[str] = No
inference/efficiency/dependencies/previous_version/efficiency_benchmark/tasks/efficiency_benchmark.py:36
↓ 2 callersMethod__init__
( self, adapter_name: str, linear_module: torch.nn.Linear, r: int = 0,
inference/compression/dependencies/AutoGPTQ/auto_gptq/utils/peft_utils.py:184
↓ 2 callersMethod__init__
( self, params, lr: float = 1e-4, betas: Tuple[float, float] = (0.9, 0.99),
olmo/optim.py:370
↓ 2 callersMethod__init__
( self, end_index: int, *, max_steps: int = 50, beam_size: int = 10,
olmo/beam_search.py:688
↓ 2 callersMethod__normalize_option
(self, doc, option)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/tasks/wsc273.py:75
↓ 2 callersMethod__normalize_option
(self, doc, option)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/tasks/wsc273.py:75
↓ 2 callersMethod_answer_to_bags
(self, answer)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/tasks/drop.py:175
↓ 2 callersMethod_answer_to_bags
(self, answer)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/tasks/drop.py:175
↓ 2 callersFunction_apply_scheduler
Apply scheduler according to OLMo style.
tests/grad_norm_test.py:118
↓ 2 callersMethod_build_and_save_global_indices
(self)
olmo/data/iterable_dataset.py:74
↓ 2 callersMethod_build_global_indices
(self)
olmo/data/iterable_dataset.py:90
↓ 2 callersMethod_ensure_object_info
(self)
olmo/util.py:876
↓ 2 callersMethod_exhaust_and_yield_stdout
Read everything from the stdout pipe. This function uses stdout.readline() to read one prediction at a time. stdout.readline(
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/stdio_wrapper.py:23
↓ 2 callersMethod_exhaust_and_yield_stdout
Read everything from the stdout pipe. This function uses stdout.readline() to read one prediction at a time. stdout.readline(
inference/efficiency/dependencies/previous_version/efficiency_benchmark/stdio_wrapper.py:23
↓ 2 callersMethod_extract_answer
(self, completion)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/tasks/gsm8k.py:85
↓ 2 callersMethod_extract_answer
(self, completion)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/tasks/gsm8k.py:85
↓ 2 callersMethod_format_answers
(self, answers)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/dependencies/lm_eval/tasks/truthfulqa.py:171
↓ 2 callersMethod_format_answers
(self, answers)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/dependencies/lm_eval/tasks/truthfulqa.py:171
↓ 2 callersMethod_fsdp_handles
(self, fsdp_model: FSDP)
olmo/checkpoint.py:1513
↓ 2 callersMethod_fsdp_modules
Returns a list of FSDP modules with their FQN.
olmo/checkpoint.py:1494
↓ 2 callersMethod_gather_state_dict_paths
( self, load_path: PathOrStr, state_dict_type: str, world_size: int, *
olmo/checkpoint.py:1877
↓ 2 callersMethod_get_blob_size
(self, blob: gcs.Blob)
scripts/storage_cleaner.py:260
↓ 2 callersMethod_get_dataset
(self, num_shots: int, fewshot_seed: int, split: str)
inference/efficiency/dependencies/efficiency-pentathlon/efficiency_benchmark/tasks/metaicl.py:28
↓ 2 callersMethod_get_dataset
(self, num_shots: int, fewshot_seed: int, split: str)
inference/efficiency/dependencies/previous_version/efficiency_benchmark/tasks/metaicl.py:28
↓ 2 callersMethod_get_dataset_item
(self, idx: int)
olmo/data/iterable_dataset.py:183
↓ 2 callersMethod_get_file_size
(self, path: Path)
scripts/storage_cleaner.py:192
↓ 2 callersFunction_get_module_names
(checkpoint_traces_folder: Path)
scripts/compare_module_outputs.py:39
← previousnext →201–300 of 3,821, ranked by callers