Load processed image IDs and return missing metrics for each key

Load processed image IDs and return missing metrics for each key

Load processed image IDs and return missing metrics for each key

Load processed image IDs and return missing metrics for each key

Load processed image IDs and return missing metrics for each key

Normalize the extracted answer to match the answer type

Normalize a score to a 100-point scale.

Parse the prediction from the generated response. Return a list of predicted strings or numbers.

nested_split_lens: A list of N lists of ints. Each int indicates the length of a split within a sample, where each sample contains multi

Process images from the dataset using the editing model.

Give position of A relative to B, factoring in object dimensions

Give position of A relative to B, factoring in object dimensions

Calculates relaxed correctness. The correctness tolerates certain error ratio defined by max_relative_change. See https://arxiv.org/pdf/2203.

Main evaluation process with multi-threading

Main evaluation process with multi-threading

Main evaluation process with multi-threading

Mirror common.run_evaluation: load existing metrics, dispatch threads, save results.

Main evaluation process with multi-threading

Check if the prediction is equal to the answer, even if they are of different types

Save evaluation result to JSONL file with thread lock

Save evaluation result to JSONL file with thread lock

Save evaluation result to JSONL file with thread lock

Save evaluation result to JSONL file with thread lock

Save evaluation result to JSONL file with thread lock

Args: Helper function for reproducible behavior to set the seed in `random`, `numpy`, `torch`. seed (`int`): The seed to set.

Set random seeds for reproducibility.

初始化分布式环境

Set up and load all required models. Args: llm_path: Path to the language model vit_path: Path to the vision model

Calculate normalized benchmark scores for each category and subitem. Only non-None scores are included in results.

Initialize Taylor cache and allocate storage for different-order derivatives in the Taylor cache. :param cache_dic: Cache dictionary

Compute Taylor expansion error. :param cache_dic: Cache dictionary :param current: Information of the current step

Create sinusoidal timestep embeddings. :param t: a 1-D Tensor of N indices, one per batch element. These ma

Fills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :

Main method to prepare for the model one or several sequences(s) and image(s). This method forwards the `text` and `kwargs` arguments