Whether this task supports decontamination against model training set.

Instantiate and evaluate a model on a list of tasks. :param model: Union[str, LM] Name of model or LM object, see lm_eval.models.get_mode

Instantiate and evaluate a model on a list of tasks. :param model: Union[str, LM] Name of model or LM object, see lm_eval.models.get_mode

Parses something like args1=val1,arg2=val2 Into a dictionary

Translation Error Rate is an error metric for machine translation that measures the number of edits required to change a system output into one

Translation Error Rate is an error metric for machine translation that measures the number of edits required to change a system output into one

If `per_node_beam_size` (which defaults to `beam_size`) is larger than the size of the target vocabulary, `BeamSearch.search` should