Class TrainingArguments

src/transformers/training_args.py:179–2848

Configuration class for controlling all aspects of model training with the Trainer. TrainingArguments centralizes all hyperparameters, optimization settings, logging preferences, and infrastructure choices needed for training. [`HfArgumentParser`] can turn this class into [argparse

@dataclass
class TrainingArguments:
    """
    Configuration class for controlling all aspects of model training with the Trainer.
    TrainingArguments centralizes all hyperparameters, optimization settings, logging preferences, and infrastructure choices needed for training.

    [`HfArgumentParser`] can turn this class into
    [argparse](https://docs.python.org/3/library/argparse#module-argparse) arguments that can be specified on the
    command line.

    Parameters:
        output_dir (`str`, *optional*, defaults to `"trainer_output"`):
            The output directory where the model predictions and checkpoints will be written.

        > Training Duration and Batch Size

        per_device_train_batch_size (`int`, *optional*, defaults to 8):
            The batch size *per device*. The **global batch size** is computed as:
            `per_device_train_batch_size * number_of_devices` in multi-GPU or distributed setups.
        num_train_epochs(`float`, *optional*, defaults to 3.0):
            Total number of training epochs to perform (if not an integer, will perform the decimal part percents of
            the last epoch before stopping training).
        max_steps (`int`, *optional*, defaults to -1):
            Overrides `num_train_epochs`. If set to a positive number, the total number of training steps to perform.
            For a finite dataset, training is reiterated through the dataset (if all data is exhausted) until
            `max_steps` is reached.

        > Learning Rate & Scheduler

        learning_rate (`float`, *optional*, defaults to 5e-5):
            The initial learning rate for the optimizer. This is typically the peak learning rate when using a scheduler with warmup.
        lr_scheduler_type (`str` or [`SchedulerType`], *optional*, defaults to `"linear"`):
            The learning rate scheduler type to use. See [`SchedulerType`] for all possible values. Common choices:
                - "linear" = [`get_linear_schedule_with_warmup`]
                - "cosine" = [`get_cosine_schedule_with_warmup`]
                - "constant" =  [`get_constant_schedule`]
                - "constant_with_warmup" = [`get_constant_schedule_with_warmup`]
        lr_scheduler_kwargs (`dict` or `str`, *optional*, defaults to `None`):
            The extra arguments for the lr_scheduler. See the documentation of each scheduler for possible values.
        warmup_steps (`int` or `float`, *optional*, defaults to 0):
            Number of steps for a linear warmup from 0 to `learning_rate`. Warmup helps stabilize training in the initial phase. Can be:
                - An integer: exact number of warmup steps
                - A float in range [0, 1): interpreted as ratio of total training steps

        > Optimizer

        optim (`str` or [`training_args.OptimizerNames`], *optional*, defaults to `"adamw_torch"` (for torch>=2.8 `"adamw_torch_fused"`)):
            The optimizer to use. Common options:
                - `"adamw_torch"`: PyTorch's AdamW (recommended default)
                - `"adamw_torch_fused"`: Fused AdamW kernel
                - `"adamw_hf"`: HuggingFace's AdamW implementation
                - `"sgd"`: Stochastic Gradient Descent with momentum
                - `"adafactor"`: Memory-efficient optimizer for large models
                - `"adamw_8bit"`: 8-bit AdamW (requires bitsandbytes)
                See [`OptimizerNames`] for the complete list.
        optim_args (`str`, *optional*):
            Optional arguments that are supplied to optimizers such as AnyPrecisionAdamW, AdEMAMix, and GaLore.
        weight_decay (`float`, *optional*, defaults to 0):
            Weight decay coefficient applied by the optimizer (not the loss function). Adds L2