verify megatron args are correctly consumed after sending via deepspeed

tests that there are no duplicates among parent classes of NeoXArgs

verify assertion error if required arguments are not provided

verify assertion error if required arguments are not provided

verify 13B.yml can be loaded without raising validation errors

verify 13B.yml can be loaded without raising validation errors

verify 2-7B.yml can be loaded without raising validation errors

verify 6-7B.yml can be loaded without raising validation errors

verify XL.yml can be loaded without raising validation errors

verify large.yml can be loaded without raising validation errors

verify medium.yml can be loaded without raising validation errors

verify small.yml can be loaded without raising validation errors

verify small.yml can be loaded together with text generation without raising validation errors

" checks for code pieces of the pattern "args.*" and verifies that such used arg is defined in NeoXArgs

tokenizer type to use when tokenizing data

Sets up the model for training by turning off k/v caching and setting `parallel output` of the final layer to True, so logits are not

URLs from which to download dataset

Dictionary from vocab text token to id token.

Vocab file for tokenizer

Helper function for the cross entropy.

Builds weights from a multinomial distribution over groups of data according to the number of samples in each group. We sample from a gr

Easy accessory for the pipeline engine to tie embeddings across stages.

Write timers to a tensorboard writer