Checks whether CP is the #codepoint of a CJK character.

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} ALBERT has a different architect

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTraine

Partition a sequence of items randomly into non-empty segments. Args: num_items: an integer scalar > 0 num

Reads a tab separated value file.

Reads a tab separated value file.

Reads a tab separated value file.

Initialize one model and one optimizer with an optional learning rate scheduler. This calls :func:`deepspeed.initialize` internally.

Take as input a string and return a list of strings (tokens) for words/sub-words

Returns a tokenized string.

Generate with fixed prompt of abduction

Add token-type embedding. This function is provided so we can add token-type embeddings in case the pretrained model does not have it.

Broadcast data from rank zero of each model parallel group to the members of the same model parallel group. Arguments: keys: list of

Build model inputs from a sequence or a pair of sequences for sequence classification tasks by concatenating and adding special token

Build model inputs from a sequence or a pair of sequence for sequence classification tasks by concatenating and adding special tokens

Compute the entire variational lower-bound, measured in bits-per-dim, as well as other related quantities. :param model: the

Compile helper function ar runtime. Make sure this is invoked on a single process.