Instantiate a PreTrainedBertModel from a pre-trained model file. Download and cache the pre-trained model file if needed.

Extract values from a 1-D numpy array for a batch of indices. :param arr: the 1-D numpy array. :param timesteps: a tensor of indices int

Converts a sequence of tokens into ids using the vocab.

Args: pl_model： lightning module model_params: 需要优化的模型参数

Find the most likely tag sequence using Viterbi algorithm. Args: emissions (`~torch.Tensor`): Emission score tensor of size

This method forwards all its arguments to BertTokenizerFast's [`~PreTrainedTokenizer.batch_decode`]. Please refer to the docstring of

If distributed is initialized, print only on rank 0.

Prune a linear layer (a model parameters) to keep only entries in index. Return the pruned layer as a new layer with requires_grad=True.

Loads the schedulers state. When saving or loading the scheduler, please make sure to also save or load the state of the optimizer.

:param seq: :param id2label: :param markup: :return:

Basic Tokenization of a piece of text. Split on "white spaces" only, for sub-word tokenization, see WordPieceTokenizer. Args:

Create a 1D, 2D, or 3D convolution module.

Returns the state of the scheduler as a :class:`dict`. It contains an entry for every variable in self.__dict__ which is not the opti

Return world size for the model parallel group.

Get the model parallel group the caller rank belongs to.

Converts a single index or a sequence of indices in a token or a sequence of tokens, using the vocabulary and added tokens. A

Return my rank for the model parallel group.

Make a standard normalization layer. :param channels: number of input channels. :return: an nn.Module for normalization.

Apply the model to get p(x_{t-1} | x_t), as well as a prediction of the initial x, x_0. :param model: the model, which takes

Converts a sequence of ids in wordpiece tokens using the vocab.

Get the fp32 allreduce flag

Write the sequence of args, with no separators, to the console and output files (if you've configured an output file).

Take the mean over all non-batch dimensions.

Importance-sample timesteps for a batch. :param batch_size: the number of timesteps. :param device: the torch device to save

Checks whether CP is the codepoint of a CJK character.

Converts a sequence of tokens (string) in a single string.

Ensure that numerator is divisible by the denominator and return the division value.

r""" Returns: image_features (`torch.FloatTensor` of shape `(batch_size, output_dim`): The image embeddings obtained by

Create a linear module.

Compute the mean and variance of the diffusion posterior: q(x_{t-1} | x_t, x_0)

Diffuse the data for a given number of diffusion steps. In other words, sample from q(x_t | x_0). :param x_start: the initi