Initialize model data parallel groups. Arguments: model_parallel_size: number of GPUs used to parallelize model. Let's say we h

判断是否是字符串

Useful for code segments that may be accessed with or without mpu initialization

Load tf checkpoints in a pytorch model.

Load tf checkpoints in a pytorch model.

Load tf checkpoints in a pytorch model.

Load tf checkpoints in a pytorch model

Numerically stable implementation of the operation value.exp().sum(dim, keepdim).log()

Numerically stable implementation of the operation value.exp().sum(dim, keepdim).log()

The same as logkv(), but if called many times, values averaged.

Log a dictionary of key-value pairs

Check if model and data parallel groups are initialized.

Just EOS

Sample x_{t-1} from the model at the given timestep. :param model: the model to sample from. :param x: the current tensor at

Generate samples from the model. :param model: the model module. :param shape: the shape of the samples, (N, C, H, W).

Sample x_{t-1} from the model at the given timestep. :param model: the model to sample from. :param x: the current tensor at

Pad sequences and convert them to numpy.

LM logits using word embedding weights.

returns the datatype on the basis of configured precision

Generate samples from the model using Pseudo Linear Multistep. Same usage as p_sample_loop().