" A mapping config to object (model or tokenizer for instance) that will load keys and values when it is accessed. Args: - config_m

A residual block that can optionally change the number of channels. :param channels: the number of input channels. :param emb_channels:

Optimized dropout module for stabilizing the training Args: drop_prob (float): the dropout probabilities

A sequential module that passes timestep embeddings to the children that support it as an extra input.

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added betwee

fused operation: scaling + mask + softmax Arguments: input_in_fp16: flag to indicate if input in fp16 data format. input_in_b

ZEN model ("BERT-based Chinese (Z) text encoder Enhanced by N-gram representations"). Params: `config`: a BertConfig class instance with

An attention block that allows spatial positions to attend to each other. Originally ported from here, but adapted to the N-d case. http

This class copied code from :class:`~transformers.RobertaModel` and overwrote standard self-attention with longformer self-attention to provi

Base class for masked language models outputs. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` i

Base class for outputs of token classification models.

ZEN model ("BERT-based Chinese (Z) text encoder Enhanced by N-gram representations"). Params: `config`: a BertConfig class instance with

A downsampling layer with an optional convolution. :param channels: channels in the inputs and outputs. :param use_conv: a bool determin

Dataset Used for yuyuan medical qa task. Just surpport small datasets, when deal with large datasets it may be slowly. for large datasets

Dataset Used for T5 unsuprvise pretrain. load_data_type = 0: load raw data from data path and save tokenized data, call function load_data

DeepVAE with recursive latent z extracted from every layer of encoder and applied on every layer of decoder

Multi-headed attention from 'Attention Is All You Need' paper

Multi-class Focal loss implementation

A single training/test example for simple sequence classification.

A single training/test example for simple sequence classification.

Dataset Used for LCSTS summary task.

Construct the embeddings from word, position and token_type embeddings.

An upsampling layer with an optional convolution. :param channels: channels in the inputs and outputs. :param use_conv: a bool determini

Construct the embeddings from word, position and token_type embeddings.

Transformer decoder consisting of *config.decoder_layers* layers. Each layer is a [`DeltalmDecoderLayer`] Args: config: DeltalmConfig

This module learns positional embeddings up to a fixed maximum size.

Language model embeddings. Arguments: hidden_size: hidden size vocab_size: vocabulary size max_sequence_length: maximum si

r""" **labels**: (`optional`) ``torch.LongTensor`` of shape ``(batch_size, sequence_length)``: Labels for language modeling.

Loader for IndexedDataset

r""" This is the configuration class to store the configuration of a [`LlamaModel`]. It is used to instantiate an LLama model according to the

A Parallel Linear Layer transforming the transformer outputs from hidden_size -> vocab_size

A module which performs QKV attention and splits in a different order.

Base class for outputs of token classification models.