The base class for transorfmer based components. If offers methods to build transformer tokenizers , optimizers and models. Args:

Vocabulary base class which converts tokens to indices and vice versa. Args: idx_to_token: id to token mapping. token

Return -1 if read_buffer does not contain new line otherwise the position of the rightmost newline. Args: read_buffer: bytestring Retu

Retrieves the learning rate with the given state. Args: var_device: var_dtype: apply_state: Returns:

Remove a single instance of new line at the end of l if it exists. Args: l: Returns: : bytestring

类似 -3 -2 -1 0 1 2 -3 -2 -1 0 1 2 -3 -2 -1 0 1 2 转换为 0 1 2 -1 0 1 -2 -1 0 Args:

Abstract Meaning Representation (AMR) captures “who is doing what to whom” in a sentence. Each sentence is represented as a rooted, directed, acyclic

Adjust tokens for BERT See https://github.com/DoodleJZ/HPSG-Neural-Parser/blob/master/src_joint/Zparser.py#L1204 Args: sentence:

Given a sequence corresponding to BIO tags, extracts spans. Spans are inclusive and can be of zero length, representing a single word span.

Implement this method to build criterion (loss function). Args: **kwargs: The subclass decides the method signature.

Build a :class:`logging.Logger`. Args: name: The name of this logger. save_dir: The directory this logger should save

Implement this to build metric(s). Args: **kwargs: The subclass decides the method signature.

Build model. Args: training: ``True`` if called during training. **kwargs: ``**self.config``.

Build samples for this task. Called when this task is the first task. Default behaviour is to take inputs as list of tokens and put t

Build word2vec embedding and a vocab. Args: embed: vocab: The vocabulary from training set. extend_vocab: Unlock vocabula

r""" ChuLiu/Edmonds algorithm for non-projective decoding. Some code is borrowed from `tdozat's implementation`_. Descriptions of notatio

Closes all opened it s file handler.