Parameters ---------- prediction : numpy.ndarray a predicted dependency tree where prediction[dep_idx] = head_id

A sentence boundary detector using ngram as features and LSTM as encoder (:cite:`Schweter:Ahmed:2019`). It predicts whether a punctuation mark

A linear layer with an optional CRF (:cite:`lafferty2001conditional`) layer on top of it. Args: hidden_size: Size of hidden state

A contextual word embedding builder which builds a :class:`~hanlp.layers.embeddings.contextual_word_embedding.ContextualWordEmbeddingModule` a

A `CnnEncoder` is a combination of multiple convolution layers and max pooling layers. The input to this module is of shape `(batch_size, num_

Character level RNN embedding module. Args: field: The field in samples this encoder will work on. vocab_size: The si

Base class for embedding builders.

CoNLL (:cite:`buchholz-marsi-2006-conll`) format template, see http://anthology.aclweb.org/W/W06/W06-2920.pdf Args: id (int):

r""" A dict-like structure for fast custom dictionary strategies in tokenization and tagging. It is built with a dict of key-value pai

A node in a trie tree. Args: value: The value associated with this node.

The data might include 2 annotated spans which are identical, but have different ids. This checks all clusters for spans which are identi

Save arguments to config Args: locals_: Dict: exclude: (Default value = ('trn_data') 'dev_data': '

Save arguments to config Parameters ---------- config `locals()` exclude

Data dependent initialization. Args: inputs: Returns:

Whether to use L2 weight decay for `param_name`. Args: param_name: Returns:

Return next chunk of data that we would from the file pointer. Args: fp: file previously_read_position: file pointer position that we

Return information on which file pointer position to read from and how many bytes. Args: fp: past_read_positon: int chunk_size

Set the weight g with the norm of the weight vector.

Initialize weight g. The initial value of g could either from the initial value in v, or by the input value if self.data_init

Return True when b is part of a new line separator found at index >= 1, False otherwise. Args: b: bytestring Returns: bool

Args: vd: vocabs: vocab_cls: Default class for the new vocab

Copied from https://github.com/hiroki13/span-based-srl/blob/2c8b677c4e00b6c607e09ef4f9fe3d54961e4f2e/src/utils/sent.py#L42 Args: pr

Make causal mask used for bi-directional self-attention.

For a given coref label, add it to a currently open span(s), complete a span(s) or ignore it, if it is outside of all spans. This met

Given a sequence of different label types for a single word and the current span label we are inside, compute the BIO tag for each la