Remove empty categories for all trees in this file and save them into a "noempty" file. Args: path: File path.

Save this component to a directory. Args: save_dir: The directory to save this component. **kwargs: Not used.

Save to path Args: path: Returns:

Save vocabularies to a directory. Args: save_dir: The directory to save vocabularies. filename: The name for vocabul

r"""Scale down the ``batch_size`` and ``batch_max_tokens`` to :math:`\frac{1}{\text{gradient_accumulation}}` of them respectively. Ar

Modified from https://github.com/tdozat/Parser-v3/blob/2ff4061373e8aac8c962537a6220e1d5b196abf6/scripts/semdep_eval.py Dozat claimed "I tested it

For a given input `sentence`, return a list of its tokens. Split on Unicode spaces ``\\s+`` (i.e., any kind of **Unicode** space character).

split chunk tag into IOBES prefix and chunk_type e.g. B-PER -> (B, PER) O -> (O, None) Args: chunk_tag: Returns:

Create a subset given indices of samples. Args: indices: Indices of samples. Returns: TransformableDataset:

r""" Tarjan algorithm for finding Strongly Connected Components (SCCs) of a graph. Args: sequence (list): List of head in

Text classification is the task of assigning a sentence or document an appropriate category. The categories depend on the chosen dataset and can range

Convert to a pretty text representation which can be printed to visualize linguistic structures. Args: tok: Token key.

Apply transforms to a sample. Args: sample: A sample, which is a ``dict`` holding features. inplace: ``True`` to appl

Run transformer and pool its outputs. Args: transformer: A transformer model. input_ids: Indices of subwords. attention_m

Args: transformer: input_ids: torch.LongTensor: input_mask: (Default value = None) offsets: torch.LongTensor: (Defaul

A simple tagger using a linear layer with an optional CRF (:cite:`lafferty2001conditional`) layer for any tagging tasks including PoS tagging