Warn the user if the columns are likely not in the expected order.

Applies noise by randomly deleting words. WARNING: nltk's tokenization/detokenization is designed primarily for English. For other langu

Applies noise by randomly deleting words. WARNING: nltk's tokenization/detokenization is designed primarily for English. For other langu

Applies noise by randomly deleting words. WARNING: nltk's tokenization/detokenization is designed primarily for English. For other langu

:param reconstruction: output of Autoencoder.decode (shape: [batch, n_inputs]) :param original_input: input of Autoencoder.encode (shape: [ba

Hook called once the owning model is fully constructed, with every module, the tokenizer or processor, and any legacy checkpoint fixu

Given a list of sentences / texts, this function performs paraphrase mining. It compares all sentences against all other sentences and return

* modified to run programmatically

Do forward pass on a minibatch of the input features and return corresponding logits. Pairs are tokenized and moved to the device before the

Preprocesses the input texts and returns a dictionary of preprocessed features. Args: inputs (Sequence[SingleInput | Pai

Reverse FA2 input flattening on a features dict, in place. When :class:`Transformer` runs with FA2 unpadding, ``DataCollatorWithFlattening`` flat

Run one sampler and print timing + batch count.

Save the module to disk. This method should be overridden by subclasses to implement the specific behavior of the module. Args:

Saves the tokenizer to the specified output path. Args: output_path (str): Path to save the tokenizer. **kwa

Performs semantic search using sparse embeddings with Elasticsearch. Args: query_embeddings_decoded: List of query embeddings in for

Performs semantic search using sparse embeddings with OpenSearch. Args: query_embeddings_decoded: List of query embeddings in format

Sets a specific adapter by forcing the model to use that adapter and disable the other adapters. Args: *args:

Sets the `include_prompt` attribute in the pooling layer in the model, if there is one. This is useful for INSTRUCTOR models, as the

Install a meta path finder that issues deprecation warnings for deprecated import paths and aliases them to their new locations in ``sys.modules``

Configure logging + TF32. Tees to logs/{RUN_NAME}.log and silences HTTP spam.

Configure logging + TF32. Tees to logs/{RUN_NAME}.log and silences HTTP spam.