MCPcopy Create free account

hub / github.com/huggingface/transformers / functions

Functions4,119 in github.com/huggingface/transformers

Methodprepare_data
Called to initialize data. Use the call to construct features
examples/token-classification/run_pl_ner.py:48
Methodprepare_data
Called to initialize data. Use the call to construct features
examples/text-classification/run_pl_glue.py:47
Methodprepare_for_tokenization
(self, text, is_pretokenized=False, **kwargs)
src/transformers/tokenization_roberta.py:245
Methodprepare_for_tokenization
(self, text, is_pretokenized=False, **kwargs)
src/transformers/tokenization_gpt2.py:281
Methodprepare_for_tokenization
(self, text, is_pretokenized=False, **kwargs)
src/transformers/tokenization_transfo_xl.py:358
Methodprepare_inputs_for_generation
(self, input_ids, past, **kwargs)
src/transformers/modeling_ctrl.py:494
Methodprepare_inputs_for_generation
(self, inputs, past, **model_kwargs)
src/transformers/modeling_tf_transfo_xl.py:875
Methodprepare_inputs_for_generation
(self, input_ids, past, **kwargs)
src/transformers/modeling_xlnet.py:998
Methodprepare_inputs_for_generation
(self, decoder_input_ids, past, attention_mask, use_cache, **kwargs)
src/transformers/modeling_bart.py:1015
Methodprepare_inputs_for_generation
(self, input_ids, **kwargs)
src/transformers/modeling_xlm.py:612
Methodprepare_inputs_for_generation
(self, input_ids, past, **model_kwargs)
src/transformers/modeling_transfo_xl.py:1018
Methodprepare_inputs_for_generation
(self, input_ids, past, **kwargs)
src/transformers/modeling_reformer.py:1784
Methodprepare_inputs_for_generation
(self, input_ids, attention_mask=None, **model_kwargs)
src/transformers/modeling_bert.py:994
Methodprepare_inputs_for_generation
(self, input_ids, attention_mask=None, **model_kwargs)
src/transformers/modeling_bert.py:1098
Methodprepare_inputs_for_generation
(self, input_ids, past, attention_mask, **kwargs)
src/transformers/modeling_encoder_decoder.py:295
Methodprepare_inputs_for_generation
(self, inputs, past, **kwargs)
src/transformers/modeling_tf_gpt2.py:535
Methodprepare_inputs_for_generation
(self, input_ids, past, **kwargs)
src/transformers/modeling_gpt2.py:539
Methodprepare_inputs_for_generation
(self, inputs, past, **kwargs)
src/transformers/modeling_tf_xlnet.py:883
Methodprepare_inputs_for_generation
(self, inputs, **kwargs)
src/transformers/modeling_tf_xlm.py:688
Methodprepare_inputs_for_generation
(self, input_ids, past, attention_mask, use_cache, **kwargs)
src/transformers/modeling_t5.py:1160
Methodprepare_inputs_for_generation
(self, inputs, past, attention_mask, use_cache, **kwargs)
src/transformers/modeling_tf_t5.py:1164
Methodprepare_inputs_for_generation
(self, inputs, past, **kwargs)
src/transformers/modeling_tf_ctrl.py:555
Functionprepare_transfoxl_input
(args, _, tokenizer, prompt_text)
examples/text-generation/run_generation.py:127
Methodprepare_translation_batch
Prepare model inputs for translation. For best performance, translate one sentence at a time. Arguments: src_texts: list of src la
src/transformers/tokenization_marian.py:125
Functionprepare_xlm_input
(args, model, tokenizer, prompt_text)
examples/text-generation/run_generation.py:96
Functionprepare_xlnet_input
(args, _, tokenizer, prompt_text)
examples/text-generation/run_generation.py:122
Methodprint_and_log
(*args)
src/transformers/benchmark/benchmark_utils.py:589
Methodprune_heads
(self, heads)
src/transformers/modeling_tf_albert.py:268
Methodprune_heads
(self, heads)
src/transformers/modeling_xlnet.py:224
Methodprune_heads
(self, heads)
src/transformers/modeling_tf_openai.py:91
Methodprune_heads
(self, heads)
src/transformers/modeling_tf_mobilebert.py:322
Methodprune_heads
Prunes heads of the base model. Arguments: heads_to_prune: dict with keys being selected layer indices (`int`) and asso
src/transformers/modeling_tf_utils.py:305
Methodprune_heads
(self, heads)
src/transformers/modeling_tf_bert.py:315
Methodprune_heads
(self, heads)
src/transformers/modeling_tf_distilbert.py:214
Methodprune_heads
(self, heads)
src/transformers/modeling_tf_gpt2.py:84
Methodprune_heads
(self, heads)
src/transformers/modeling_tf_xlnet.py:129
Methodprune_heads
(self, heads)
src/transformers/modeling_tf_t5.py:135
Functionprune_layer
Prune a Conv1D or nn.Linear layer (a model parameters) to keep only entries in index. Return the pruned layer as a new layer with requires_gr
src/transformers/modeling_utils.py:2146
Functionquery_qa_dense_index_nn
(passage, qa_embedder, tokenizer, wiki_passages, wiki_index, n_results=10, min_length=20)
examples/longform-qa/eli5_utils.py:626
Methodregister_subcommand
Register this command to argparse so it's available for the transformer-cli :param parser: Root parser to register command-specific a
src/transformers/commands/train.py:28
Methodregister_subcommand
(parser: ArgumentParser)
src/transformers/commands/user.py:18
Methodregister_subcommand
(parser: ArgumentParser)
src/transformers/commands/download.py:12
Methodregister_subcommand
(parser: ArgumentParser)
src/transformers/commands/__init__.py:8
Methodregister_subcommand
(parser: ArgumentParser)
src/transformers/commands/env.py:15
Methodregister_subcommand
Register this command to argparse so it's available for the transformer-cli :param parser: Root parser to register command-specific a
src/transformers/commands/convert.py:19
Methodregister_subcommand
Register this command to argparse so it's available for the transformer-cli :param parser: Root parser to register command-specific a
src/transformers/commands/serving.py:80
Methodrel_shift
perform relative shift to form the relative attention score.
src/transformers/modeling_xlnet.py:228
Methodrepeat_beam_size_times
Repeat beam_size times along batch dimension.
examples/seq2seq/bertabs/modeling_bertabs.py:608
Functionrequire_multigpu
Decorator marking a test that requires a multi-GPU setup (in PyTorch). These tests are skipped on a machine without multiple GPUs. To r
tests/utils.py:97
Functionrequire_tf
Decorator marking a test that requires TensorFlow. These tests are skipped when TensorFlow isn't installed.
tests/utils.py:85
Functionrequire_torch
Decorator marking a test that requires PyTorch. These tests are skipped when PyTorch isn't installed.
tests/utils.py:73
Methodreset_length
(self, tgt_len, ext_len, mem_len)
src/transformers/modeling_tf_transfo_xl.py:798
Methodreset_length
(self, tgt_len, ext_len, mem_len)
src/transformers/modeling_transfo_xl.py:932
Methodreset_stats
(self)
examples/bert-loses-patience/pabee/modeling_pabee_albert.py:81
Methodresize_token_embeddings
(self, new_num_tokens: int)
src/transformers/modeling_bart.py:909
Methodresize_token_embeddings
Resize input token embeddings matrix of the model if new_num_tokens != config.vocab_size. Take care of tying weights embeddings afterwards if
src/transformers/modeling_tf_utils.py:226
Methodresize_token_embeddings
Resize input token embeddings matrix of the model if new_num_tokens != config.vocab_size. Take care of tying weights embeddings afterwards if
src/transformers/modeling_transfo_xl.py:513
Methodroberta_dict_integration_testing
(self)
tests/test_tokenization_roberta.py:91
Methodrun
(self)
src/transformers/commands/train.py:124
Methodrun
(self)
src/transformers/commands/user.py:72
Methodrun
(self)
src/transformers/commands/user.py:99
Methodrun
(self)
src/transformers/commands/user.py:116
Methodrun
(self)
src/transformers/commands/user.py:142
Methodrun
(self)
src/transformers/commands/user.py:161
Methodrun
(self)
src/transformers/commands/user.py:187
Methodrun
(self)
src/transformers/commands/download.py:28
Methodrun
(self)
src/transformers/commands/__init__.py:12
Methodrun
(self)
src/transformers/commands/env.py:19
Methodrun
(self)
src/transformers/commands/convert.py:64
Methodrun
(self)
src/transformers/commands/serving.py:156
Methodrun
(self)
src/transformers/benchmark/benchmark_utils.py:249
Methodrun
(self)
src/transformers/benchmark/benchmark_utils.py:636
Functionrun_command_factory
(args)
src/transformers/commands/run.py:25
Functionrun_func
(func)
src/transformers/benchmark/benchmark_tf.py:51
Functionrun_in_eager_mode
(*args, **kwargs)
src/transformers/benchmark/benchmark_tf.py:53
Functionrun_in_graph_mode
(*args, **kwargs)
src/transformers/benchmark/benchmark_tf.py:58
Functionrun_with_tf_optimizations
(do_eager_mode: bool, use_xla: bool)
src/transformers/benchmark/benchmark_tf.py:50
Methodsanity_check_gradients
T5
examples/seq2seq/distillation.py:305
Methodsave
(self, data: List[dict])
src/transformers/pipelines.py:240
Methodsave
(self, data: dict)
src/transformers/pipelines.py:264
Methodsave
(self, data: dict)
src/transformers/pipelines.py:293
Methodsave_binary
(self, data: Union[dict, List[dict]])
src/transformers/pipelines.py:296
Methodsave_pretrained
Save a model and its configuration file to a directory, so that it can be re-loaded using the `:func:`~transformers.PreTrainedModel.from_
src/transformers/modeling_utils.py:473
Methodsave_pretrained
Save a model and its configuration file to a directory, so that it can be re-loaded using the :func:`~transformers.PreTrainedModel.from_p
src/transformers/modeling_tf_utils.py:314
Methodsave_pretrained
(self, save_directory)
src/transformers/tokenization_transfo_xl.py:486
Methodsave_pretrained
Save a model card object to the directory or file `save_directory_or_file`.
src/transformers/modelcard.py:75
Methodsave_pretrained
Save the tokenizer vocabulary files together with: - added tokens, - special-tokens-to-class-attributes-mapping,
src/transformers/tokenization_utils_base.py:1334
Methodsave_pretrained
Save a configuration object to the directory `save_directory`, so that it can be re-loaded using the :func:`~transformers.PretrainedC
src/transformers/configuration_utils.py:126
Methodsave_vocabulary
Save the sentencepiece vocabulary (copy original file) and special tokens file to a directory. Args: save_directory (:ob
src/transformers/tokenization_xlm_roberta.py:292
Methodsave_vocabulary
Save the vocabulary and special tokens file to a directory. Args: save_directory (:obj:`str`): The direc
src/transformers/tokenization_openai.py:203
Methodsave_vocabulary
Save the sentencepiece vocabulary (copy original file) and special tokens file to a directory. Args: save_directory (:ob
src/transformers/tokenization_albert.py:323
Methodsave_vocabulary
Save the sentencepiece vocabulary (copy original file) and special tokens file to a directory.
src/transformers/tokenization_reformer.py:168
Methodsave_vocabulary
Save the vocabulary and special tokens file to a directory. Args: save_directory (:obj:`str`): The direc
src/transformers/tokenization_gpt2.py:246
Methodsave_vocabulary
Save the vocabulary and special tokens file to a directory. Args: vocab_path (:obj:`str`): The directory
src/transformers/tokenization_transfo_xl.py:189
Methodsave_vocabulary
Save the vocabulary and special tokens file to a directory. Args: save_directory (:obj:`str`): The direc
src/transformers/tokenization_ctrl.py:228
Methodsave_vocabulary
Save the vocabulary and special tokens file to a directory. Args: save_directory (:obj:`str`): The direc
src/transformers/tokenization_xlm.py:941
Methodsave_vocabulary
(self, save_directory: str)
src/transformers/tokenization_utils_fast.py:447
Methodsave_vocabulary
Save the sentencepiece vocabulary (copy original file) and special tokens file to a directory. Args: save_directory (:ob
src/transformers/tokenization_xlnet.py:324
Methodsave_vocabulary
Save the sentencepiece vocabulary (copy original file) and special tokens file to a directory. Args: vocab_path (:obj:`s
src/transformers/tokenization_bert.py:322
Methodsave_vocabulary
save vocab file to json and copy spm files from their original path.
src/transformers/tokenization_marian.py:177
← previousnext →3,201–3,300 of 4,119, ranked by callers