MCPcopy Create free account

hub / github.com/huggingface/transformers / functions

Functions4,119 in github.com/huggingface/transformers

Methodget_output_embeddings
(self)
src/transformers/modeling_openai.py:480
Methodget_output_embeddings
(self)
src/transformers/modeling_openai.py:572
Methodget_output_embeddings
(self)
src/transformers/modeling_roberta.py:178
Methodget_output_embeddings
(self)
src/transformers/modeling_tf_t5.py:541
Methodget_output_embeddings
(self)
src/transformers/modeling_tf_t5.py:891
Methodget_output_embeddings
(self)
src/transformers/modeling_electra.py:560
Methodget_output_embeddings
(self)
src/transformers/modeling_tf_ctrl.py:552
Methodget_rust_tokenizer
(self, **kwargs)
tests/test_tokenization_distilbert.py:27
Methodget_rust_tokenizer
(self, **kwargs)
tests/test_tokenization_roberta.py:71
Functionget_setup_file
()
examples/test_examples.py:44
Functionget_setup_file
()
examples/bert-loses-patience/test_run_glue_with_pabee.py:15
Methodget_special_tokens_mask
Retrieves sequence ids from a token list that has no special tokens added. This method is called when adding special tokens using the
templates/adding_a_new_model/tokenization_xxx.py:171
Methodget_special_tokens_mask
Retrieves sequence ids from a token list that has no special tokens added. This method is called when adding special tokens using the
src/transformers/tokenization_xlm_roberta.py:204
Methodget_special_tokens_mask
Retrieves sequence ids from a token list that has no special tokens added. This method is called when adding special tokens using the
src/transformers/tokenization_roberta.py:191
Methodget_special_tokens_mask
Retrieves sequence ids from a token list that has no special tokens added. This method is called when adding special tokens using the
src/transformers/tokenization_albert.py:261
Methodget_special_tokens_mask
Retrieves sequence ids from a token list that has no special tokens added. This method is called when adding special tokens using the
src/transformers/tokenization_xlnet.py:265
Methodget_special_tokens_mask
Retrieves sequence ids from a token list that has no special tokens added. This method is called when adding special tokens using the
src/transformers/tokenization_bert.py:261
Methodget_special_tokens_mask
Get list where entries are [1] if a token is [eos] or [pad] else 0.
src/transformers/tokenization_marian.py:215
Methodget_special_tokens_mask
Retrieves sequence ids from a token list that has no special tokens added. This method is called when adding special tokens using the
src/transformers/tokenization_camembert.py:169
Methodget_special_tokens_mask
Retrieves sequence ids from a token list that has no special tokens added. This method is called when adding special tokens using the
src/transformers/tokenization_utils_base.py:2051
Methodget_test_examples
Gets a collection of :class:`InputExample` for the test set.
src/transformers/data/processors/utils.py:102
Methodget_test_examples
See base class.
src/transformers/data/processors/glue.py:188
Methodget_test_examples
See base class.
src/transformers/data/processors/glue.py:230
Methodget_test_examples
See base class.
src/transformers/data/processors/glue.py:259
Methodget_test_examples
See base class.
src/transformers/data/processors/glue.py:284
Methodget_test_examples
See base class.
src/transformers/data/processors/glue.py:327
Methodget_test_examples
See base class.
src/transformers/data/processors/glue.py:369
Methodget_test_examples
See base class.
src/transformers/data/processors/glue.py:459
Methodget_test_examples
See base class.
src/transformers/data/processors/glue.py:501
Methodget_test_examples
See base class.
src/transformers/data/processors/glue.py:543
Methodget_test_examples
See base class.
src/transformers/data/processors/xnli.py:52
Methodget_test_examples
Gets a collection of `InputExample`s for the test set.
examples/multiple-choice/utils_multiple_choice.py:227
Methodget_test_examples
See base class.
examples/multiple-choice/utils_multiple_choice.py:257
Methodget_test_examples
See base class.
examples/multiple-choice/utils_multiple_choice.py:316
Methodget_test_examples
See base class.
examples/multiple-choice/utils_multiple_choice.py:362
Methodget_tokenizer
(self, **kwargs)
templates/adding_a_new_model/tests/test_tokenization_xxx.py:51
Methodget_tokenizer
(self, **kwargs)
tests/test_tokenization_bert_japanese.py:172
Methodget_tokenizer
(self, **kwargs)
tests/test_tokenization_marian.py:54
Methodget_train_examples
Gets a collection of :class:`InputExample` for the train set.
src/transformers/data/processors/utils.py:94
Methodget_train_examples
See base class.
src/transformers/data/processors/glue.py:179
Methodget_train_examples
See base class.
src/transformers/data/processors/glue.py:222
Methodget_train_examples
See base class.
src/transformers/data/processors/glue.py:276
Methodget_train_examples
See base class.
src/transformers/data/processors/glue.py:319
Methodget_train_examples
See base class.
src/transformers/data/processors/glue.py:361
Methodget_train_examples
See base class.
src/transformers/data/processors/glue.py:451
Methodget_train_examples
See base class.
src/transformers/data/processors/glue.py:493
Methodget_train_examples
See base class.
src/transformers/data/processors/glue.py:535
Methodget_train_examples
See base class.
src/transformers/data/processors/xnli.py:36
Methodget_train_examples
Returns the training examples from the data directory. Args: data_dir: Directory containing the data files used for trai
src/transformers/data/processors/squad.py:513
Methodget_train_examples
Gets a collection of `InputExample`s for the train set.
examples/multiple-choice/utils_multiple_choice.py:219
Methodget_train_examples
See base class.
examples/multiple-choice/utils_multiple_choice.py:239
Methodget_train_examples
See base class.
examples/multiple-choice/utils_multiple_choice.py:306
Methodget_train_examples
See base class.
examples/multiple-choice/utils_multiple_choice.py:352
Methodget_varlen_iter
(self, start=0, std=5, min_len=5, max_deviation=3)
src/transformers/tokenization_transfo_xl.py:538
Methodget_vocab
(self)
src/transformers/tokenization_xlm_roberta.py:264
Methodget_vocab
(self)
src/transformers/tokenization_openai.py:128
Methodget_vocab
(self)
src/transformers/tokenization_albert.py:161
Methodget_vocab
(self)
src/transformers/tokenization_reformer.py:121
Methodget_vocab
(self)
src/transformers/tokenization_gpt2.py:177
Methodget_vocab
(self)
src/transformers/tokenization_transfo_xl.py:336
Methodget_vocab
(self)
src/transformers/tokenization_ctrl.py:157
Methodget_vocab
(self)
src/transformers/tokenization_xlm.py:702
Methodget_vocab
(self)
src/transformers/tokenization_xlnet.py:164
Methodget_vocab
(self)
src/transformers/tokenization_bert.py:205
Methodget_vocab
(self)
src/transformers/tokenization_marian.py:190
Methodget_vocab
Returns the vocabulary as a dict of {token: index} pairs. `tokenizer.get_vocab()[token]` is equivalent to `tokenizer.convert_tokens_to_ids(token)` wh
src/transformers/tokenization_utils.py:174
Methodgpu_list
(self)
src/transformers/benchmark/benchmark_args_tf.py:93
Functionhans_data_collator
Data collator that removes the "pairID" key if present.
examples/adversarial/run_hans.py:86
Methodhidden_size
(self)
templates/adding_a_new_model/configuration_xxx.py:105
Methodhidden_size
(self)
src/transformers/configuration_bart.py:135
Methodhidden_size
(self)
src/transformers/configuration_ctrl.py:129
Methodhidden_size
(self)
src/transformers/configuration_openai.py:163
Methodhidden_size
(self)
src/transformers/configuration_xlm.py:243
Methodhidden_size
(self)
src/transformers/configuration_distilbert.py:130
Methodhidden_size
(self)
src/transformers/configuration_t5.py:104
Methodhidden_size
(self)
src/transformers/configuration_xlnet.py:211
Methodhidden_size
(self)
src/transformers/configuration_transfo_xl.py:198
Methodhidden_size
(self)
src/transformers/configuration_gpt2.py:168
Functioninfo_command_factory
(_)
src/transformers/commands/env.py:9
Methodinit_mems
(self, bsz)
src/transformers/modeling_tf_transfo_xl.py:801
Methodinit_mems
(self, bsz)
src/transformers/modeling_transfo_xl.py:935
Methodis_fast
(self)
src/transformers/tokenization_utils_fast.py:116
Methodis_fast
Indicate if this BatchEncoding was generated from the result of a PreTrainedTokenizerFast Returns: True if generated from subclasses
src/transformers/tokenization_utils_base.py:162
Methodis_fast
(self)
src/transformers/tokenization_utils.py:166
Methodis_gpu
(self)
src/transformers/benchmark/benchmark_args.py:85
Methodis_gpu
(self)
src/transformers/benchmark/benchmark_args_tf.py:104
Functionis_memory_tracing_enabled
()
src/transformers/benchmark/benchmark_utils.py:103
Methodis_tpu
(self)
src/transformers/benchmark/benchmark_args.py:65
Methodis_tpu
(self)
src/transformers/benchmark/benchmark_args_tf.py:83
Functionkeras_serializable
Decorate a Keras Layer class to support Keras serialization. This is done by: 1. adding a `transformers_config` dict to the Keras config
src/transformers/modeling_tf_utils.py:49
Methodlength_average
Returns the average probability of tokens in a sequence.
examples/seq2seq/bertabs/modeling_bertabs.py:725
Methodlength_none
Returns unmodified scores.
examples/seq2seq/bertabs/modeling_bertabs.py:731
Methodlength_wu
NMT length re-ranking score from "Google's Neural Machine Translation System" :cite:`wu2016google`.
examples/seq2seq/bertabs/modeling_bertabs.py:716
Methodload_dataset
Load datasets. Called after prepare data.
examples/text-classification/run_pl_glue.py:75
Methodload_hf_checkpoint
(self, *args, **kwargs)
examples/lightning_base.py:98
Functionload_pytorch_model_in_tf2_model
Load pytorch checkpoints in a TF 2.0 model
src/transformers/modeling_tf_pytorch_utils.py:97
Methodlog_stats
(self)
examples/bert-loses-patience/pabee/modeling_pabee_albert.py:85
Functionlr_lambda
(current_step)
src/transformers/optimization.py:39
Functionmake_es_index_snippets
(es_client, passages_dset, index_name="english_wiki_kilt_snippets_100w")
examples/longform-qa/eli5_utils.py:27
Functionmake_qa_dense_index
( qa_embedder, tokenizer, passages_dset, batch_size=512, max_length=128, index_name="k
examples/longform-qa/eli5_utils.py:555
← previousnext →3,001–3,100 of 4,119, ranked by callers