MCPcopy Create free account

hub / github.com/huggingface/transformers / functions

Functions4,119 in github.com/huggingface/transformers

↓ 1 callersMethodrun_torch
(self)
src/transformers/commands/train.py:129
↓ 1 callersFunctionsample_without_replacement
categorical sampling witouth replacement is currently not implemented the gumbel-max trick will do for now see https://github
src/transformers/modeling_tf_utils.py:1801
↓ 1 callersMethodsanity_check_gradients
(self)
examples/seq2seq/distillation.py:64
↓ 1 callersFunctionsanity_checks
A bunch of args sanity checks to perform even starting...
examples/distillation/train.py:55
↓ 1 callersFunctionsave_git_info
Save git information to output_dir/git_log.json
examples/seq2seq/utils.py:198
↓ 1 callersMethodsave_metrics
(self, latest_metrics, type_path)
examples/seq2seq/finetune.py:152
↓ 1 callersFunctionsave_rouge_scores
(str_scores)
examples/seq2seq/bertabs/run_summarization.py:175
↓ 1 callersFunctionsave_summaries
Write the summaries in fies that are prefixed by the original files' name with the `_summary` appended. Attributes: original_documen
examples/seq2seq/bertabs/run_summarization.py:100
↓ 1 callersMethodsave_to_csv
(self, result_dict, filename)
src/transformers/benchmark/benchmark_utils.py:840
↓ 1 callersFunctionsave_tokenizer
(self, save_directory)
src/transformers/convert_marian_to_pytorch.py:300
↓ 1 callersFunctionsave_tokenizer_config
(dest_dir: Path)
src/transformers/convert_marian_to_pytorch.py:270
↓ 1 callersMethodsave_vocabulary
Save the tokenizer vocabulary to a directory or file.
templates/adding_a_new_model/tokenization_xxx.py:214
↓ 1 callersFunctionscaled_dot_product_attention
(q, k, v, mask, attention_mask=None, head_mask=None)
src/transformers/modeling_ctrl.py:61
↓ 1 callersFunctionscaled_dot_product_attention
(q, k, v, mask, attention_mask=None, head_mask=None)
src/transformers/modeling_tf_ctrl.py:63
↓ 1 callersFunctionscan_code_for_links
Scans the file to find links using a regular expression. Returns a list of links.
utils/link_tester.py:45
↓ 1 callersFunctionscatter_values_on_batch_indices
(values, batch_indices)
src/transformers/modeling_tf_utils.py:1525
↓ 1 callersFunctionschedule_threshold
( step: int, total_step: int, warmup_steps: int, initial_threshold: float, final_threshold
examples/movement-pruning/masked_run_glue.py:70
↓ 1 callersFunctionschedule_threshold
( step: int, total_step: int, warmup_steps: int, initial_threshold: float, final_threshold
examples/movement-pruning/masked_run_squad.py:74
↓ 1 callersMethodscore
Rescores a prediction based on penalty functions
examples/seq2seq/bertabs/modeling_bertabs.py:684
↓ 1 callersFunctionselect_field
(features, field)
examples/contrib/run_swag.py:212
↓ 1 callersMethodsetUp
(self)
tests/test_tokenization_bert_japanese.py:37
↓ 1 callersFunctionset_block_weights_in_torch
(weights, torch_block, hidden_size)
src/transformers/convert_reformer_trax_checkpoint_to_pytorch.py:79
↓ 1 callersFunctionset_generic_model_params
(discrim_weights, discrim_meta)
examples/text-generation/pplm/run_pplm.py:562
↓ 1 callersMethodset_input_embeddings
(self, new_embeddings)
templates/adding_a_new_model/modeling_xxx.py:314
↓ 1 callersMethodset_input_embeddings
Set model's input embeddings Args: value (:obj:`nn.Module`): A module mapping vocabulary to hidden state
src/transformers/modeling_utils.py:323
↓ 1 callersMethodset_input_embeddings
Set model's input embeddings Args: value (:obj:`tf.keras.layers.Layer`): A module mapping vocabulary to
src/transformers/modeling_tf_utils.py:202
↓ 1 callersMethodset_input_embeddings
(self, new_embeddings)
src/transformers/modeling_transfo_xl.py:702
↓ 1 callersMethodset_lang
Set the current language code in order to call tokenizer properly.
src/transformers/tokenization_bart.py:127
↓ 1 callersFunctionset_layer_weights_in_torch_local
(weights, torch_layer, hidden_size)
src/transformers/convert_reformer_trax_checkpoint_to_pytorch.py:58
↓ 1 callersFunctionset_layer_weights_in_torch_lsh
(weights, torch_layer, hidden_size)
src/transformers/convert_reformer_trax_checkpoint_to_pytorch.py:40
↓ 1 callersFunctionset_model_weights_in_torch
(weights, torch_model, hidden_size)
src/transformers/convert_reformer_trax_checkpoint_to_pytorch.py:128
↓ 1 callersMethodset_patience
(self, patience)
examples/bert-loses-patience/pabee/modeling_pabee_bert.py:83
↓ 1 callersMethodset_regression_threshold
(self, threshold)
examples/bert-loses-patience/pabee/modeling_pabee_bert.py:80
↓ 1 callersFunctionset_seed
(seed: int)
src/transformers/trainer_tf.py:25
↓ 1 callersFunctionset_seed
(args: argparse.Namespace)
examples/lightning_base.py:45
↓ 1 callersFunctionset_seed
(args)
examples/text-generation/run_generation.py:74
↓ 1 callersFunctionset_seed
Set the random seed.
examples/distillation/utils.py:125
↓ 1 callersMethodset_truncation_and_padding
Define the truncation and the padding strategies for fast tokenizers (provided by HuggingFace tokenizers library) and restore the tokeniz
src/transformers/tokenization_utils_fast.py:243
↓ 1 callersFunctionsetup
(app)
docs/source/conf.py:192
↓ 1 callersFunctionsimple_accuracy
(preds, labels)
examples/multiple-choice/run_multiple_choice.py:42
↓ 1 callersFunctionsimple_accuracy
(preds, labels)
examples/multiple-choice/run_tf_multiple_choice.py:42
↓ 1 callersFunctionsix_ensure_text
(s, encoding="utf-8", errors="strict")
src/transformers/tokenization_flaubert.py:68
↓ 1 callersMethodsplit_heads
(self, x)
src/transformers/modeling_tf_openai.py:140
↓ 1 callersMethodsplit_heads
(self, x)
src/transformers/modeling_tf_gpt2.py:133
↓ 1 callersMethodsplit_heads
(self, x, k=False)
src/transformers/modeling_gpt2.py:175
↓ 1 callersMethodsplit_heads
(self, x, k=False)
src/transformers/modeling_openai.py:193
↓ 1 callersMethodsplit_into_heads
(self, x, batch_size)
src/transformers/modeling_ctrl.py:119
↓ 1 callersMethodsplit_into_heads
(self, x, batch_size)
src/transformers/modeling_tf_ctrl.py:102
↓ 1 callersMethodsub_keys
(self, layer_prefix)
src/transformers/convert_marian_to_pytorch.py:445
↓ 1 callersMethodtabulate
Inspired by: stackoverflow.com/a/8356620/593036 stackoverflow.com/questions/9535954/printing-lists-as-tabular-data
src/transformers/commands/user.py:127
↓ 1 callersMethodtest_all_tokenizers
(self)
tests/test_tokenization_fast.py:48
↓ 1 callersMethodtest_pretokenized_tokenizers
(self)
tests/test_tokenization_fast.py:65
↓ 1 callersFunctiontext_standardize
fixes some issues the spacy tokenizer had on books corpus also does some whitespace standardization
src/transformers/tokenization_openai.py:60
↓ 1 callersMethodtfds_map
Some tensorflow_datasets datasets are not formatted the same way the GLUE datasets are. This method converts examples to the correct format.
src/transformers/data/processors/utils.py:110
↓ 1 callersMethodtie_weights
Run this to be sure output and input (adaptive) softmax weights are tied
src/transformers/modeling_transfo_xl.py:911
↓ 1 callersMethodtie_weights
(self)
src/transformers/modeling_albert.py:592
↓ 1 callersMethodtie_weights
(self)
src/transformers/modeling_albert.py:750
↓ 1 callersFunctiontile
Tiles x on dimension dim count times.
examples/seq2seq/bertabs/modeling_bertabs.py:960
↓ 1 callersMethodto_dict
Serializes this instance to a Python dictionary. Override the default `to_dict()` from `PretrainedConfig`. Returns: :obj
src/transformers/configuration_encoder_decoder.py:93
↓ 1 callersMethodto_json_file
Save this instance to a json file. Args: json_file_path (:obj:`string`): Path to the JSON file in which
src/transformers/configuration_utils.py:386
↓ 1 callersFunctionto_list
(tensor)
templates/adding_a_new_example_script/run_xxx.py:75
↓ 1 callersFunctionto_list
(tensor)
examples/distillation/run_squad_w_distillation.py:88
↓ 1 callersFunctionto_list
(tensor)
examples/movement-pruning/masked_run_squad.py:111
↓ 1 callersFunctionto_list
(tensor)
examples/question-answering/run_squad.py:70
↓ 1 callersMethodto_sanitized_dict
Sanitized serialization to use with TensorBoard’s hparams
src/transformers/training_args.py:209
↓ 1 callersFunctionto_tf_var_name
(name: str)
src/transformers/convert_bert_pytorch_checkpoint_to_original_tf.py:64
↓ 1 callersMethodtoken_to_chars
Get the character span corresponding to an encoded token in a sequence of the batch. Character spans are returned as a CharSpan Name
src/transformers/tokenization_utils_base.py:321
↓ 1 callersMethodtoken_to_word
Get the index of the word corresponding (i.e. comprising) to an encoded token in a sequence of the batch. Can be called as:
src/transformers/tokenization_utils_base.py:231
↓ 1 callersMethodtokenize
Tokenizes a piece of text into characters. For example: input = "apple" output = ["a", "p", "p", "l", "e"] Ar
src/transformers/tokenization_bert_japanese.py:237
↓ 1 callersMethodtokenize
Converts a string in a sequence of tokens (string), using the tokenizer. Split in words for word-based vocabulary or sub-words for sub-wo
src/transformers/tokenization_utils.py:256
↓ 1 callersFunctiontokenize_and_encode
Tokenize and encode a nested object
examples/contrib/run_openai_gpt.py:182
↓ 1 callersMethodtokens
(self, batch_index: int = 0)
src/transformers/tokenization_utils_base.py:221
↓ 1 callersFunctiontop_k_filter
Masks everything but the k top entries as -infinity (1e10). Used to mask logits such that e^-infinity -> 0 won't contribute to the sum of
examples/text-generation/pplm/run_pplm.py:78
↓ 1 callersFunctiontrain
Train the model
templates/adding_a_new_example_script/run_xxx.py:79
↓ 1 callersFunctiontrain
Train the model
examples/contrib/run_swag.py:269
↓ 1 callersFunctiontrain
Train the model
examples/contrib/mm-imdb/run_mmimdb.py:64
↓ 1 callersFunctiontrain
Train the model
examples/distillation/run_squad_w_distillation.py:92
↓ 1 callersFunctiontrain
Train the model
examples/movement-pruning/masked_run_glue.py:107
↓ 1 callersFunctiontrain
Train the model
examples/movement-pruning/masked_run_squad.py:115
↓ 1 callersFunctiontrain
Train the model
examples/text-classification/run_xnli.py:63
↓ 1 callersFunctiontrain
Train the model
examples/bert-loses-patience/run_glue_with_pabee.py:71
↓ 1 callersFunctiontrain
Train the model
examples/question-answering/run_squad.py:74
↓ 1 callersMethodtrain_custom
(self)
examples/text-generation/pplm/run_pplm_discrim_train.py:60
↓ 1 callersFunctiontrain_discriminator
( dataset, dataset_fp=None, pretrained_model="gpt2-medium", epochs=10, batch_size=64,
examples/text-generation/pplm/run_pplm_discrim_train.py:219
↓ 1 callersFunctiontrain_epoch
(data_loader, discriminator, optimizer, epoch=0, log_interval=10, device="cpu")
examples/text-generation/pplm/run_pplm_discrim_train.py:134
↓ 1 callersMethodtrain_memory
(self, *args, **kwargs)
src/transformers/benchmark/benchmark_utils.py:633
↓ 1 callersFunctiontrain_qa_retriever_epoch
(model, dataset, tokenizer, optimizer, scheduler, args, e=0)
examples/longform-qa/eli5_utils.py:209
↓ 1 callersFunctiontrain_qa_s2s_epoch
(model, dataset, tokenizer, optimizer, scheduler, args, e=0, curriculum=False)
examples/longform-qa/eli5_utils.py:399
↓ 1 callersMethodtrain_speed
(self, *args, **kwargs)
src/transformers/benchmark/benchmark_utils.py:627
↓ 1 callersMethodtranslate_src_text
(self, **tokenizer_kwargs)
tests/test_modeling_marian.py:100
↓ 1 callersMethodtranspose_for_scores
(self, x, batch_size)
src/transformers/modeling_tf_mobilebert.py:242
↓ 1 callersMethodtranspose_for_scores
(self, x, batch_size)
src/transformers/modeling_tf_bert.py:239
↓ 1 callersMethodtranspose_for_scores
(self, x)
src/transformers/modeling_mobilebert.py:221
↓ 1 callersMethodtranspose_for_scores
(self, x)
examples/movement-pruning/emmental/modeling_bert_masked.py:117
↓ 1 callersMethodtrim_seq2seq_batch
(batch, pad_token_id)
examples/seq2seq/utils.py:122
↓ 1 callersMethodtruncate_sequences
Truncates a sequence pair in place to the maximum length. Args: ids: list of tokenized input ids. Can be obtained from a string
src/transformers/tokenization_utils.py:734
↓ 1 callersFunctiontry_infer_format_from_ext
(path: str)
src/transformers/commands/run.py:11
↓ 1 callersFunctionunzip
(zip_path: str, dest_dir: str)
src/transformers/convert_marian_to_pytorch.py:537
↓ 1 callersMethodupdate_state
(self, new_input, previous_layer_inputs)
examples/seq2seq/bertabs/modeling_bertabs.py:593
↓ 1 callersFunctionurl_to_filename
Convert `url` into a hashed filename in a repeatable way. If `etag` is specified, append its hash to the url's, delimited by a period.
src/transformers/file_utils.py:475
← previousnext →1,301–1,400 of 4,119, ranked by callers