MCPcopy Create free account

hub / github.com/huggingface/transformers / functions

Functions4,119 in github.com/huggingface/transformers

↓ 1 callersFunctioncreate_tf_var
(tensor: np.ndarray, name: str, session: tf.Session)
src/transformers/convert_bert_pytorch_checkpoint_to_original_tf.py:69
↓ 1 callersMethodcreate_token_type_ids_from_sequences
Creates a mask from the two sequences passed to be used in a sequence-pair classification task. A BERT sequence pair mask has the fol
templates/adding_a_new_model/tokenization_xxx.py:199
↓ 1 callersMethodcreate_token_type_ids_from_sequences
(self, token_ids_0: List, token_ids_1: Optional[List] = None)
src/transformers/tokenization_utils.py:804
↓ 1 callersMethodcreate_transfo_xl_lm_head
(self, config, input_ids_1, input_ids_2, lm_labels)
tests/test_modeling_transfo_xl.py:117
↓ 1 callersMethodcreate_transfo_xl_model
(self, config, input_ids_1, input_ids_2, lm_labels)
tests/test_modeling_transfo_xl.py:86
↓ 1 callersMethoddecode
Converts a sequence of ids (integer) in a string, using the tokenizer and vocabulary with options to remove special tokens and clean
src/transformers/tokenization_utils_base.py:2036
↓ 1 callersMethoddetokenize
Detokenize the provided tokens ids to readable text: - **tokens_ids**: List of tokens ids - **skip_special_tokens**: Flag ind
src/transformers/commands/serving.py:180
↓ 1 callersMethoddevice
(self)
src/transformers/training_args.py:195
↓ 1 callersMethoddevice
(self)
src/transformers/benchmark/benchmark_args.py:76
↓ 1 callersMethoddistributed_concat
(self, tensor: torch.Tensor, num_total_examples: int)
src/transformers/trainer.py:817
↓ 1 callersFunctiondocuments_dir_is_valid
(path)
examples/seq2seq/bertabs/run_summarization.py:312
↓ 1 callersFunctiondownload_and_extract
(task, data_dir)
utils/download_glue_data.py:47
↓ 1 callersFunctiondownload_and_unzip
(url, dest_dir)
src/transformers/convert_marian_to_pytorch.py:483
↓ 1 callersFunctiondownload_diagnostic
(data_dir)
utils/download_glue_data.py:104
↓ 1 callersMethodembed_answers
( self, input_ids, attention_mask=None, checkpoint_batch_size=-1, )
src/transformers/modeling_retribert.py:135
↓ 1 callersMethodembed_questions
( self, input_ids, attention_mask=None, checkpoint_batch_size=-1, )
src/transformers/modeling_retribert.py:129
↓ 1 callersFunctionencode_file
( tokenizer, data_path, max_length, pad_to_max_length=True, return_tensors="pt", overw
examples/seq2seq/utils.py:18
↓ 1 callersFunctionencode_for_summarization
Encode the story and summary lines, and join them as specified in [1] by using `[SEP] [CLS]` tokens to separate sentences.
examples/seq2seq/bertabs/utils_summarization.py:130
↓ 1 callersMethodend_epoch
Finally arrived at the end of epoch (full pass on dataset). Do some tensorboard logging and checkpoint saving.
examples/distillation/distiller.py:576
↓ 1 callersMethodenforce_repetition_penalty_
repetition penalty (from CTRL paper https://arxiv.org/abs/1909.05858).
src/transformers/modeling_utils.py:818
↓ 1 callersFunctionentropy
Compute the entropy of a probability distribution
examples/bertology/run_bertology.py:49
↓ 1 callersFunctioneval_qa_s2s_epoch
(model, dataset, tokenizer, args)
examples/longform-qa/eli5_utils.py:437
↓ 1 callersFunctionevaluate
(eval_iter)
examples/contrib/run_transfo_xl.py:100
↓ 1 callersFunctionevaluate
(args)
examples/seq2seq/bertabs/run_summarization.py:31
↓ 1 callersFunctionevaluate_checkpoint
(ckpt_path: Path, dest_dir=None)
examples/seq2seq/distillation.py:398
↓ 1 callersFunctionevaluate_performance
(data_loader, discriminator, device="cpu")
examples/text-generation/pplm/run_pplm_discrim_train.py:161
↓ 1 callersFunctionevaluate_qa_retriever
(model, dataset, tokenizer, args)
examples/longform-qa/eli5_utils.py:283
↓ 1 callersMethodfast_align_python
(self, tokenizer_r, tokenizer_p, tok_case, pretrained_name)
tests/test_tokenization_fast.py:80
↓ 1 callersFunctionfill_mask
(masked_input, model, tokenizer, topk=5)
examples/contrib/run_camembert.py:7
↓ 1 callersFunctionfill_with_neg_inf
FP16-compatible function that fills a input_ids with -inf.
src/transformers/modeling_bart.py:790
↓ 1 callersFunctionfind_all_best_thresh
(main_eval, preds, exact_raw, f1_raw, na_probs, qid_to_has_ans)
src/transformers/data/metrics/squad_metrics.py:201
↓ 1 callersFunctionfind_all_best_thresh_v2
(main_eval, preds, exact_raw, f1_raw, na_probs, qid_to_has_ans)
src/transformers/data/metrics/squad_metrics.py:167
↓ 1 callersFunctionfind_all_links
(file_paths)
utils/link_tester.py:37
↓ 1 callersFunctionfind_best_thresh
(preds, scores, na_probs, qid_to_has_ans)
src/transformers/data/metrics/squad_metrics.py:178
↓ 1 callersFunctionfind_best_thresh_v2
(preds, scores, na_probs, qid_to_has_ans)
src/transformers/data/metrics/squad_metrics.py:133
↓ 1 callersFunctionfind_model_file
(dest_dir)
src/transformers/convert_marian_to_pytorch.py:91
↓ 1 callersFunctionfind_nearest_training
(question, n_results=10)
examples/longform-qa/eli5_app.py:81
↓ 1 callersFunctionfind_vocab_file
(model_dir)
src/transformers/convert_marian_to_pytorch.py:287
↓ 1 callersFunctionflatten_list
(summary_ids: List[List])
examples/seq2seq/utils.py:194
↓ 1 callersMethodformat_dict
(d)
src/transformers/commands/env.py:57
↓ 1 callersFunctionformat_log
(loss, split)
examples/contrib/run_transfo_xl.py:128
↓ 1 callersFunctionformat_mrpc
(data_dir, path_to_data)
utils/download_glue_data.py:57
↓ 1 callersFunctionformat_rouge_scores
(scores)
examples/seq2seq/bertabs/run_summarization.py:145
↓ 1 callersFunctionformat_summary
Transforms the output of the `from_batch` function into nicely formatted summaries.
examples/seq2seq/bertabs/run_summarization.py:126
↓ 1 callersMethodforward
(self, features, **kwargs)
src/transformers/modeling_roberta.py:268
↓ 1 callersMethodfreeze_embeds
Freeze token embeddings and positional embeddings for bart, just token embeddings for t5.
examples/seq2seq/finetune.py:99
↓ 1 callersFunctionfreeze_pos_embeddings
(student, args)
examples/distillation/train.py:85
↓ 1 callersFunctionfreeze_token_type_embeddings
(student, args)
examples/distillation/train.py:92
↓ 1 callersMethodfrom_dict
Constructs a `Config` from a Python dictionary of parameters. Args: config_dict (:obj:`Dict[str, any]`):
src/transformers/configuration_utils.py:269
↓ 1 callersMethodfrom_str
( format: str, output_path: Optional[str], input_path: Optional[str], column: Optional[str], overwrite
src/transformers/pipelines.py:212
↓ 1 callersFunctionfull_text_generation
( model, tokenizer, context=None, num_samples=1, device="cuda", bag_of_words=None,
examples/text-generation/pplm/run_pplm.py:317
↓ 1 callersFunctiongelu_new
Implementation of the gelu activation function currently in Google Bert repo (identical to OpenAI GPT). Also see https://arxiv.org/abs/1606.0
src/transformers/activations.py:25
↓ 1 callersFunctiongenerate_text_pplm
( model, tokenizer, context=None, past=None, device="cuda", perturb=True, bow_indi
examples/text-generation/pplm/run_pplm.py:417
↓ 1 callersMethodget_added_vocab
(self)
src/transformers/tokenization_utils.py:178
↓ 1 callersFunctionget_cached_data_loader
(dataset, batch_size, discriminator, shuffle=False, device="cpu")
examples/text-generation/pplm/run_pplm_discrim_train.py:199
↓ 1 callersFunctionget_checkpoint_callback
Saves the best model by validation ROUGE2 score.
examples/seq2seq/callbacks.py:74
↓ 1 callersFunctionget_classifier
( name: Optional[str], class_label: Union[str, int], device: str )
examples/text-generation/pplm/run_pplm.py:248
↓ 1 callersMethodget_classifier
(self)
examples/text-generation/pplm/run_pplm_discrim_train.py:57
↓ 1 callersMethodget_config
(self)
src/transformers/modeling_tf_utils.py:1636
↓ 1 callersMethodget_config
(self)
src/transformers/optimization_tf.py:51
↓ 1 callersFunctionget_constant_schedule
Create a schedule with a constant learning rate.
src/transformers/optimization.py:28
↓ 1 callersFunctionget_constant_schedule_with_warmup
Create a schedule with a constant learning rate preceded by a warmup period during which the learning rate increases linearly between 0 and 1.
src/transformers/optimization.py:34
↓ 1 callersFunctionget_cosine_schedule_with_warmup
Create a schedule with a learning rate that decreases following the values of the cosine function between 0 and `pi * cycles` after a warmup
src/transformers/optimization.py:62
↓ 1 callersFunctionget_cosine_with_hard_restarts_schedule_with_warmup
Create a schedule with a learning rate that decreases following the values of the cosine function with several hard restarts, after a warmup
src/transformers/optimization.py:77
↓ 1 callersFunctionget_dataset
(args: DataTrainingArguments, tokenizer: PreTrainedTokenizer, evaluate=False)
examples/language-modeling/run_language_modeling.py:118
↓ 1 callersMethodget_dataset
(self, type_path)
examples/seq2seq/finetune.py:180
↓ 1 callersMethodget_dataset
(self)
examples/multiple-choice/utils_multiple_choice.py:206
↓ 1 callersMethodget_dataset
(self)
examples/token-classification/utils_ner.py:223
↓ 1 callersMethodget_dev_examples
See base class.
src/transformers/data/processors/glue.py:407
↓ 1 callersMethodget_eval_dataloader
(self, eval_dataset: Optional[Dataset] = None)
src/transformers/trainer.py:234
↓ 1 callersMethodget_eval_tfdataset
(self, eval_dataset: Optional[tf.data.Dataset] = None)
src/transformers/trainer_tf.py:103
↓ 1 callersMethodget_example_from_tensor_dict
See base class.
src/transformers/data/processors/glue.py:394
↓ 1 callersMethodget_extended_attention_mask
(self, attention_mask, input_shape)
src/transformers/modeling_tf_electra.py:173
↓ 1 callersMethodget_fixlen_iter
(self, start=0)
src/transformers/tokenization_transfo_xl.py:534
↓ 1 callersFunctionget_from_cache
Given a URL, look for the corresponding file in the local cache. If it's not there, download it. Then return the path to the cached file.
src/transformers/file_utils.py:650
↓ 1 callersMethodget_head_mask
(self, head_mask)
src/transformers/modeling_tf_electra.py:195
↓ 1 callersFunctionget_image_transforms
()
examples/contrib/mm-imdb/utils_mmimdb.py:135
↓ 1 callersMethodget_input_embeddings
Returns the model's input embeddings. Returns: :obj:`tf.keras.layers.Layer`: A torch module mapping voca
src/transformers/modeling_tf_utils.py:188
↓ 1 callersMethodget_input_output_texts
(self, tokenizer)
tests/test_tokenization_common.py:70
↓ 1 callersMethodget_input_output_texts
(self, tokenizer)
tests/test_tokenization_bert_japanese.py:63
↓ 1 callersMethodget_iterator
(self, split, *args, **kwargs)
src/transformers/tokenization_transfo_xl.py:749
↓ 1 callersMethodget_label_frequencies
(self)
examples/contrib/mm-imdb/utils_mmimdb.py:81
↓ 1 callersMethodget_labels
(self)
src/transformers/data/datasets/glue.py:145
↓ 1 callersMethodget_labels
Gets the list of labels for this data set.
src/transformers/data/processors/utils.py:106
↓ 1 callersMethodget_labels
See base class.
src/transformers/data/processors/glue.py:415
↓ 1 callersFunctionget_masks
Generate hidden states mask, and optionally an attention mask.
src/transformers/modeling_tf_xlm.py:85
↓ 1 callersMethodget_optimizers
Setup the optimizer and the learning rate scheduler. We provide a reasonable default that works well. If you want to use som
src/transformers/trainer_tf.py:121
↓ 1 callersMethodget_optimizers
Setup the optimizer and the learning rate scheduler. We provide a reasonable default that works well. If you want to use som
src/transformers/trainer.py:280
↓ 1 callersMethodget_output_embeddings
Returns the model's output embeddings. Returns: :obj:`tf.keras.layers.Layer`: A torch module mapping hid
src/transformers/modeling_tf_utils.py:216
↓ 1 callersMethodget_output_embeddings
(self)
src/transformers/modeling_encoder_decoder.py:87
↓ 1 callersMethodget_output_embeddings
(self)
src/transformers/modeling_mobilebert.py:872
↓ 1 callersMethodget_output_embeddings
(self)
src/transformers/modeling_mobilebert.py:993
↓ 1 callersMethodget_output_embeddings
(self)
src/transformers/modeling_tf_t5.py:1034
↓ 1 callersFunctionget_pairs
Return set of symbol pairs in a word. word is represented as tuple of symbols (symbols being variable-length strings)
src/transformers/tokenization_openai.py:47
↓ 1 callersFunctionget_pairs
Return set of symbol pairs in a word. Word is represented as tuple of symbols (symbols being variable-length strings).
src/transformers/tokenization_gpt2.py:90
↓ 1 callersFunctionget_pairs
Return set of symbol pairs in a word. Word is represented as tuple of symbols (symbols being variable-length strings).
src/transformers/tokenization_ctrl.py:102
↓ 1 callersFunctionget_pairs
Return set of symbol pairs in a word. word is represented as tuple of symbols (symbols being variable-length strings)
src/transformers/tokenization_xlm.py:430
↓ 1 callersMethodget_rust_tokenizer
(self, **kwargs)
tests/test_tokenization_gpt2.py:73
↓ 1 callersMethodget_rust_tokenizer
(self, **kwargs)
tests/test_tokenization_bert.py:64
↓ 1 callersMethodget_sent_stream
(self)
src/transformers/tokenization_transfo_xl.py:568
← previousnext →1,001–1,100 of 4,119, ranked by callers