Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/google-research/language
/ functions
Functions
6,154 in github.com/google-research/language
⨍
Functions
6,154
◇
Types & classes
800
↳
Endpoints
12
↓ 2 callers
Function
_read_numpy_item
Read the item found in a numpy save file. Args: input_file: Filename string or FileLike object. Returns: The first item.
language/nql/nql/io.py:66
↓ 2 callers
Method
_reformat_bert_inputs
Reformats BERT input features.
language/realm/featurization.py:151
↓ 2 callers
Method
_relations_roundtrip
Test full io pipeline on serializing relations. Very basic test that creates two sparse tensors, serializes them, deserializes them, then com
language/nql/nql/io_test.py:62
↓ 2 callers
Method
_remove_table_joins
(self, lir_partial, regex)
language/compir/dataset_parsers/sql_parser.py:36
↓ 2 callers
Function
_resolve_named_dims
Resolves the name of any named dimensions and returns the scalar value.
language/canine/tensor_contracts.py:850
↓ 2 callers
Function
_rhs_count
Returns count of occurances of rhs_b in rhs_a.
language/compgen/csl/induction/unification_utils.py:59
↓ 2 callers
Function
_run_model_with_strategy
(strategy, config, dataset_fn)
language/compgen/csl/model/training/training_utils_test.py:49
↓ 2 callers
Function
_run_model_with_strategy
(strategy, config, bert_config, dataset_fn)
language/compgen/nqg/model/parser/training/training_utils_test.py:53
↓ 2 callers
Method
_run_on_features
Run predictions for given features.
language/labs/drkit/bert_utils.py:219
↓ 2 callers
Method
_run_on_features
Run predictions for given features.
language/labs/drkit/bert_utils_v2.py:214
↓ 2 callers
Function
_safe_add
(a: tf.Tensor, b: tf.Tensor)
language/canine/local_attention.py:30
↓ 2 callers
Function
_score_seeds
Helper to score a single bootstrap sample.
language/multiberts/multibootstrap.py:110
↓ 2 callers
Function
_shifted_enumerate
Yields (index, item) pairs starting from start_idx. This function yields the same elements as `enumerate`, but starting from the specified start_
language/compgen/nqg/tasks/mcd_utils.py:146
↓ 2 callers
Method
_shutdown_queue
Shuts down the multiprocessing Queue.
language/realm/parallel.py:108
↓ 2 callers
Method
_stack
(self, d)
language/mentionmemory/tasks/memory_generation_task_test.py:166
↓ 2 callers
Function
_string_feature
(value)
language/question_answering/decatt_docreader/preprocessing/create_nq_short_pipeline_examples.py:47
↓ 2 callers
Function
_string_list_feature
(values)
language/question_answering/decatt_docreader/preprocessing/create_nq_long_examples.py:51
↓ 2 callers
Function
_strip_spaces
(text)
language/labs/drkit/run_dualencoder_qa.py:985
↓ 2 callers
Function
_strip_spaces
(text)
language/labs/drkit/run_dualencoder_lsf.py:1917
↓ 2 callers
Function
_strip_spaces
(text)
language/labs/drkit/hotpotqa/answer_extractor.py:1077
↓ 2 callers
Function
_strip_spaces
(text)
language/bert_extraction/steal_bert_qa/models/run_squad.py:968
↓ 2 callers
Function
_strip_spaces
(text)
language/conpono/evals/run_squad.py:955
↓ 2 callers
Function
_swap_nt_order
(rhs, old_to_new)
language/compgen/csl/induction/rule_utils.py:20
↓ 2 callers
Function
_swap_nt_order
(rhs)
language/compgen/nqg/model/induction/rule_utils.py:20
↓ 2 callers
Method
_symbol_table_dict_roundtrip
Test full io pipeline on serializing relations. Very basic test that creates two symbol tables, serializes them, deserializes them, then comp
language/nql/nql/io_test.py:95
↓ 2 callers
Function
_table_reader
Yields tables from the table file. Tables are parsed into a list of tuples with tokenized entries. Args: table_file: String filename.
language/totto/totto_parent_eval.py:138
↓ 2 callers
Function
_target_codelength
Return codelength for encoding `target` given `source` and `rules`. The codelength of the target is computed as -log_2(P(y|x)). For P(y|x) we us
language/compgen/nqg/model/induction/codelength_utils.py:116
↓ 2 callers
Function
_text
Lowercase and tokenize text.
language/table_text_eval/webnlg_correlations.py:64
↓ 2 callers
Function
_text_reader
Yields lines from the text file. Performs lowercasing and white-space tokenization on each line before returning. Args: text_file: String
language/table_text_eval/table_text_eval.py:78
↓ 2 callers
Method
_to_orig_relation
Converts a relation back to its original name.
language/compir/dataset_parsers/cfq_parser.py:211
↓ 2 callers
Function
_to_tensor
Converts value to a tensor if possible; otherwise, returns the original.
language/canine/tensor_contracts.py:944
↓ 2 callers
Function
_tokenize
(x)
language/table_text_eval/webnlg_correlations.py:58
↓ 2 callers
Function
_update_state
Sparsely update the state for rules that may be affected.
language/compgen/nqg/model/induction/induction_utils.py:157
↓ 2 callers
Function
_write_numpy_item
Write the item to a numpy save file. Args: output_file: Filename string or FileLike object. item: Some object to save as the first item.
language/nql/nql/io.py:83
↓ 2 callers
Method
accept_neighbor
Returns a new state after accepting the neighbor. The default behavior is to return the old state unchanged. Args: neighbor_ex: Neighb
language/casper/retrieve/query_retrievers.py:75
↓ 2 callers
Function
add_beam_dim
Creates new beam dimension in non-scalar array and tiles into it.
language/gscan/xattn_model/model/decode.py:50
↓ 2 callers
Function
add_int64_feature
(key, values, example)
language/multivec/utils/data_processor.py:47
↓ 2 callers
Function
add_mean_metric
Add the mean of the values Tensor as a metric.
language/realm/model.py:245
↓ 2 callers
Method
add_memories
Save generated memories in-memory storage.
language/mentionmemory/tasks/memory_generation_task.py:266
↓ 2 callers
Function
add_one_fact
(kb, subj, rel, obj)
language/labs/drkit/metaqa/preprocessing/metaqa_preprocess.py:37
↓ 2 callers
Function
add_one_fact
(kb, subj, rel, obj)
language/emql/preprocess/metaqa_preprocess.py:37
↓ 2 callers
Method
add_sentence
(self, sentence)
language/gscan/xattn_model/dataset/preprocess.py:103
↓ 2 callers
Method
add_set
Add a set of elements to the sketch. Args: sketch: sketch to add xs to xs: a set of name or id in string
language/emql/cm_sketch.py:126
↓ 2 callers
Function
ag_gnmt_bahdanau_att
Hparams for LSTM with bahdanau attention.
language/labs/consistent_zero_shot_nmt/models/agreement.py:639
↓ 2 callers
Function
ag_gnmt_luong_att
Hparams for LSTM with luong attention.
language/labs/consistent_zero_shot_nmt/models/agreement.py:655
↓ 2 callers
Function
ag_nmt_bilstm_bahdanau_att
Hparams for LSTM with bahdanau attention.
language/labs/consistent_zero_shot_nmt/models/agreement.py:572
↓ 2 callers
Function
ag_nmt_bilstm_luong_att
Hparams for LSTM with luong attention.
language/labs/consistent_zero_shot_nmt/models/agreement.py:588
↓ 2 callers
Function
all_compare
(xs: Array, ys: Array)
language/mentionmemory/utils/mention_utils.py:24
↓ 2 callers
Method
ancestor_types
Find all types frame x belongs to (explicitly or implicitly).
language/templama/sling2facts.py:283
↓ 2 callers
Method
answer_match
Compute whether the query matches the passage.
language/multivec/preprocessing/create_training_data.py:765
↓ 2 callers
Function
apply_no_ans_threshold
(scores, na_probs, qid_to_has_ans, na_prob_thresh)
language/bert_extraction/steal_bert_qa/utils/evaluate_squad_2.py:118
↓ 2 callers
Function
aprf
Returns precision, recall and F of the given statistics.
language/labs/drkit/evaluate.py:483
↓ 2 callers
Method
are_sentences_duplicate
( self, paired_sentences: List[Tuple[str, str]] )
language/diffqg/metrics.py:94
↓ 2 callers
Method
as_triples
Print qualified triples for every slot filler.
language/templama/sling2facts.py:443
↓ 2 callers
Method
assertNumpyArraysEqual
(self, actual, expected)
language/mentionmemory/tasks/ultra_fine_entity_typing_task_test.py:47
↓ 2 callers
Function
base_nmt_bilstm
Base hparams for LSTM-based NMT models with bidirectional encoders.
language/labs/consistent_zero_shot_nmt/models/basic.py:263
↓ 2 callers
Function
base_nmt_gnmt_lstm
Base hparams for LSTM-based GNMT models.
language/labs/consistent_zero_shot_nmt/models/basic.py:314
↓ 2 callers
Function
batch_namedtuples
Combines a batch of namedtuples into a single namedtuple of the same type. Args: namedtuples: a list of namedtuples, all of the same type. R
language/realm/featurization.py:649
↓ 2 callers
Function
bounded_rr
(y_true, k)
language/multivec/models/metrics.py:50
↓ 2 callers
Function
brevity_penalty
Brevity penalty function for beam search penalizing short sequences. Args: alpha: float: brevity-penalty scaling parameter. length: int: le
language/gscan/xattn_model/model/decode.py:34
↓ 2 callers
Method
build_estimator
Produce an Estimator for this Model. Args: model_dir: passed in to Estimator - location of tmp files used by Estimator to checkpoin
language/nql/nql/util.py:218
↓ 2 callers
Method
build_input_fn
Construct tf.Dataset object as input. Args: name: name of training task batch_size: batch_size mode: 'train' or 'test' ep
language/emql/data_loader.py:571
↓ 2 callers
Function
byte_slice
(text, start, end, errors="replace")
language/canine/tydiqa/data.py:125
↓ 2 callers
Function
byte_str
(text)
language/canine/tydiqa/data.py:115
↓ 2 callers
Function
bytes_feature
(values)
language/realm/generate_retrieval_corpus.py:97
↓ 2 callers
Function
candidates_iter
Yield's the candidates that should not be skipped in an example.
language/question_answering/bert_joint/run_nq.py:339
↓ 2 callers
Method
check
(self, context: _Context, args_dict: Dict[Text, Any], named_dims: Dict[Text, Union[int, tf.Tensor]
language/canine/tensor_contracts.py:323
↓ 2 callers
Method
check_batch
(self, m, i, typename)
language/nql/nql/util_test.py:198
↓ 2 callers
Method
check_neighbor
Returns whether the neighbor is usable. Optionally, a state can be maintained (e.g., to count how many neighbors so far belong to each intent
language/casper/retrieve/query_retrievers.py:57
↓ 2 callers
Method
check_neighbor
Returns whether the neighbor is usable.
language/casper/retrieve/query_retrievers.py:117
↓ 2 callers
Method
check_neighbor
(self, input_ex: Example, neighbor_ex: Example, state: Any)
language/casper/retrieve/query_retrievers.py:165
↓ 2 callers
Method
close
(self)
language/labs/drkit/preprocessing/preprocess_qa.py:235
↓ 2 callers
Function
compute_assignments
Assigns observations to cluster centroids. Computes l2 distance between each pair of observation and centroids, and assigns observation to closes
language/mentionmemory/modules/kmeans.py:38
↓ 2 callers
Function
compute_bow_reconstruction_loss
Creates loss for BOW reconstruction from hidden states. Args: hidden: (tf.float32) batch_size x dim1 word_emb: (tf.float32) vocab_size x di
language/labs/drkit/run_dualencoder_lsf.py:1390
↓ 2 callers
Function
compute_divergence
Compute compound divergence using Chernoff coefficient.
language/compgen/nqg/tasks/mcd_utils.py:24
↓ 2 callers
Function
compute_f1
(precision, recall)
language/xsp/evaluation/official_evaluation.py:152
↓ 2 callers
Function
compute_label_loss
(logits, labels)
language/question_answering/bert_joint/run_nq.py:962
↓ 2 callers
Method
compute_logits
Compute logits between query embedding and candidate embeddings. Args: candidates: candidate embeddings query: query embeddings
language/emql/model.py:867
↓ 2 callers
Function
compute_loss
(logits, positions)
language/labs/drkit/run_dualencoder_qa.py:651
↓ 2 callers
Function
compute_loss
Cross-entropy loss when multiple labels per example.
language/labs/drkit/model_fns.py:567
↓ 2 callers
Function
compute_loss
(logits, positions)
language/bert_extraction/steal_bert_qa/models/run_squad.py:658
↓ 2 callers
Function
compute_loss
(logits, positions)
language/question_answering/bert_joint/run_nq.py:953
↓ 2 callers
Function
compute_positions_shift_with_entity_tokens
Computes the new position for every position in the old sequence.
language/mentionmemory/utils/mention_preprocess_utils.py:526
↓ 2 callers
Method
compute_query_embs_and_sketch
Compute embeddings and sketch with logits of entities. Given entity_ids and logits in the same order, compute weighted average of entity embe
language/emql/model.py:764
↓ 2 callers
Function
compute_scores
Read predictions and ground truth and return P, R, F.
language/labs/drkit/evaluate.py:418
↓ 2 callers
Function
compute_set_f1
Computes F1 of items given two sets of items.
language/xsp/evaluation/official_evaluation.py:159
↓ 2 callers
Function
compute_topk_scores_and_seq
Given sequences and scores, will gather the top k=beam size sequences. This function is used to grow alive, and finished. It takes sequences, sco
language/xsp/model/beam_search.py:78
↓ 2 callers
Function
compute_transformer
Multi-headed, multi-layer Transformer.
language/capwap/utils/transformer_utils.py:475
↓ 2 callers
Method
config_model_evaluation
Add additional attributes to the model which allow evaluation. This should also set model.evaluations to an appropriate dictionary. Args:
language/nql/nql/util.py:302
↓ 2 callers
Method
config_model_prediction
Add additional attributes to model which make predictions. The model's inputs will be tf.Placeholders named in the feature_ph_dict dictionary
language/nql/nql/util.py:272
↓ 2 callers
Method
config_model_training
Add additional attributes to the model which allow training. These should include at least model.loss and model.train_op. For use with Estim
language/nql/nql/util.py:288
↓ 2 callers
Function
convert_examples_to_features
Loads a data file into a list of `InputBatch`s.
language/labs/drkit/hotpotqa/answer_extractor.py:387
↓ 2 callers
Function
convert_examples_to_features
Loads a data file into a list of `InputBatch`s.
language/bert_extraction/steal_bert_qa/models/run_squad.py:320
↓ 2 callers
Function
convert_examples_to_features
Loads a data file into a list of `InputBatch`s.
language/conpono/evals/run_squad.py:308
↓ 2 callers
Function
convert_instance_to_tf_example
Convert instance object to TFExample. Args: tokenizer: instance of language.bert.tokenization.Tokenizer instance: TrainingInstance obje
language/conpono/cpc/preproc/preprocessing_utils.py:52
↓ 2 callers
Function
convert_instance_to_tf_example
Convert instance object to TFExample. Args: tokenizer: instance of language.bert.tokenization.Tokenizer instance: TrainingInstance obje
language/conpono/create_pretrain_data/preprocessing_utils.py:47
↓ 2 callers
Function
convert_single_example
Converts a single `InputExample` into a single `InputFeatures`.
language/bert_extraction/steal_bert_classifier/models/run_classifier_membership.py:324
↓ 2 callers
Function
convert_single_example
Converts a single `InputExample` into a single `InputFeatures`.
language/bert_extraction/steal_bert_classifier/models/run_classifier.py:571
↓ 2 callers
Function
convert_single_example
Converts a single `InputExample` into a single `InputFeatures`.
language/bert_extraction/steal_bert_qa/utils/run_bert_boolq_diff.py:185
↓ 2 callers
Function
convert_single_example
Converts a single `InputExample` into a single `InputFeatures`.
language/bert_extraction/steal_bert_qa/models/run_bert_boolq_distill.py:222
← previous
next →
801–900 of 6,154, ranked by callers