MCPcopy Create free account
hub / github.com/google-research/language / encode_example

Method encode_example

language/serene/preprocessing.py:439–462  ·  view source on GitHub ↗
(self, *, claim, evidence, claim_label, evidence_label,
                     metadata, wikipedia_url, sentence_id)

Source from the content-addressed store, hash-verified

437 self._pad_id = self._tokenizer.vocab['[PAD]']
438
439 def encode_example(self, *, claim, evidence, claim_label, evidence_label,
440 metadata, wikipedia_url, sentence_id):
441 claim_tokens, claim_word_ids, claim_mask, claim_segment_ids = self.encode(
442 claim)
443 evidence = create_evidence_input(
444 wikipedia_url if self._include_title else None,
445 sentence_id if self._include_sentence_id else None,
446 evidence
447 )
448 evidence_tokens, evidence_word_ids, evidence_mask, evidence_segment_ids = self.encode(
449 evidence)
450 return {
451 'claim_text_tokens': claim_tokens,
452 'claim_text_word_ids': claim_word_ids,
453 'claim_text_mask': claim_mask,
454 'claim_text_segment_ids': claim_segment_ids,
455 'evidence_text_tokens': evidence_tokens,
456 'evidence_text_word_ids': evidence_word_ids,
457 'evidence_text_mask': evidence_mask,
458 'evidence_text_segment_ids': evidence_segment_ids,
459 'claim_label': claim_label,
460 'evidence_label': evidence_label,
461 'metadata': metadata
462 }
463
464 def build_encoder_fn(self):
465

Callers

nothing calls this directly

Calls 2

encodeMethod · 0.95
create_evidence_inputFunction · 0.85

Tested by

no test coverage detected