MCPcopy Create free account
hub / github.com/google-research/language / __init__

Method __init__

language/multivec/utils/data_processor.py:54–57  ·  view source on GitHub ↗
(self, max_seq_length, tokenizer)

Source from the content-addressed store, hash-verified

52 """Preprocessor."""
53
54 def __init__(self, max_seq_length, tokenizer):
55 self._tokenizer = tokenizer
56 self._max_seq_length = max_seq_length
57 tf.logging.info("Max sequence length {}".format(self._max_seq_length))
58
59 def create_example(self, key, text):
60 """Create example."""

Callers

nothing calls this directly

Calls

no outgoing calls

Tested by

no test coverage detected