MCPcopy Create free account
hub / github.com/google-research/language / input_function

Function input_function

language/labs/memory/run_models.py:239–274  ·  view source on GitHub ↗

Pulls in pattern dataset. Args: is_train: A boolean indicating whether we are constructing the dataset for the train set or the eval set. Returns: dataset: A tf.data.Dataset object containing the features and labels. Raises: NameError: If FLAGS.task_name is not "pattern" or

(is_train)

Source from the content-addressed store, hash-verified

237
238
239def input_function(is_train):
240 """Pulls in pattern dataset.
241
242 Args:
243 is_train: A boolean indicating whether we are constructing the dataset for
244 the train set or the eval set.
245
246 Returns:
247 dataset: A tf.data.Dataset object containing the features and labels.
248
249 Raises:
250 NameError: If FLAGS.task_name is not "pattern" or "symbolic".
251 """
252 if FLAGS.task_name == "pattern":
253 dataset = synthetic_dataset.get_pattern_dataset(
254 FLAGS.num_examples,
255 FLAGS.num_sets_per_sequence,
256 FLAGS.pattern_size,
257 selective=FLAGS.selective_task,
258 num_patterns_store=FLAGS.num_patterns_store)
259 elif FLAGS.task_name == "symbolic":
260 dataset = synthetic_dataset.get_symbolic_dataset(
261 is_train,
262 FLAGS.num_examples,
263 FLAGS.num_sets_per_sequence)
264 else:
265 raise NameError("Task %s not found" % FLAGS.task_name)
266
267 dataset = (
268 dataset.repeat(FLAGS.num_epochs).shuffle(buffer_size=1000)
269 .batch(FLAGS.batch_size, drop_remainder=True))
270
271 # Estimator expects a tuple.
272 dataset = dataset.map(lambda d: (d, d["targets"]))
273
274 return dataset
275
276
277def experiment_function(run_config, hparams):

Callers

nothing calls this directly

Calls

no outgoing calls

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…