lingvo.compat module

Compatibility layer for TF2 migration.

lingvo.compat._clone_module(m)[source]: Shallow clone of module m.

lingvo.compat.summarize_tf2_status()[source]: Summarize the TF version environment.

class lingvo.compat._CacheDataset(input_dataset, filename)[source]

Bases: UnaryUnchangedStructureDataset

A Dataset that caches elements of its input.

_abc_impl = <_abc._abc_data object>

class lingvo.compat._ShuffleDataset(input_dataset, buffer_size, seed=None, reshuffle_each_iteration=None)[source]

Bases: UnaryUnchangedStructureDataset

A Dataset that randomly shuffles the elements of its input.

_abc_impl = <_abc._abc_data object>

lingvo.compat.stateless_shuffle_dataset(buffer_size, seed=None, reshuffle_each_iteration=None)[source]

Randomly shuffles the elements of the dataset based on a stateless shuffle implementation.

This method returns a stateless ShuffleDataset unconditionally. It can be used with dataset.apply() to obtain a stateless shuffled dataset, which supports the TF1 compatibility API tf.data.make_one_shot_iterator() in TF2. .. rubric:: Example

>>> dataset = tf.data.Dataset.range(3)
>>> dataset = dataset.apply(
...     stateless_shuffle_dataset((3, reshuffle_each_iteration=True))

Parameters

buffer_size – A tf.int64 scalar tf.Tensor, representing the number of elements from this dataset from which the new dataset will sample.
seed – (Optional.) A tf.int64 scalar tf.Tensor, representing the random seed that will be used to create the distribution. See tf.random.set_seed for behavior.
reshuffle_each_iteration – (Optional.) A boolean, which if true indicates that the dataset should be pseudorandomly reshuffled each time it is iterated over. (Defaults to True.)

Returns

A Dataset.

Return type

Dataset

lingvo.compat.stateless_cache_dataset(filename='')[source]

Caches the elements in the dataset based on a stateless cache implementation.

This method returns a stateless CacheDataset unconditionally. It can be used with dataset.apply() to obtain a stateless cached dataset, which supports the TF1 compatibility API tf.data.make_one_shot_iterator() in TF2.

Example

>>> dataset = tf.data.Dataset.range(3)
>>> dataset = dataset.apply(stateless_cache_dataset())

Parameters: filename – A tf.string scalar tf.Tensor, representing the name of a directory on the filesystem to use for caching elements in this Dataset. If a filename is not provided, the dataset will be cached in memory.
Returns: A Dataset.
Return type: Dataset

lingvo.compat.stateless_list_files(file_pattern, shuffle=None, seed=None)[source]

A dataset of all files matching one or more glob patterns.

Note that, if shuffle is not None, it will use a stateless shuffle implementation. Then the returned dataset supports the TF1 compatibility API tf.data.make_one_shot_iterator() in TF2.

Example

>>> dataset = tf.stateless_list_files("some_file_pattern")

Parameters

file_pattern – A string, a list of strings, or a tf.Tensor of string type (scalar or vector), representing the filename glob (i.e. shell wildcard) pattern(s) that will be matched.
shuffle – (Optional.) If True, the file names will be shuffled randomly based on a stateless implementation. Defaults to True.
seed – (Optional.) A tf.int64 scalar tf.Tensor, representing the random seed that will be used to create the distribution. See tf.random.set_seed for behavior.

Returns

A Dataset of strings corresponding to file names.

Return type

Dataset

class lingvo.compat.variable_scope(*args, **kwargs)[source]

Bases: variable_scope

Override tf.compat.v1.variable_scope with an additional error message.