lingvo.core.program_utils module

Utils function for program.py.

lingvo.core.program_utils.SummaryToCsv(summaries)

Convert summary (Dict[str, tf.Summary]) to csv format.

lingvo.core.program_utils.CsvToSummary(csv)

Convert csv format to summary (Dict[str, tf.Summary]).

class lingvo.core.program_utils.DecodeStatusCache(program_dir)

Bases: object

Maintain status file to keep decoding datasets status.

Status file should have following format: - 1st line is checkpoint key, e.g. ckpt-123 - the rest lines are dataset names that has been decoded. Here’s an example: ckpt-123 Dev Test

UpdateCkpt(ckpt_key)

Update checkpoint key in the status.

UpdateDataset(dataset_name, summaries)

Update decoded dataset in the status.

TryLoadCache(ckpt_key, dataset_name)

Try load summary cache for ckpt_key, dataset_name.

Parameters

• ckpt_key – str, checkpoint key, e.g. ckpt-123

• dataset_name – str, the dataset name, e.g. Test

Returns

summaries if load successful, otherwise, return None

class lingvo.core.program_utils.TriggerScheduler(offset, interval)

Bases: object

A trigger scheduler with offset, and interval.

Maintains an counter, incremented when Trigger() called. ShouldRun() only returns True when (counter - offset) % interval == 0.

Trigger()

ShouldRun()