Class AutoShardDataset

java.lang.Object

org.tensorflow.op.RawOp

org.tensorflow.op.data.experimental.AutoShardDataset

All Implemented Interfaces:

Shaped, Op, Operand<TType>

@Operator(group="data.experimental")
public final class AutoShardDataset
extends RawOp
implements Operand<TType>

Creates a dataset that shards the input dataset. Creates a dataset that shards the input dataset by num_workers, returning a sharded dataset for the index-th worker. This attempts to automatically shard a dataset by examining the Dataset graph and inserting a shard op before the inputs to a reader Dataset (e.g. CSVDataset, TFRecordDataset).

This dataset will throw a NotFound error if we cannot shard the dataset automatically.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	AutoShardDataset.Inputs
static class	AutoShardDataset.Options	Optional attributes for AutoShardDataset

Field Summary

Fields

Modifier and Type	Field	Description
static final String	OP_NAME	The name of this op, as known by TensorFlow core engine

Fields inherited from class RawOp

operation

Constructor Summary

Constructors

Constructor	Description
AutoShardDataset(Operation operation)

Method Summary

Modifier and Type	Method	Description
Output<TType>	asOutput()	Returns the symbolic handle of the tensor.
static AutoShardDataset.Options	autoShardPolicy(Long autoShardPolicy)	Sets the autoShardPolicy option.
static AutoShardDataset	create(Scope scope, Operand<? extends TType> inputDataset, Operand<TInt64> numWorkers, Operand<TInt64> index, List<Class<? extends TType>> outputTypes, List<Shape> outputShapes, AutoShardDataset.Options... options)	Factory method to create a class wrapping a new ExperimentalAutoShardDataset operation.
Output<? extends TType>	handle()	Gets handle.

Methods inherited from class RawOp

equals, hashCode, op, toString

Methods inherited from class Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface Op

env, op

Methods inherited from interface Operand

asTensor, shape, type

Methods inherited from interface Shaped

rank, size

Field Details

OP_NAME

public static final String OP_NAME

The name of this op, as known by TensorFlow core engine

See Also:

• Constant Field Values

Constructor Details

AutoShardDataset

public AutoShardDataset(Operation operation)

Method Details

create

@Endpoint(describeByClass=true)
public static AutoShardDataset create(Scope scope,
 Operand<? extends TType> inputDataset,
 Operand<TInt64> numWorkers,
 Operand<TInt64> index,
 List<Class<? extends TType>> outputTypes,
 List<Shape> outputShapes,
 AutoShardDataset.Options... options)

Factory method to create a class wrapping a new ExperimentalAutoShardDataset operation.

Parameters:

scope - current scope

inputDataset - A variant tensor representing the input dataset.

numWorkers - A scalar representing the number of workers to distribute this dataset across.

index - A scalar representing the index of the current worker out of num_workers.

outputTypes - The value of the outputTypes attribute

outputShapes - The value of the outputShapes attribute

options - carries optional attribute values

Returns:

a new instance of AutoShardDataset

autoShardPolicy

public static AutoShardDataset.Options autoShardPolicy(Long autoShardPolicy)

Sets the autoShardPolicy option.

Parameters:

autoShardPolicy - the autoShardPolicy option

Returns:

this Options instance.

handle

public Output<? extends TType> handle()

Gets handle.

Returns:

handle.

asOutput

public Output<TType> asOutput()

Description copied from interface: Operand

Returns the symbolic handle of the tensor.

Inputs to TensorFlow operations are outputs of another TensorFlow operation. This method is used to obtain a symbolic handle that represents the computation of the input.

Specified by:

asOutput in interface Operand<TType>

See Also:

• OperationBuilder.addInput(Output)