Enum DataService.ProcessingModeDef.ShardingPolicy

java.lang.Object

java.lang.Enum<DataService.ProcessingModeDef.ShardingPolicy>

org.tensorflow.proto.data.DataService.ProcessingModeDef.ShardingPolicy

All Implemented Interfaces:

Internal.EnumLite, ProtocolMessageEnum, Serializable, Comparable<DataService.ProcessingModeDef.ShardingPolicy>

Enclosing class:

DataService.ProcessingModeDef

public static enum DataService.ProcessingModeDef.ShardingPolicy
extends Enum<DataService.ProcessingModeDef.ShardingPolicy>
implements ProtocolMessageEnum

Specifies how data is sharded among tf.data service workers.

Protobuf enum tensorflow.data.ProcessingModeDef.ShardingPolicy

Enum Constant Summary

Enum Constants

Enum Constant	Description
DATA	Shards by elements produced by the dataset.
DYNAMIC	The input dataset is dynamically split among workers at runtime.
FILE	The following are static sharding policies.
FILE_OR_DATA	Attempts FILE-based sharding, falling back to DATA-based sharding on failures.
HINT	Looks for the presence of `shard(SHARD_HINT, ...)` which is treated as a placeholder to replace with `shard(num_workers, worker_index)`.
OFF	No sharding will be performed.
UNRECOGNIZED

Field Summary

Fields

Modifier and Type	Field	Description
static final int	DATA_VALUE	Shards by elements produced by the dataset.
static final int	DYNAMIC_VALUE	The input dataset is dynamically split among workers at runtime.
static final int	FILE_OR_DATA_VALUE	Attempts FILE-based sharding, falling back to DATA-based sharding on failures.
static final int	FILE_VALUE	The following are static sharding policies.
static final int	HINT_VALUE	Looks for the presence of `shard(SHARD_HINT, ...)` which is treated as a placeholder to replace with `shard(num_workers, worker_index)`.
static final int	OFF_VALUE	No sharding will be performed.

Method Summary

Modifier and Type	Method	Description
static DataService.ProcessingModeDef.ShardingPolicy	forNumber(int value)
static final Descriptors.EnumDescriptor	getDescriptor()
final Descriptors.EnumDescriptor	getDescriptorForType()
final int	getNumber()
final Descriptors.EnumValueDescriptor	getValueDescriptor()
static Internal.EnumLiteMap<DataService.ProcessingModeDef.ShardingPolicy>	internalGetValueMap()
static DataService.ProcessingModeDef.ShardingPolicy	valueOf(int value)	Deprecated.
static DataService.ProcessingModeDef.ShardingPolicy	valueOf(Descriptors.EnumValueDescriptor desc)	Returns the enum constant of this type with the specified name.
static DataService.ProcessingModeDef.ShardingPolicy	valueOf(String name)	Returns the enum constant of this type with the specified name.
static DataService.ProcessingModeDef.ShardingPolicy[]	values()	Returns an array containing the constants of this enum type, in the order they are declared.

Methods inherited from class Enum

clone, compareTo, equals, finalize, getDeclaringClass, hashCode, name, ordinal, toString, valueOf

Methods inherited from class Object

getClass, notify, notifyAll, wait, wait, wait

Enum Constant Details

OFF

public static final DataService.ProcessingModeDef.ShardingPolicy OFF

No sharding will be performed. Each worker produces the entire dataset
without any sharding. With this mode, the best practice is to shuffle the
dataset nondeterministically so that workers process the dataset in
different orders.

OFF = 0;

DYNAMIC

public static final DataService.ProcessingModeDef.ShardingPolicy DYNAMIC

The input dataset is dynamically split among workers at runtime. Each
worker gets the next split when it reads data from the dispatcher. There
is no fixed sharding with this mode.

DYNAMIC = 1;

FILE

public static final DataService.ProcessingModeDef.ShardingPolicy FILE

The following are static sharding policies. The semantics are similar to
`tf.data.experimental.AutoShardPolicy`. These policies require:
* The tf.data service cluster has a fixed size, and you need to specify
  the workers in DispatcherConfig.
* Each client only reads from the local tf.data service worker.
Shards by input files (each worker will get a set of files to process).
When this option is selected, make sure that there is at least as many
files as workers. If there are fewer input files than workers, a runtime
error will be raised.

FILE = 2;

DATA

public static final DataService.ProcessingModeDef.ShardingPolicy DATA

Shards by elements produced by the dataset. Each worker will process the
whole dataset and discard the portion that is not for itself. Note that
for this mode to correctly partitions the dataset elements, the dataset
needs to produce elements in a deterministic order.

DATA = 3;

FILE_OR_DATA

public static final DataService.ProcessingModeDef.ShardingPolicy FILE_OR_DATA

Attempts FILE-based sharding, falling back to DATA-based sharding on
failures.

FILE_OR_DATA = 4;

HINT

public static final DataService.ProcessingModeDef.ShardingPolicy HINT

Looks for the presence of `shard(SHARD_HINT, ...)` which is treated as a
placeholder to replace with `shard(num_workers, worker_index)`.

HINT = 5;

UNRECOGNIZED

public static final DataService.ProcessingModeDef.ShardingPolicy UNRECOGNIZED

Field Details

OFF_VALUE

public static final int OFF_VALUE

No sharding will be performed. Each worker produces the entire dataset
without any sharding. With this mode, the best practice is to shuffle the
dataset nondeterministically so that workers process the dataset in
different orders.

OFF = 0;

See Also:

• Constant Field Values

DYNAMIC_VALUE

public static final int DYNAMIC_VALUE

The input dataset is dynamically split among workers at runtime. Each
worker gets the next split when it reads data from the dispatcher. There
is no fixed sharding with this mode.

DYNAMIC = 1;

See Also:

• Constant Field Values

FILE_VALUE

public static final int FILE_VALUE

The following are static sharding policies. The semantics are similar to
`tf.data.experimental.AutoShardPolicy`. These policies require:
* The tf.data service cluster has a fixed size, and you need to specify
  the workers in DispatcherConfig.
* Each client only reads from the local tf.data service worker.
Shards by input files (each worker will get a set of files to process).
When this option is selected, make sure that there is at least as many
files as workers. If there are fewer input files than workers, a runtime
error will be raised.

FILE = 2;

See Also:

• Constant Field Values

DATA_VALUE

public static final int DATA_VALUE

Shards by elements produced by the dataset. Each worker will process the
whole dataset and discard the portion that is not for itself. Note that
for this mode to correctly partitions the dataset elements, the dataset
needs to produce elements in a deterministic order.

DATA = 3;

See Also:

• Constant Field Values

FILE_OR_DATA_VALUE

public static final int FILE_OR_DATA_VALUE

Attempts FILE-based sharding, falling back to DATA-based sharding on
failures.

FILE_OR_DATA = 4;

See Also:

• Constant Field Values

HINT_VALUE

public static final int HINT_VALUE

Looks for the presence of `shard(SHARD_HINT, ...)` which is treated as a
placeholder to replace with `shard(num_workers, worker_index)`.

HINT = 5;

See Also:

• Constant Field Values

Method Details

values

public static DataService.ProcessingModeDef.ShardingPolicy[] values()

Returns an array containing the constants of this enum type, in the order they are declared.

Returns:

an array containing the constants of this enum type, in the order they are declared

valueOf

public static DataService.ProcessingModeDef.ShardingPolicy valueOf(String name)

Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)

Parameters:

name - the name of the enum constant to be returned.

Returns:

the enum constant with the specified name

Throws:

IllegalArgumentException - if this enum type has no constant with the specified name

NullPointerException - if the argument is null

getNumber

public final int getNumber()

Specified by:

getNumber in interface Internal.EnumLite

Specified by:

getNumber in interface ProtocolMessageEnum

valueOf

@Deprecated
public static DataService.ProcessingModeDef.ShardingPolicy valueOf(int value)

Deprecated.

Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)

Parameters:

value - the name of the enum constant to be returned.

Returns:

the enum constant with the specified name

Throws:

IllegalArgumentException - if this enum type has no constant with the specified name

NullPointerException - if the argument is null

forNumber

public static DataService.ProcessingModeDef.ShardingPolicy forNumber(int value)

Parameters:

value - The numeric wire value of the corresponding enum entry.

Returns:

The enum associated with the given numeric wire value.

internalGetValueMap

public static Internal.EnumLiteMap<DataService.ProcessingModeDef.ShardingPolicy> internalGetValueMap()

getValueDescriptor

public final Descriptors.EnumValueDescriptor getValueDescriptor()

Specified by:

getValueDescriptor in interface ProtocolMessageEnum

getDescriptorForType

public final Descriptors.EnumDescriptor getDescriptorForType()

Specified by:

getDescriptorForType in interface ProtocolMessageEnum

getDescriptor

public static final Descriptors.EnumDescriptor getDescriptor()

valueOf

public static DataService.ProcessingModeDef.ShardingPolicy valueOf(Descriptors.EnumValueDescriptor desc)

Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)

Parameters:

desc - the name of the enum constant to be returned.

Returns:

the enum constant with the specified name

Throws:

IllegalArgumentException - if this enum type has no constant with the specified name

NullPointerException - if the argument is null