Enum DatasetOptions.AutoShardPolicy

java.lang.Object
java.lang.Enum<DatasetOptions.AutoShardPolicy>
org.tensorflow.proto.data.DatasetOptions.AutoShardPolicy
All Implemented Interfaces:
Internal.EnumLite, ProtocolMessageEnum, Serializable, Comparable<DatasetOptions.AutoShardPolicy>
Enclosing class:
DatasetOptions

public static enum DatasetOptions.AutoShardPolicy extends Enum<DatasetOptions.AutoShardPolicy> implements ProtocolMessageEnum
Represents the type of auto-sharding we enable.
Protobuf enum tensorflow.data.AutoShardPolicy
  • Enum Constant Details

    • AUTO

      public static final DatasetOptions.AutoShardPolicy AUTO
      AUTO: Attempts FILE-based sharding, falling back to DATA-based sharding.
      
      AUTO = 0;
    • FILE

      public static final DatasetOptions.AutoShardPolicy FILE
      FILE: Shards by input files (i.e. each worker will get a set of files to
      process). When this option is selected, make sure that there is at least as
      many files as workers. If there are fewer input files than workers, a
      runtime error will be raised.
      
      FILE = 1;
    • DATA

      public static final DatasetOptions.AutoShardPolicy DATA
      DATA: Shards by elements produced by the dataset. Each worker will process
      the whole dataset and discard the portion that is not for itself. Note that
      for this mode to correctly partitions the dataset elements, the dataset
      needs to produce elements in a deterministic order.
      
      DATA = 2;
    • HINT

      public static final DatasetOptions.AutoShardPolicy HINT
      HINT: Looks for the presence of `shard(SHARD_HINT, ...)` which is treated
      as a placeholder to replace with `shard(num_workers, worker_index)`.
      
      HINT = 3;
    • OFF

      public static final DatasetOptions.AutoShardPolicy OFF
      OFF: No sharding will be performed.
      
      OFF = -1;
    • UNRECOGNIZED

      public static final DatasetOptions.AutoShardPolicy UNRECOGNIZED
  • Field Details

    • AUTO_VALUE

      public static final int AUTO_VALUE
      AUTO: Attempts FILE-based sharding, falling back to DATA-based sharding.
      
      AUTO = 0;
      See Also:
    • FILE_VALUE

      public static final int FILE_VALUE
      FILE: Shards by input files (i.e. each worker will get a set of files to
      process). When this option is selected, make sure that there is at least as
      many files as workers. If there are fewer input files than workers, a
      runtime error will be raised.
      
      FILE = 1;
      See Also:
    • DATA_VALUE

      public static final int DATA_VALUE
      DATA: Shards by elements produced by the dataset. Each worker will process
      the whole dataset and discard the portion that is not for itself. Note that
      for this mode to correctly partitions the dataset elements, the dataset
      needs to produce elements in a deterministic order.
      
      DATA = 2;
      See Also:
    • HINT_VALUE

      public static final int HINT_VALUE
      HINT: Looks for the presence of `shard(SHARD_HINT, ...)` which is treated
      as a placeholder to replace with `shard(num_workers, worker_index)`.
      
      HINT = 3;
      See Also:
    • OFF_VALUE

      public static final int OFF_VALUE
      OFF: No sharding will be performed.
      
      OFF = -1;
      See Also:
  • Method Details