Class DataLayouts

java.lang.Object
org.tensorflow.ndarray.buffer.layout.DataLayouts
public final class DataLayouts extends Object
Exposes DataLayout instances of data formats frequently used in linear algebra computation.

Example of usage: 

// Storing boolean values in a ByteDataBuffer
BooleanDataBuffer boolBuffer = DataLayouts.BOOL.applyTo(byteDataBuffer);

// Allocating a new buffer of 256 half floats
FloatDataBuffer halfBuffer = DataLayouts.FLOAT16.applyTo(DataBuffers.ofShorts(256 * DataLayouts.FLOAT16.scale());

  • Field Details

    • BFLOAT16

      public static final FloatDataLayout<ShortDataBuffer> BFLOAT16
      Data layout for converting 16-bit bfloats to/from short values.

      This format used to be specific to TensorFlow but has now been adopted more broadly in the machine learning field. It is optimized for fast conversion with single-precision 32-bit floating points by simply shifting their value and truncating the mantissa to only 7 bits.

      Therefore, this is a lost of precision in the fraction part compared to the IEEE-754 half-precision floating point specification (see FLOAT16 but it has a larger range of possible values in the whole part as it preserves the 8-bit exponent and uses the same bias, (i.e. an absolute range above 0 of approximately [10-40, 3.39 × 1038]

      Some CPUs support the bfloat16 format natively for better performances.

    • FLOAT16

      public static final FloatDataLayout<ShortDataBuffer> FLOAT16
      Data layout for converting 16-bit half floats to/from short values.

      Half floats are stored in memory accordingly to the IEEE-754 half-precision floating point specification, and are converted to/from 32-bit floats in the user space.

      There is a potential loss of precision when converting a single float (32-bit) to a half float (16-bit). Absolute range of values above 0 for a half float is approximately [5.96 × 10-8, 6.55 × 104] and their decimal part is rounded up to a 10 bits mantissa.

      In general, half float computation perform better on GPUs since, in general, CPUs do not support this format natively.

    • BOOL

      public static final BooleanDataLayout<ByteDataBuffer> BOOL
      Data layout for converting booleans to/from byte values.

      Since there is no Java NIO boolean buffer, this layout is particularly useful for mapping booleans values to standard byte buffers. The conversion between a boolean and a byte requires explicit type casting.

  • Constructor Details

    • DataLayouts

      public DataLayouts()

  • Method Details

    • ofStrings

      public static DataLayout<DataBuffer<byte[]>, String> ofStrings(Charset charset)
      Creates a data layout for converting strings to/from byte sequences.

      This layout requires a charset in parameter to specify how the strings must be encoded/decoded as byte sequences. So a new layout instance is always returned.

      Parameters:
      charset - charset to use
      Returns:
      a new string layout