Class ReLU

java.lang.Object
org.tensorflow.framework.activations.AbstractActivation
org.tensorflow.framework.activations.ReLU
All Implemented Interfaces:
Activation
public class ReLU extends AbstractActivation
Rectified Linear Unit(ReLU) activation.

With default values, this returns the standard ReLU activation: max(x, 0), the element-wise maximum of 0 and the input tensor.

Modifying default parameters allows you to use non-zero thresholds, change the max value of the activation, and to use a non-zero multiple of the input for values below the threshold.

For example: 

Operand<TFloat32> input = tf.constant(
         new float[] {-10f, -5f, 0.0f, 5f, 10f});

// With default parameters
ReLU<TFloat32> relu = new ReLU<>(tf);
Operand<TFloat32> result = relu.call(input);
// result is [0.f,  0.f,  0.f,  5.f, 10.f]

// With alpha = 0.5
relu = new ReLU<>(tf, 0.5f, ReLU.MAX_VALUE_DEFAULT, ReLU.THRESHOLD_DEFAULT);
result = relu.call(input);
// result is [-5.f , -2.5f,  0.f ,  5.f , 10.f]

// With maxValue = 5
relu = new ReLU<>(tf, ReLU.ALPHA_DEFAULT, 5f, ReLU.THRESHOLD_DEFAULT);
result = relu.call(input);
// result is [0.f, 0.f, 0.f, 5.f, 5.f]

// With threshold = 5
relu = new ReLU<>(tf, ReLU.ALPHA_DEFAULT, ReLU.MAX_VALUE_DEFAULT, 5f);
result = relu.call(input);
// result is [-0.f, -0.f,  0.f,  0.f, 10.f]

  • Field Details

  • Constructor Details

    • ReLU

      public ReLU()
      Creates a new ReLU with alpha=ALPHA_DEFAULT, maxValue=MAX_VALUE_DEFAULT, threshold=THRESHOLD_DEFAULT,

    • ReLU

      public ReLU(float alpha, float maxValue, float threshold)
      Creates a new ReLU
      Parameters:
      alpha - governs the slope for values lower than the threshold.
      maxValue - sets the saturation threshold (the largest value the function will return).
      threshold - the threshold value of the activation function below which values will be damped or set to zero.

    • ReLU

      public ReLU(Map<String,Object> config)
      Creates a ReLU activation from a config map.
      Parameters:
      config - the configuration map,
      • if the map contains an entry for alpha that value is used, otherwise ALPHA_DEFAULT is used.
      • if the map contains an entry for max_value that value is used, otherwise MAX_VALUE_DEFAULT is used.
      • if the map contains an entry for threshold that value is used, otherwise THRESHOLD_DEFAULT is used.
      Throws:
      IllegalArgumentException - if the configuration contains unsupported keys for this class or if the value for the name key does not match the name for the Activation

  • Method Details

    • relu

      public static <T extends TNumber> Operand<T> relu(Ops tf, Operand<T> input)
      Applies the rectified linear unit activation function with default values.

      Example Usage: 

      Operand<TFloat32> input = ...;
Operand<TFloat32> result = ReLU.relu(tf, input);
      Type Parameters:
      T - the data type for the input
      Parameters:
      tf - the TensorFlow Ops
      input - the input
      Returns:
      the input, unmodified.

    • relu

      public static <T extends TNumber> Operand<T> relu(Ops tf, Operand<T> input, float alpha, float maxValue, float threshold)
      Applies the rectified linear unit activation function.

      Example Usage: 

      Operand<TFloat32> input = ...;
Operand<TFloat32> result = ReLU.relu(tf, input);
      Type Parameters:
      T - the data type for the input
      Parameters:
      tf - the TensorFlow Ops
      input - the input
      alpha - governs the slope for values lower than the threshold.
      maxValue - sets the saturation threshold (the largest value the function will return).
      threshold - the threshold value of the activation function below which values will be damped or set to zero.
      Returns:
      the input, unmodified.

    • getConfig

      public Map<String,Object> getConfig()
      Gets a configuration map with entries
      • alpha and value set with alpha.
      • max_value and value set with maxValue.
      • threshold and value set with threshold.
      Specified by:
      getConfig in class AbstractActivation
      Returns:
      config the configuration map

    • call

      public <T extends TNumber> Operand<T> call(Ops tf, Operand<T> input)
      Gets the calculation operation for the activation.
      Type Parameters:
      T - the data type of the input and the result
      Parameters:
      tf - the TensorFlow Ops
      input - the input tensor
      Returns:
      The operand for the activation

    • getName

      public String getName()
      Get the name of the activation as known by the TensorFlow Engine
      Specified by:
      getName in class AbstractActivation
      Returns:
      the name of the activation as known by the TensorFlow Engine

    • getAlpha

      public float getAlpha()
      Gets the value that governs the slope for values lower than the threshold.
      Returns:
      the value that governs the slope for values lower than the threshold.

    • getThreshold

      public float getThreshold()
      Gets the saturation threshold (the largest value the function will return).
      Returns:
      the saturation threshold (the largest value the function will return). public float getMaxValue() { return maxValue; }

      /** Gets the threshold value of the activation function below which values will be damped or set to zero.

    • getMaxValue

      public float getMaxValue()
      Gets the saturation threshold (the largest value the function will return).
      Returns:
      the saturation threshold (the largest value the function will return).