Class SELU

java.lang.Object
org.tensorflow.framework.activations.AbstractActivation
org.tensorflow.framework.activations.SELU
All Implemented Interfaces:
Activation
public class SELU extends AbstractActivation
Scaled Exponential Linear Unit (SELU).

The Scaled Exponential Linear Unit (SELU) activation function is defined as:

  • if x > 0: return scale * x
  • if x < 0: return scale * alpha * (exp(x) - 1)

where alpha and scale are pre-defined constants (alpha=1.67326324 and scale=1.05070098).

Basically, the SELU activation function multiplies scale (> 1) with the output of the elu function to ensure a slope larger than one for positive inputs.

The values of alpha and scale are chosen so that the mean and variance of the inputs are preserved between two consecutive layers as long as the weights are initialized correctly (see LeCun with Normal Distribution) and the number of input units is "large enough"

Notes: To be used together with the LeCun initializer with Normal Distribution.

See Also:

  • Field Details

  • Constructor Details

    • SELU

      public SELU()
      Creates a Scaled Exponential Linear Unit (SELU) activation.

    • SELU

      public SELU(Map<String,Object> config)
      Creates a new Exponential from a configuration Map
      Parameters:
      config - the configuration map, this class does not use any of the entries in the configuration map
      Throws:
      IllegalArgumentException - if the configuration contains unsupported keys for this class or if the value for the name key does not match the name for the Activation

  • Method Details

    • selu

      public static <T extends TNumber> Operand<T> selu(Ops tf, Operand<T> input)
      Applies Scaled Exponential Linear Unit (SELU) activation function

      Example Usage: 

      Operand<TFloat32> input = ...;
Operand<TFloat32> result = SELU.selu(tf, input);
      Type Parameters:
      T - the data type for the input
      Parameters:
      tf - the TensorFlow Ops
      input - the input
      Returns:
      the input, unmodified.

    • call

      public <T extends TNumber> Operand<T> call(Ops tf, Operand<T> input)
      Gets the calculation operation for the activation.
      Type Parameters:
      T - the data type of the input and the result
      Parameters:
      tf - the TensorFlow Ops
      input - the input tensor
      Returns:
      The operand for the activation

    • getConfig

      public Map<String,Object> getConfig()
      Gets a configuration map
      Specified by:
      getConfig in class AbstractActivation
      Returns:
      the configuration map

    • getName

      public String getName()
      Get the name of the activation as known by the TensorFlow Engine
      Specified by:
      getName in class AbstractActivation
      Returns:
      the name of the activation as known by the TensorFlow Engine