Class GELU

java.lang.Object
org.tensorflow.framework.activations.AbstractActivation
org.tensorflow.framework.activations.GELU
All Implemented Interfaces:
Activation
public class GELU extends AbstractActivation
Applies the Gaussian error linear unit (GELU) activation function.

Gaussian error linear unit (GELU) computes x * P(X <= x), where P(X) ~ N(0, 1). The (GELU) nonlinearity weights inputs by their value, rather than gates inputs by their sign as in ReLU.

For example: 

x = tf.constant(new float[] {-3.0f, -1.0f, 0.0f, 1.0f, 3.f});
GELU gelu = new GELU();
y = gelu.call(tf, x);
// output [-0.00404951f, -0.15865529f, 0.f , 0.8413447f , 2.9959507f ]


}

  • Field Details

  • Constructor Details

    • GELU

      public GELU()
      Creates a Gaussian error linear unit (GELU) activation.

    • GELU

      public GELU(boolean approximate)
      Creates a Gaussian error linear unit (GELU) activation.
      Parameters:
      approximate - whether to enable approximation.

    • GELU

      public GELU(Map<String,Object> config)
      Creates a GELU activation from a config map.
      Parameters:
      config - the configuration map, if the map contains an entry for approximate that value is used, otherwise false is used.
      Throws:
      IllegalArgumentException - if the configuration contains unsupported keys for this class or if the value for the name key does not match the name for the Activation

  • Method Details

    • gelu

      public static <T extends TNumber> Operand<T> gelu(Ops tf, Operand<T> input)
      Applies the Gaussian error linear unit (GELU) activation function with approximate set to false.

      Example Usage: 

      Operand<TFloat32> input = ...;
Operand<TFloat32> result = Gelu.gelu(tf, input);
      Type Parameters:
      T - the data type for the input
      Parameters:
      tf - the TensorFlow Ops
      input - the input
      Returns:
      the exponential activation: exp(x).

    • gelu

      public static <T extends TNumber> Operand<T> gelu(Ops tf, Operand<T> input, boolean approximate)
      Applies the Gaussian error linear unit (GELU) activation function.

      Example Usage: 

      Operand<TFloat32> input = ...;
Operand<TFloat32> result = Gelu.gelu(tf, input, true);
      Type Parameters:
      T - the data type for the input
      Parameters:
      tf - the TensorFlow Ops
      input - the input
      approximate - whether to enable approximation.
      Returns:
      the exponential activation: exp(x).

    • getConfig

      public Map<String,Object> getConfig()
      Gets a configuration map with entries
      • approximate and value set with approximate.
      Specified by:
      getConfig in class AbstractActivation
      Returns:
      config the configuration map

    • call

      public <T extends TNumber> Operand<T> call(Ops tf, Operand<T> input)
      Gets the calculation operation for the activation.
      Type Parameters:
      T - the data type of the input and the result
      Parameters:
      tf - the TensorFlow Ops
      input - the input tensor
      Returns:
      The operand for the activation

    • getName

      public String getName()
      Get the name of the activation as known by the TensorFlow Engine
      Specified by:
      getName in class AbstractActivation
      Returns:
      the name of the activation as known by the TensorFlow Engine

    • isApproximate

      public boolean isApproximate()
      Gets the flag whether to enable approximation.
      Returns:
      the flag whether to enable approximation.