Class ELU

java.lang.Object

org.tensorflow.framework.activations.AbstractActivation

org.tensorflow.framework.activations.ELU

All Implemented Interfaces:

Activation

public class ELU
extends AbstractActivation

Exponential linear unit.

The exponential linear unit (ELU) with alpha > 0 is:

x if x > 0 and alpha * (exp(x) - 1) if x < 0.

The ELU hyperparameter alpha controls the value to which an ELU saturates for negative net inputs. ELUs diminish the vanishing gradient effect.

ELUs have negative values which pushes the mean of the activations closer to zero. Mean activations that are closer to zero enable faster learning as they bring the gradient closer to the natural gradient. ELUs saturate to a negative value when the argument gets smaller. Saturation means a small derivative which decreases the variation and the information that is propagated to the next layer.

Example Usage:

Operand<TFloat32> input = ...;
ELU<TFloat32> elu = new ELU<>(tf, 2.0);
Operand<TFloat32> result = elu.call(input);

}

See Also:

• Clevert et al, 2016, Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)

Field Summary

Fields

Modifier and Type	Field	Description
static final String	NAME	The activation name as known by TensorFlow

Fields inherited from class AbstractActivation

NAME_KEY, tf

Modifier and Type	Field	Description
protected static final String	NAME_KEY
protected Ops	tf	The TensorFlow Ops

Constructor Summary

Constructors

Constructor	Description
ELU()	Creates a new ELU with alpha=ALPHA_DEFAULT.
ELU(double alpha)	Creates a new ELU
ELU(Map<String,Object> config)	Creates a new ELU from a configuration Map

Method Summary

Modifier and Type	Method	Description
<T extends TNumber> Operand<T>	call(Ops tf, Operand<T> input)	Gets the calculation operation for the activation.
static <T extends TNumber> Operand<T>	elu(Ops tf, Operand<T> input, double alpha)	Computes the Exponential linear unit.
double	getAlpha()	Gets the slope of negative section.
Map<String,Object>	getConfig()	Gets a configuration map
String	getName()	Get the name of the activation as known by the TensorFlow Engine

Methods inherited from class AbstractActivation

checkClassName, checkConfigKeys, getDefaultConfig, getTF, setTF

Modifier and Type	Method	Description
protected void	checkClassName(Map<String,Object> config)	Verifies that the configuration is for the same Activation class.
protected void	checkConfigKeys(Set<String> keysToCheck, Set<String> allowedKeys)	Verifies that any key in keysToCheck is also in the allowedKeys set.
protected Map<String,Object>	getDefaultConfig(String name)	Gets a configuration map, this default implementation returns a singleton Map, with AbstractActivation.NAME_KEY as the key, and the name parameter as its value;
protected Ops	getTF()	Gets the TensorFlow Ops
protected void	setTF(Ops tf)	Sets the TensorFlow Ops

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

NAME

public static final String NAME

The activation name as known by TensorFlow

See Also:

• Constant Field Values

Constructor Details

ELU

public ELU()

Creates a new ELU with alpha=ALPHA_DEFAULT.

ELU

public ELU(double alpha)

Creates a new ELU

Parameters:

alpha - A scalar, slope of negative section. It controls the value to which an ELU saturates for negative net inputs.

ELU

public ELU(Map<String,Object> config)

Creates a new ELU from a configuration Map

Parameters:

config - the configuration map, if the map contains an entry for alpha that value is used, otherwise ALPHA_DEFAULT is used.

Throws:

IllegalArgumentException - if the configuration contains unsupported keys for this class or if the value for the name key does not match the name for the Activation

Method Details

elu

public static <T extends TNumber> Operand<T> elu(Ops tf,
 Operand<T> input,
 double alpha)

Computes the Exponential linear unit.

The exponential linear unit (ELU) with alpha > 0 is:

x if x > 0 and alpha * (exp(x) - 1) if x < 0.}

The ELU hyperparameter alpha controls the value to which an ELU saturates for negative net inputs. ELUs diminish the vanishing gradient effect.

ELUs have negative values which pushes the mean of the activations closer to zero. Mean activations that are closer to zero enable faster learning as they bring the gradient closer to the natural gradient. ELUs saturate to a negative value when the argument gets smaller. Saturation means a small derivative which decreases the variation and the information that is propagated to the next layer.

Example Usage:

Operand<TFloat32> input = ...;
Operand<TFloat32> result = ELU.elu(tf, input, 2.0);

}

Type Parameters:

T - the data type for the input

Parameters:

tf - the TensorFlow Ops

input - the input

alpha - scalar, slope of negative section. alpha controls the value to which an ELU saturates for negative net inputs.

Returns:

The exponential linear unit (ELU) activation function: x if x > 0 and alpha * (exp(x) - 1) if x < 0 .

getConfig

public Map<String,Object> getConfig()

Gets a configuration map

Specified by:

getConfig in class AbstractActivation

Returns:

the configuration map

call

public <T extends TNumber> Operand<T> call(Ops tf,
 Operand<T> input)

Gets the calculation operation for the activation.

Type Parameters:

T - the data type of the input and the result

Parameters:

tf - the TensorFlow Ops

input - the input tensor

Returns:

The operand for the activation

getName

public String getName()

Get the name of the activation as known by the TensorFlow Engine

Specified by:

getName in class AbstractActivation

Returns:

the name of the activation as known by the TensorFlow Engine

getAlpha

public double getAlpha()

Gets the slope of negative section.

Returns:

the slope of negative section.