Class AdaGrad

java.lang.Object

org.tensorflow.framework.optimizers.Optimizer

org.tensorflow.framework.optimizers.AdaGrad

public class AdaGrad
extends Optimizer

Optimizer that implements the Adagrad algorithm.

Adagrad is an optimizer with parameter-specific learning rates, which are adapted relative to how frequently a parameter gets updated during training. The more updates a parameter receives, the smaller the updates.

See Also:

• Duchi, J, et al., 2011, Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
• Duchi, J, et al., 2013, Proximal and First-Order Methods for Convex Optimization, Introduction Section 1

Nested Class Summary

Nested classes/interfaces inherited from class Optimizer

Optimizer.GradAndVar<T>, Optimizer.Options

Field Summary

Fields

Modifier and Type	Field	Description
static final String	ACCUMULATOR
static final float	INITIAL_ACCUMULATOR_DEFAULT
static final float	LEARNING_RATE_DEFAULT

Fields inherited from class Optimizer

globals, graph, tf, VARIABLE_V2

Constructor Summary

Constructors

Constructor	Description
AdaGrad(Graph graph)	Creates an AdaGrad Optimizer
AdaGrad(Graph graph, float learningRate)	Creates an AdaGrad Optimizer
AdaGrad(Graph graph, float learningRate, float initialAccumulatorValue)	Creates an AdaGrad Optimizer
AdaGrad(Graph graph, String name, float learningRate)	Creates an AdaGrad Optimizer
AdaGrad(Graph graph, String name, float learningRate, float initialAccumulatorValue)	Creates an AdaGrad Optimizer

Method Summary

Modifier and Type	Method	Description
protected <T extends TType> Op	applyDense(Ops deps, Output<T> gradient, Output<T> variable)	Generates the gradient update operations for the specific variable and gradient.
protected void	createSlots(List<Output<? extends TType>> variables)	Performs a No-op slot creation method.
String	getOptimizerName()	Get the Name of the optimizer.
String	toString()

Methods inherited from class Optimizer

applyGradients, computeGradients, createName, createSlot, finish, getSlot, getTF, minimize, minimize, prepare

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

ACCUMULATOR

public static final String ACCUMULATOR

See Also:

• Constant Field Values

LEARNING_RATE_DEFAULT

public static final float LEARNING_RATE_DEFAULT

See Also:

• Constant Field Values

INITIAL_ACCUMULATOR_DEFAULT

public static final float INITIAL_ACCUMULATOR_DEFAULT

See Also:

• Constant Field Values

Constructor Details

AdaGrad

public AdaGrad(Graph graph)

Creates an AdaGrad Optimizer

Parameters:

graph - the TensorFlow Graph

AdaGrad

public AdaGrad(Graph graph,
 float learningRate)

Creates an AdaGrad Optimizer

Parameters:

graph - the TensorFlow Graph

learningRate - the learning rate

AdaGrad

public AdaGrad(Graph graph,
 float learningRate,
 float initialAccumulatorValue)

Creates an AdaGrad Optimizer

Parameters:

graph - the TensorFlow Graph

learningRate - the learning rate

initialAccumulatorValue - Starting value for the accumulators, must be non-negative.

Throws:

IllegalArgumentException - if initialAccumulatorValue is negative

AdaGrad

public AdaGrad(Graph graph,
 String name,
 float learningRate)

Creates an AdaGrad Optimizer

Parameters:

graph - the TensorFlow Graph

name - the name for this Optimizer (defaults to 'Adagrad')

learningRate - the learning rate

AdaGrad

public AdaGrad(Graph graph,
 String name,
 float learningRate,
 float initialAccumulatorValue)

Creates an AdaGrad Optimizer

Parameters:

graph - the TensorFlow Graph

name - the name for this Optimizer (defaults to 'Adagrad')

learningRate - the learning rate

initialAccumulatorValue - Starting value for the accumulators, must be non-negative.

Throws:

IllegalArgumentException - if initialAccumulatorValue is negative

Method Details

createSlots

protected void createSlots(List<Output<? extends TType>> variables)

Performs a No-op slot creation method.

Overrides:

createSlots in class Optimizer

Parameters:

variables - The variables to create slots for.

applyDense

protected <T extends TType> Op applyDense(Ops deps,
 Output<T> gradient,
 Output<T> variable)

Generates the gradient update operations for the specific variable and gradient.

Specified by:

applyDense in class Optimizer

Type Parameters:

T - The type of the variable.

Parameters:

gradient - The gradient to use.

variable - The variable to update.

Returns:

An operand which applies the desired optimizer update to the variable.

toString

public String toString()

Overrides:

toString in class Object

getOptimizerName

public String getOptimizerName()

Get the Name of the optimizer.

Specified by:

getOptimizerName in class Optimizer

Returns:

The optimizer name.