Class AdaGradDA

java.lang.Object
org.tensorflow.framework.optimizers.Optimizer
org.tensorflow.framework.optimizers.AdaGradDA
public class AdaGradDA extends Optimizer
Optimizer that implements the Adagrad Dual-Averaging algorithm.

This optimizer takes care of regularization of unseen features in a mini batch by updating them when they are seen with a closed form update rule that is equivalent to having updated them on every mini-batch.

AdagradDA is typically used when there is a need for large sparsity in the trained model. This optimizer only guarantees sparsity for linear models. Be careful when using AdagradDA for deep networks as it will require careful initialization of the gradient accumulators for it to train.

See Also:

  • Field Details

  • Constructor Details

    • AdaGradDA

      public AdaGradDA(Graph graph)
      Creates an AdaGradDA Optimizer
      Parameters:
      graph - the TensorFlow Graph

    • AdaGradDA

      public AdaGradDA(Graph graph, float learningRate)
      Creates an AdaGradDA Optimizer
      Parameters:
      graph - the TensorFlow Graph
      learningRate - the learning rate

    • AdaGradDA

      public AdaGradDA(Graph graph, float learningRate, float initialAccumulatorValue, float l1Strength, float l2Strength)
      Creates an AdaGradDA Optimizer
      Parameters:
      graph - the TensorFlow Graph
      learningRate - the learning rate
      initialAccumulatorValue - Starting value for the accumulators, must be greater than zero.
      l1Strength - l1 regularization strength, must be greater than or equal to zero.
      l2Strength - l2 regularization strength, must be greater than or equal to zero.
      Throws:
      IllegalArgumentException - if initialAccumulatorValue is not greater than zero, or l1Strength or l2Strength is less than zero

    • AdaGradDA

      public AdaGradDA(Graph graph, String name, float learningRate)
      Creates an AdaGradDA Optimizer
      Parameters:
      graph - the TensorFlow Graph
      name - the name for this Optimizer (defaults to 'adagrad-da')
      learningRate - the learning rate

    • AdaGradDA

      public AdaGradDA(Graph graph, String name, float learningRate, float initialAccumulatorValue, float l1Strength, float l2Strength)
      Creates an AdaGradDA Optimizer
      Parameters:
      graph - the TensorFlow Graph
      name - the name for this Optimizer (defaults to 'adagrad-da')
      learningRate - the learning rate
      initialAccumulatorValue - Starting value for the accumulators, must be positive
      l1Strength - l1 regularization strength, must be greater than or equal to zero.
      l2Strength - l2 regularization strength, must be greater than or equal to zero.
      Throws:
      IllegalArgumentException - if initialAccumulatorValue is not greater than zero, or * l1Strength or l2Strength is less than zero

  • Method Details

    • prepare

      protected Optional<Op> prepare(String name)
      Returns a No-op prepare.
      Overrides:
      prepare in class Optimizer
      Parameters:
      name - The scope name to use for any variable creations.
      Returns:
      a No-op to prepare this optimizer, or empty if none.

    • createSlots

      protected void createSlots(List<Output<? extends TType>> variables)
      Performs a No-op slot creation method.
      Overrides:
      createSlots in class Optimizer
      Parameters:
      variables - The variables to create slots for.

    • applyDense

      protected <T extends TType> Op applyDense(Ops deps, Output<T> gradient, Output<T> variable)
      Generates the gradient update operations for the specific variable and gradient.
      Specified by:
      applyDense in class Optimizer
      Type Parameters:
      T - The type of the variable.
      Parameters:
      gradient - The gradient to use.
      variable - The variable to update.
      Returns:
      An operand which applies the desired optimizer update to the variable.

    • finish

      protected Op finish(List<Op> updateOperations, String name)
      Gathers up the update operations into a single op that can be used as a run target.

      Adds the global step update to the end of the updates list.

      Overrides:
      finish in class Optimizer
      Parameters:
      updateOperations - The update operations.
      name - The name of the run target.
      Returns:
      A NoOp with a control dependency on each update operation.

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getOptimizerName

      public String getOptimizerName()
      Get the Name of the optimizer.
      Specified by:
      getOptimizerName in class Optimizer
      Returns:
      The optimizer name.