Package edu.stanford.nlp.optimization

Class SGDToQNMinimizer

  • All Implemented Interfaces:
    Minimizer<DiffFunction>, Serializable
    public class SGDToQNMinimizer
extends Object
implements Minimizer<DiffFunction>, Serializable
    Stochastic Gradient Descent To Quasi Newton Minimizer An experimental minimizer which takes a stochastic function (one implementing AbstractStochasticCachingDiffFunction) and executes SGD for the first couple passes. During the final iterations a series of approximate hessian vector products are built up. These are then passed to the QNminimizer so that it can start right up without the typical delay. Note [2012] The basic idea here is good, but the original ScaledSGDMinimizer wasn't efficient, and so this would be much more useful if rewritten to use the good StochasticInPlaceMinimizer instead.
    Since:
    1.0
    Version:
    1.0
    Author:
    Alex Kleeman
    See Also:
    Serialized Form

    • Constructor Summary

      Constructors 
      Constructor Description
      SGDToQNMinimizer​(double SGDGain, int batchSize, int SGDPasses, int QNPasses)  
      SGDToQNMinimizer​(double SGDGain, int batchSize, int sgdPasses, int qnPasses, int hessSamples, int QNMem)  
      SGDToQNMinimizer​(double SGDGain, int batchSize, int sgdPasses, int qnPasses, int hessSamples, int QNMem, boolean outputToFile)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      protected String getName()  
      double[] minimize​(DiffFunction function, double functionTolerance, double[] initial)
      Attempts to find an unconstrained minimum of the objective function starting at initial, accurate to within functionTolerance (normally implemented as a multiplier of the range value to give range tolerance).
      double[] minimize​(DiffFunction function, double functionTolerance, double[] initial, int maxIterations)
      Attempts to find an unconstrained minimum of the objective function starting at initial, accurate to within functionTolerance (normally implemented as a multiplier of the range value to give range tolerance), but running only for at most maxIterations iterations.
      void shutUp()  

    • Field Detail

      • outputIterationsToFile

        public boolean outputIterationsToFile

      • gain

        public double gain

      • SGDPasses

        public int SGDPasses

      • QNPasses

        public int QNPasses

    • Constructor Detail

      • SGDToQNMinimizer

        public SGDToQNMinimizer​(double SGDGain,
                        int batchSize,
                        int SGDPasses,
                        int QNPasses)

      • SGDToQNMinimizer

        public SGDToQNMinimizer​(double SGDGain,
                        int batchSize,
                        int sgdPasses,
                        int qnPasses,
                        int hessSamples,
                        int QNMem)

      • SGDToQNMinimizer

        public SGDToQNMinimizer​(double SGDGain,
                        int batchSize,
                        int sgdPasses,
                        int qnPasses,
                        int hessSamples,
                        int QNMem,
                        boolean outputToFile)

    • Method Detail

      • shutUp

        public void shutUp()

      • getName

        protected String getName()

      • minimize

        public double[] minimize​(DiffFunction function,
                         double functionTolerance,
                         double[] initial)
        Description copied from interface: Minimizer
        Attempts to find an unconstrained minimum of the objective function starting at initial, accurate to within functionTolerance (normally implemented as a multiplier of the range value to give range tolerance).
        Specified by:
        minimize in interface Minimizer<DiffFunction>
        Parameters:
        function - The objective function
        functionTolerance - A double value
        initial - An initial feasible point
        Returns:
        Unconstrained minimum of function

      • minimize

        public double[] minimize​(DiffFunction function,
                         double functionTolerance,
                         double[] initial,
                         int maxIterations)
        Description copied from interface: Minimizer
        Attempts to find an unconstrained minimum of the objective function starting at initial, accurate to within functionTolerance (normally implemented as a multiplier of the range value to give range tolerance), but running only for at most maxIterations iterations.
        Specified by:
        minimize in interface Minimizer<DiffFunction>
        Parameters:
        function - The objective function
        functionTolerance - A double value
        initial - An initial feasible point
        maxIterations - Maximum number of iterations
        Returns:
        Unconstrained minimum of function