edu.stanford.nlp.optimization

Class StochasticMinimizer<T extends Function>

java.lang.Object

edu.stanford.nlp.optimization.StochasticMinimizer<T>

All Implemented Interfaces:

HasEvaluators, Minimizer<T>

Direct Known Subclasses:

InefficientSGDMinimizer, ScaledSGDMinimizer, SMDMinimizer, SQNMinimizer

public abstract class StochasticMinimizer<T extends Function>
extends java.lang.Object
implements Minimizer<T>, HasEvaluators

Stochastic Gradient Descent Minimizer. Note: If you want a fast SGD minimizer, then you probably want to use StochasticInPlaceMinimizer, not this class! The basic way to use the minimizer is with a null constructor, then the simple minimize method:

Minimizer smd = new SGDMinimizer();
DiffFunction df = new SomeDiffFunction(); //Note that it must be a incidence of AbstractStochasticCachingDiffFunction
double tol = 1e-4;
double[] initial = getInitialGuess();
int maxIterations = someSafeNumber;
double[] minimum = qnm.minimize(df,tol,initial,maxIterations);
Constructing with a null constructor will use the default values of

batchSize = 15;
initialGain = 0.1;

Since:

1.0

Version:

1.0

Author:

Alex Kleeman

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	StochasticMinimizer.PropertySetter<T1>

Field Summary

Fields

Modifier and Type	Field and Description
protected int	bSize
protected java.io.PrintWriter	file
double	gain
protected java.util.Random	gen
protected double[]	grad
protected java.util.List<double[]>	gradList
protected java.io.PrintWriter	infoFile
protected int	k
protected long	maxTime
protected int	memory
protected double[]	newGrad
protected double[]	newX
protected static java.text.NumberFormat	nf
protected int	numBatches
protected int	numPasses
int	outputFrequency
boolean	outputIterationsToFile
protected boolean	quiet
protected double[]	v
protected double[]	x

Constructor Summary

Constructors

Constructor and Description
StochasticMinimizer()

Method Summary

Modifier and Type	Method and Description
protected static double	gainSchedule(int it, double tau)
protected abstract java.lang.String	getName()
protected void	init(AbstractStochasticCachingDiffFunction func)
double[]	minimize(Function function, double functionTolerance, double[] initial) Attempts to find an unconstrained minimum of the objective function starting at initial, accurate to within functionTolerance (normally implemented as a multiplier of the range value to give range tolerance).
double[]	minimize(Function function, double functionTolerance, double[] initial, int maxIterations) Attempts to find an unconstrained minimum of the objective function starting at initial, accurate to within functionTolerance (normally implemented as a multiplier of the range value to give range tolerance), but running only for at most maxIterations iterations.
protected void	say(java.lang.String s)
protected void	sayln(java.lang.String s)
void	setEvaluators(int iters, Evaluator[] evaluators)
void	shutUp()
protected static double[]	smooth(java.util.List<double[]> toSmooth)
protected abstract void	takeStep(AbstractStochasticCachingDiffFunction dfunction)
abstract Pair<java.lang.Integer,java.lang.Double>	tune(Function function, double[] initial, long msPerTest)
Pair<java.lang.Integer,java.lang.Double>	tune(Function function, double[] initial, long msPerTest, java.util.List<java.lang.Integer> batchSizes, java.util.List<java.lang.Double> gains)
int	tuneBatch(Function function, double[] initial, long msPerTest, int bStart)
double	tuneDouble(Function function, double[] initial, long msPerTest, StochasticMinimizer.PropertySetter<java.lang.Double> ps, double lower, double upper)
double	tuneDouble(Function function, double[] initial, long msPerTest, StochasticMinimizer.PropertySetter<java.lang.Double> ps, double lower, double upper, double TOL)
double	tuneGain(Function function, double[] initial, long msPerTest, double lower, double upper)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

outputIterationsToFile

public boolean outputIterationsToFile

outputFrequency

public int outputFrequency

gain

public double gain

x

protected double[] x

newX

protected double[] newX

grad

protected double[] grad

newGrad

protected double[] newGrad

v

protected double[] v

numBatches

protected int numBatches

k

protected int k

bSize

protected int bSize

quiet

protected boolean quiet

gradList

protected java.util.List<double[]> gradList

memory

protected int memory

numPasses

protected int numPasses

gen

protected java.util.Random gen

file

protected java.io.PrintWriter file

infoFile

protected java.io.PrintWriter infoFile

maxTime

protected long maxTime

nf

protected static final java.text.NumberFormat nf

Constructor Detail

StochasticMinimizer

public StochasticMinimizer()

Method Detail

shutUp

public void shutUp()

getName

protected abstract java.lang.String getName()

takeStep

protected abstract void takeStep(AbstractStochasticCachingDiffFunction dfunction)

setEvaluators

public void setEvaluators(int iters,
                          Evaluator[] evaluators)

Specified by:

setEvaluators in interface HasEvaluators

gainSchedule

protected static double gainSchedule(int it,
                                     double tau)

smooth

protected static double[] smooth(java.util.List<double[]> toSmooth)

tune

public abstract Pair<java.lang.Integer,java.lang.Double> tune(Function function,
                                                              double[] initial,
                                                              long msPerTest)

tuneDouble

public double tuneDouble(Function function,
                         double[] initial,
                         long msPerTest,
                         StochasticMinimizer.PropertySetter<java.lang.Double> ps,
                         double lower,
                         double upper)

tuneDouble

public double tuneDouble(Function function,
                         double[] initial,
                         long msPerTest,
                         StochasticMinimizer.PropertySetter<java.lang.Double> ps,
                         double lower,
                         double upper,
                         double TOL)

tuneGain

public double tuneGain(Function function,
                       double[] initial,
                       long msPerTest,
                       double lower,
                       double upper)

tuneBatch

public int tuneBatch(Function function,
                     double[] initial,
                     long msPerTest,
                     int bStart)

tune

public Pair<java.lang.Integer,java.lang.Double> tune(Function function,
                                                     double[] initial,
                                                     long msPerTest,
                                                     java.util.List<java.lang.Integer> batchSizes,
                                                     java.util.List<java.lang.Double> gains)

init

protected void init(AbstractStochasticCachingDiffFunction func)

minimize

public double[] minimize(Function function,
                         double functionTolerance,
                         double[] initial)

Description copied from interface: Minimizer

Attempts to find an unconstrained minimum of the objective function starting at initial, accurate to within functionTolerance (normally implemented as a multiplier of the range value to give range tolerance).

Specified by:

minimize in interface Minimizer<T extends Function>

Parameters:

function - The objective function

functionTolerance - A double value

initial - An initial feasible point

Returns:

Unconstrained minimum of function

minimize

public double[] minimize(Function function,
                         double functionTolerance,
                         double[] initial,
                         int maxIterations)

Description copied from interface: Minimizer

Attempts to find an unconstrained minimum of the objective function starting at initial, accurate to within functionTolerance (normally implemented as a multiplier of the range value to give range tolerance), but running only for at most maxIterations iterations.

Specified by:

minimize in interface Minimizer<T extends Function>

Parameters:

function - The objective function

functionTolerance - A double value

initial - An initial feasible point

maxIterations - Maximum number of iterations

Returns:

Unconstrained minimum of function

sayln

protected void sayln(java.lang.String s)

say

protected void say(java.lang.String s)