Package edu.stanford.nlp.trees.treebank

Class AbstractDataset

java.lang.Object

edu.stanford.nlp.trees.treebank.AbstractDataset

All Implemented Interfaces:

Dataset

Direct Known Subclasses:

ATBArabicDataset, FTBDataset

public abstract class AbstractDataset
extends Object
implements Dataset

Author:

Spence Green

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static class	AbstractDataset.SplitFilter

Nested classes/interfaces inherited from interface edu.stanford.nlp.trees.treebank.Dataset

Dataset.Encoding

Field Summary

Fields

Modifier and Type	Field	Description
protected boolean	addDeterminer
protected boolean	addRoot
protected Set<String>	configuredOptions
protected TreeVisitor	customTreeVisitor
protected Dataset.Encoding	encoding
protected Pattern	fileNameNormalizer
protected String	flatFileName
protected String	lexMapOptions
protected Mapper	lexMapper
protected boolean	makeFlatFile
protected int	maxLen
protected String	morphDelim
protected Properties	options	Provides access for sub-classes to the data set parameters
protected String	outFileName
protected List<String>	outputFileList
protected List<File>	pathsToData
protected List<File>	pathsToMappings
protected String	posMapOptions
protected Mapper	posMapper
protected boolean	removeDashTags
protected boolean	removeEscapeTokens
protected Set<String>	requiredOptions
protected FileFilter	splitFilter
protected StringBuilder	toStringBuilder
protected Treebank	treebank
protected String	treeFileExtension

Constructor Summary

Constructors

Constructor	Description
AbstractDataset()

Method Summary

Modifier and Type	Method	Description
abstract void	build()	Generic method for loading, processing, and writing a dataset.
protected Set<String>	buildSplitMap(String path)
List<String>	getFilenames()	Returns the filenames written by Dataset.build().
boolean	setOptions(Properties opts)	Sets options for a dataset.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

outputFileList

protected final List<String> outputFileList

posMapper

protected Mapper posMapper

posMapOptions

protected String posMapOptions

lexMapper

protected Mapper lexMapper

lexMapOptions

protected String lexMapOptions

encoding

protected Dataset.Encoding encoding

pathsToData

protected final List<File> pathsToData

pathsToMappings

protected final List<File> pathsToMappings

splitFilter

protected FileFilter splitFilter

addDeterminer

protected boolean addDeterminer

removeDashTags

protected boolean removeDashTags

addRoot

protected boolean addRoot

removeEscapeTokens

protected boolean removeEscapeTokens

maxLen

protected int maxLen

morphDelim

protected String morphDelim

customTreeVisitor

protected TreeVisitor customTreeVisitor

outFileName

protected String outFileName

flatFileName

protected String flatFileName

makeFlatFile

protected boolean makeFlatFile

fileNameNormalizer

protected final Pattern fileNameNormalizer

treebank

protected Treebank treebank

configuredOptions

protected final Set<String> configuredOptions

requiredOptions

protected final Set<String> requiredOptions

toStringBuilder

protected final StringBuilder toStringBuilder

treeFileExtension

protected String treeFileExtension

options

protected Properties options

Provides access for sub-classes to the data set parameters

Constructor Detail

AbstractDataset

public AbstractDataset()

Method Detail

build

public abstract void build()

Description copied from interface: Dataset

Generic method for loading, processing, and writing a dataset.

Specified by:

build in interface Dataset

setOptions

public boolean setOptions(Properties opts)

Description copied from interface: Dataset

Sets options for a dataset.

Specified by:

setOptions in interface Dataset

Parameters:

opts - A map from parameter types defined in ConfigParser to values

Returns:

true if opts contains all required options. false, otherwise.

buildSplitMap

protected Set<String> buildSplitMap(String path)

getFilenames

public List<String> getFilenames()

Description copied from interface: Dataset

Returns the filenames written by Dataset.build().

Specified by:

getFilenames in interface Dataset

Returns:

A collection of filenames

toString

public String toString()

Overrides:

toString in class Object