Package edu.stanford.nlp.international.spanish.process

Class SpanishTokenizer.SpanishTokenizerFactory<T extends HasWord>

java.lang.Object

edu.stanford.nlp.international.spanish.process.SpanishTokenizer.SpanishTokenizerFactory<T>

All Implemented Interfaces:

IteratorFromReaderFactory<T>, TokenizerFactory<T>, Serializable

Enclosing class:

SpanishTokenizer<T extends HasWord>

public static class SpanishTokenizer.SpanishTokenizerFactory<T extends HasWord>
extends Object
implements TokenizerFactory<T>

A factory for Spanish tokenizer instances.

Author:

Spence Green

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
protected LexedTokenFactory<T>	factory
protected Properties	lexerProperties
protected boolean	splitCompoundOption
protected boolean	splitContractionOption
protected boolean	splitVerbOption

Method Summary

Modifier and Type	Method	Description
Iterator<T>	getIterator(Reader r)	Return an iterator over the contents read from r.
Tokenizer<T>	getTokenizer(Reader r)	Get a tokenizer for this reader.
Tokenizer<T>	getTokenizer(Reader r, String extraOptions)	Get a tokenizer for this reader.
static TokenizerFactory<CoreLabel>	newCoreLabelTokenizerFactory()
static <T extends HasWord> SpanishTokenizer.SpanishTokenizerFactory<T>	newSpanishTokenizerFactory(LexedTokenFactory<T> factory, String options)	Constructs a new SpanishTokenizer that returns T objects and uses the options passed in.
void	setOptions(String options)	Set underlying tokenizer options.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

factory

protected final LexedTokenFactory<T extends HasWord> factory

lexerProperties

protected Properties lexerProperties

splitCompoundOption

protected boolean splitCompoundOption

splitVerbOption

protected boolean splitVerbOption

splitContractionOption

protected boolean splitContractionOption

Method Detail

newCoreLabelTokenizerFactory

public static TokenizerFactory<CoreLabel> newCoreLabelTokenizerFactory()

newSpanishTokenizerFactory

public static <T extends HasWord> SpanishTokenizer.SpanishTokenizerFactory<T> newSpanishTokenizerFactory(LexedTokenFactory<T> factory,
                                                                                                         String options)

Constructs a new SpanishTokenizer that returns T objects and uses the options passed in.

Parameters:

options - a String of options, separated by commas

factory - a factory for the token type that the tokenizer will return

Returns:

A TokenizerFactory that returns the right token types

getIterator

public Iterator<T> getIterator(Reader r)

Description copied from interface: IteratorFromReaderFactory

Return an iterator over the contents read from r.

Specified by:

getIterator in interface IteratorFromReaderFactory<T extends HasWord>

Parameters:

r - Where to read objects from

Returns:

An Iterator over the objects

getTokenizer

public Tokenizer<T> getTokenizer(Reader r)

Description copied from interface: TokenizerFactory

Get a tokenizer for this reader.

Specified by:

getTokenizer in interface TokenizerFactory<T extends HasWord>

Parameters:

r - A Reader (which is assumed to already by buffered, if appropriate)

Returns:

A Tokenizer

setOptions

public void setOptions(String options)

Set underlying tokenizer options.

Specified by:

setOptions in interface TokenizerFactory<T extends HasWord>

Parameters:

options - A comma-separated list of options

getTokenizer

public Tokenizer<T> getTokenizer(Reader r,
                                 String extraOptions)

Description copied from interface: TokenizerFactory

Get a tokenizer for this reader.

Specified by:

getTokenizer in interface TokenizerFactory<T extends HasWord>

Parameters:

r - A Reader (which is assumed to already by buffered, if appropriate)

extraOptions - Options for how this tokenizer should behave

Returns:

A Tokenizer