Package edu.stanford.nlp.process

Class CoreLabelProcessor

java.lang.Object

edu.stanford.nlp.process.CoreLabelProcessor

Direct Known Subclasses:

CodepointCoreLabelProcessor, GermanTokenizerPostProcessor

public abstract class CoreLabelProcessor
extends Object

Abstract class for processing a List<CoreLabel>. Example use case is merging on hyphens for German NER (process) and then restoring after classification (restore).

Constructor Summary

Constructors

Constructor	Description
CoreLabelProcessor()

Method Summary

Modifier and Type	Method	Description
abstract List<CoreLabel>	process(List<CoreLabel> tokens)	Alter the tokenization of list of tokens (e.g.
abstract List<CoreLabel>	restore(List<CoreLabel> originalTokens, List<CoreLabel> processedTokens)	Undo the tokenization changes of process, maintaining any tagging

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CoreLabelProcessor

public CoreLabelProcessor()

Method Detail

process

public abstract List<CoreLabel> process(List<CoreLabel> tokens)

Alter the tokenization of list of tokens (e.g. map split on hyphen to don't split)

restore

public abstract List<CoreLabel> restore(List<CoreLabel> originalTokens,
                                        List<CoreLabel> processedTokens)

Undo the tokenization changes of process, maintaining any tagging