public class CodepointCoreLabelProcessor extends CoreLabelProcessor
Constructor and Description |
---|
CodepointCoreLabelProcessor() |
Modifier and Type | Method and Description |
---|---|
java.util.List<CoreLabel> |
process(java.util.List<CoreLabel> tokens)
Adds codepoint offsets to the tokens (parallel to character offsets).
|
java.util.List<CoreLabel> |
restore(java.util.List<CoreLabel> originalTokens,
java.util.List<CoreLabel> processedTokens)
Removes the codepoint offsets in-place and returns the deannotated tokens
|
public java.util.List<CoreLabel> process(java.util.List<CoreLabel> tokens)
process
in class CoreLabelProcessor
public java.util.List<CoreLabel> restore(java.util.List<CoreLabel> originalTokens, java.util.List<CoreLabel> processedTokens)
restore
in class CoreLabelProcessor