edu.stanford.nlp.process

Interface TSVSentenceProcessor

All Known Implementing Classes:

CreateClauseDataset

public interface TSVSentenceProcessor

An interface for running an action (a callback function) on each line of a TSV file representing a collection of sentences in a corpus. This is a useful callback for processing a large batch of sentences; e.g., out of a Greenplum database.

Author:

Gabor Angeli

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	TSVSentenceProcessor.SentenceField A list of possible fields in the sentence table.

Field Summary

Fields

Modifier and Type	Field and Description
static java.util.List<TSVSentenceProcessor.SentenceField>	DEFAULT_SENTENCE_TABLE The list of fields actually in the sentence table being passed as a query to TSVSentenceProcessor.

Method Summary

Modifier and Type	Method and Description
void	process(long id, Annotation doc) Process a given sentence.
default void	runAndExit(java.io.InputStream in, java.io.PrintStream debugStream, java.util.function.IntUnaryOperator cleanup)
default void	runAndExit(java.io.InputStream in, java.io.PrintStream debugStream, java.util.function.IntUnaryOperator cleanup, java.util.List<TSVSentenceProcessor.SentenceField> sentenceTableSpec) Runs the given implementation of TSVSentenceProcessor, and then exits with the appropriate error code.

Field Detail

DEFAULT_SENTENCE_TABLE

static final java.util.List<TSVSentenceProcessor.SentenceField> DEFAULT_SENTENCE_TABLE

The list of fields actually in the sentence table being passed as a query to TSVSentenceProcessor.

Method Detail

process

void process(long id,
             Annotation doc)

Process a given sentence.

Parameters:

id - The sentence id (database id) of the sentence being processed.

doc - The single-sentence document to annotate. This contains:

• Tokens
• A parse tree (Collapsed dependencies)
• POS Tags
• NER tags
• Lemmas
• DocID
• Sentence index

runAndExit

default void runAndExit(java.io.InputStream in,
                        java.io.PrintStream debugStream,
                        java.util.function.IntUnaryOperator cleanup,
                        java.util.List<TSVSentenceProcessor.SentenceField> sentenceTableSpec)

Runs the given implementation of TSVSentenceProcessor, and then exits with the appropriate error code. The error code is the number of exceptions encountered during processing.

Parameters:

in - The input stream to read examples off of.

debugStream - The stream to write debugging information to (e.g., stderr).

cleanup - A function to run after annotation is over, to clean up open files, etc. Takes as input the candidate error code, and returns a new error code to exit on.

sentenceTableSpec - The header of the sentence table fields being fed as input to this function. By default, this can be DEFAULT_SENTENCE_TABLE.

runAndExit

default void runAndExit(java.io.InputStream in,
                        java.io.PrintStream debugStream,
                        java.util.function.IntUnaryOperator cleanup)

See Also:

runAndExit(InputStream, PrintStream, IntUnaryOperator, List)