Package edu.stanford.nlp.simple

Class SentenceAlgorithms

  • public class SentenceAlgorithms
extends Object

    A set of common utility algorithms for working with sentences (e.g., finding the head of a span). These are not intended to be perfect, or even the canonical version of these algorithms. They should only be trusted for prototyping, and more careful attention should be paid in cases where the performance of the task is important or the domain is unusual.

    For developers: this class is intended to be where domain independent and broadly useful functions on a sentence would go, rather than polluting the Sentence class itself.

    Author:
    Gabor Angeli

    • Constructor Detail

      • SentenceAlgorithms

        public SentenceAlgorithms​(Sentence impl)
        Create a new algorithms object, based off of a sentence.
        See Also:
        Sentence.algorithms()

    • Method Detail

      • keyphraseSpans

        public List<Span> keyphraseSpans()
        Returns a collection of keyphrases, defined as relevant noun phrases and verbs in the sentence. Each token of the sentence is consumed at most once. What counts as a keyphrase is in general quite subjective -- this method is just one possible interpretation (in particular, Gabor's interpretation). Please don't rely on this method to produce exactly your interpretation of what a keyphrase is.
        Returns:
        A list of spans in the sentence, where each one corresponds to a keyphrase.

      • keyphrases

        public List<String> keyphrases​(Function<Sentence,​List<String>> toString)
        Get the keyphrases of the sentence as a list of Strings.
        Parameters:
        toString - The function to use to convert a span to a string. The canonical case is Sentence::words
        Returns:
        A list of keyphrases, as Strings.
        See Also:
        keyphraseSpans()

      • keyphrases

        public List<String> keyphrases()
        The keyphrases of the sentence, using the words of the sentence to convert a span into a keyphrase.
        Returns:
        A list of String keyphrases in the sentence.
        See Also:
        keyphraseSpans()

      • headOfSpan

        public int headOfSpan​(Span tokenSpan)
        Get the index of the head word for a given span, based off of the dependency parse.
        Parameters:
        tokenSpan - The span of tokens we are finding the head of.
        Returns:
        The head index of the given span of tokens.

      • allSpans

        public <E> Iterable<List<E>> allSpans​(Function<Sentence,​List<E>> selector,
                                      int maxLength)
        Return all the spans of a sentence. So, for example, a sentence "a b c" would return: [a], [b], [c], [a b], [b c], [a b c].
        Type Parameters:
        E - The type of the element we are getting.
        Parameters:
        selector - The function to apply to each token. For example, Sentence.words(). For that example, you can use allSpans(Sentence::words).
        maxLength - The maximum length of the spans to extract. The default to extract all spans is to set this to sentence.length().
        Returns:
        A streaming iterable of spans for this sentence.

      • modeInSpan

        public <E> E modeInSpan​(Span span,
                        Function<Sentence,​List<E>> selector)
        Select the most common element of the given type in the given span. This is useful for, e.g., finding the most likely NER span of a given span, or the most likely POS tag of a given span. Null entries are removed.
        Type Parameters:
        E - The type of the element we are getting.
        Parameters:
        span - The span of the sentence to find the mode element in. This must be entirely contained in the sentence.
        selector - The property of the sentence we are getting the mode of. For example, Sentence::posTags
        Returns:
        The most common element of the given property in the sentence.

      • loopyDependencyPathBetween

        protected List<String> loopyDependencyPathBetween​(int start,
                                                  int end,
                                                  Optional<Function<Sentence,​List<String>>> selector)
        Run a proper BFS over a dependency graph, finding the shortest path between two vertices.
        Parameters:
        start - The start index.
        end - The end index.
        selector - The selector to use for the word nodes.
        Returns:
        A path string, analogous to dependencyPathBetween(int, int)

      • dependencyPathBetween

        public List<String> dependencyPathBetween​(int start,
                                          int end,
                                          Optional<Function<Sentence,​List<String>>> selector)
        Find the dependency path between two words in a sentence.
        Parameters:
        start - The start word, 0-indexed.
        end - The end word, 0-indexed.
        selector - The selector for the strings between the path, if any. If left empty, these will be omitted from the list.
        Returns:
        A list encoding the dependency path between the vertices, suitable for inclusion as features.

      • dependencyPathBetween

        public List<String> dependencyPathBetween​(int start,
                                          int end)

      • unescapeHTML

        public void unescapeHTML()
        A funky little helper method to interpret each token of the sentence as an HTML string, and translate it back to text. Note that this is in place.