public class SpanishUnknownWordModel extends BaseUnknownWordModel
Modifier and Type | Field and Description |
---|---|
protected boolean |
smartMutation |
protected int |
unknownPrefixSize |
protected int |
unknownSuffixSize |
NULL_ITW, nullTag, nullWord, tagHash, tagIndex, trainOptions, unknown, unknownLevel, unSeenCounter, useFirst, useGT, VERBOSE, wordIndex
Constructor and Description |
---|
SpanishUnknownWordModel(Options op,
Lexicon lex,
Index<java.lang.String> wordIndex,
Index<java.lang.String> tagIndex)
This constructor creates an UWM with empty data structures.
|
SpanishUnknownWordModel(Options op,
Lexicon lex,
Index<java.lang.String> wordIndex,
Index<java.lang.String> tagIndex,
ClassicCounter<IntTaggedWord> unSeenCounter) |
Modifier and Type | Method and Description |
---|---|
java.lang.String |
getSignature(java.lang.String word,
int loc)
TODO Can add various signatures, setting the signature via Options.
|
int |
getSignatureIndex(int index,
int sentencePosition,
java.lang.String word)
Returns the index of the signature of the word numbered wordIndex, where
the signature is the String representation of unknown word features.
|
float |
score(IntTaggedWord iTW,
int loc,
double c_Tseen,
double total,
double smooth,
java.lang.String word)
Currently we don't consider loc or the other parameters in determining
score in the default implementation; only English uses them.
|
addTagging, getLexicon, getUnknownLevel, score, scoreGT, scoreProbTagGivenWordSignature, unSeenCounter
protected final boolean smartMutation
protected final int unknownSuffixSize
protected final int unknownPrefixSize
public SpanishUnknownWordModel(Options op, Lexicon lex, Index<java.lang.String> wordIndex, Index<java.lang.String> tagIndex, ClassicCounter<IntTaggedWord> unSeenCounter)
public SpanishUnknownWordModel(Options op, Lexicon lex, Index<java.lang.String> wordIndex, Index<java.lang.String> tagIndex)
public float score(IntTaggedWord iTW, int loc, double c_Tseen, double total, double smooth, java.lang.String word)
BaseUnknownWordModel
score
in interface UnknownWordModel
score
in class BaseUnknownWordModel
iTW
- An IntTaggedWord pairing a word and POS tagloc
- The position in the sentence. In the default implementation
this is used only for unknown words to change their
probability distribution when sentence initial. Now,
a negative value c_Tseen
- Total count of this tag (on seen words) in trainingtotal
- Total count of word tokens in trainingsmooth
- Weighting on prior P(T|U) in estimateword
- The word itself; useful so we don't look it up in the indexpublic int getSignatureIndex(int index, int sentencePosition, java.lang.String word)
getSignatureIndex
in interface UnknownWordModel
getSignatureIndex
in class BaseUnknownWordModel
public java.lang.String getSignature(java.lang.String word, int loc)
getSignature
in interface UnknownWordModel
getSignature
in class BaseUnknownWordModel
word
- The word to make a signature forloc
- Its position in the sentence (mainly so sentence-initial
capitalized words can be treated differently)