Tanl Linguistic Pipeline |
A Lexicon stores the possible tags of each word in the training corpus, with their frequency. More...
#include <lexicon.h>
Public Member Functions | |
void | add (std::string &word, TagID tag) |
std::vector< TagID > * | find_nofreq (std::string &w) |
void | serialize (std::ostream &) |
void | serialize (std::istream &) |
A Lexicon stores the possible tags of each word in the training corpus, with their frequency.
A word shouldn't have too many possible tags, so we store these in a simple vector.