Tanl Linguistic Pipeline |
Classes | |
struct | Token |
Represents a token recognized by the scanner. More... | |
Public Types | |
enum | TokenType { Eof = 0, Word = 200, Abbrev, Date, Number, Phone, Tag, Url, Punct } |
Token types. More... | |
Public Member Functions | |
Token | scan () |
We define our own method scan() instead of yylex(), since we return a Token instead of int. | |
Static Public Attributes | |
static char const * | TypeName [] |
Token types.
char const * Parser::Scanner::TypeName [static] |
{ "Eof", "Word", "Abbrev", "Date", "Number", "Phone", "Tag", "Url", "Punct" }