Tanl::SST::TokenCategorizer Class Reference

Assigns a type to a token, according to language locale. More...

#include <TokenCategorizer.h>

List of all members.

Public Member Functions
	TokenCategorizer (char const *lang)
EntityType	analyze (char const *token)
	Determine token type.

---

Detailed Description

Assigns a type to a token, according to language locale.

---

Constructor & Destructor Documentation

Tanl::SST::TokenCategorizer::TokenCategorizer

(

char const *

lang

)

Parameters:

lang

the language (locale) to use

---

The documentation for this class was generated from the following files:

• tag/SST/TokenCategorizer.h
• tag/SST/TokenCategorizer.cpp