Tanl::Text::StringTokenizer Class Reference

Inheritance diagram for Tanl::Text::StringTokenizer:

List of all members.

Public Member Functions
	StringTokenizer (std::istream &is, char const *delim=delimiters)
	Tokenize into words delimited by.
	StringTokenizer (char const *s, char const *end=0, char const *delim=delimiters)
	Tokenize into words delimited by.
char const *	hasNext ()
char const *	next ()
int	tokenLength ()
	Return the length of the current token.
int	tokenStart ()
	Return the number of characters before the current token.
void	advance (int n)
	Advance count.
Protected Attributes
int	count
int	lineNumber
std::istream *	is
char	line [MAX_LINE_LEN]
char const *	tok
int	toklen
char const *	rest
char const *	end
char const *	delim

---

Constructor & Destructor Documentation

Tanl::Text::StringTokenizer::StringTokenizer	(	std::istream &	is,
		char const *	delim = delimiters
	)

Tokenize into words delimited by.

Parameters:

	delim.	Read text from stream
	is	input stream
	delim	string of deliminting characters

Tanl::Text::StringTokenizer::StringTokenizer	(	char const *	s,
		char const *	end = 0,
		char const *	delim = delimiters
	)

Tokenize into words delimited by.

Parameters:

	delim.	Read from text between
	start	and
	end.
	start	string beginning
	end	string end
	delim	string of deliminting characters

---

Member Function Documentation

void Tanl::Text::StringTokenizer::advance

(

int

n

)

[inline]

Advance count.

Used when token had to be split.

---

The documentation for this class was generated from the following files:

• text/StringTokenizer.h
• text/StringTokenizer.cpp