DeSR Dependency Parser |
#include <Tokenizer.h>
Public Member Functions | |
Token (char const *text=0, TokenType type=(TokenType) 0) | |
Token const & | operator= (Token const &o) |
void | serialize (std::ostream &stream) |
Public Attributes | |
char const * | text |
the text of the token | |
TokenType | type |
the type of the token |
Definition at line 70 of file Tokenizer.h.