| DeSR Dependency Parser |
#include <Tokenizer.h>
Public Member Functions | |
| Token (char const *text=0, TokenType type=(TokenType) 0) | |
| Token const & | operator= (Token const &o) |
| void | serialize (std::ostream &stream) |
Public Attributes | |
| char const * | text |
| the text of the token | |
| TokenType | type |
| the type of the token | |
Definition at line 70 of file Tokenizer.h.