Clone wiki

inf225public / glossary / Token

[Alphabetical Index | Tag Index]

Token

A Lexeme or group of characters that forms a basic unit of parsing, categorised according to type, e.g., identifier, number, addition operator, etc. Forms the alphabet of the parser in Scannerful parsing.

[Wikipedia]

Updated