Tokenizer (API documentation of hlt.language packages)
Date: 2013-01-30

hlt.language.syntax

## Interface Tokenizer

- All Known Subinterfaces:
    FileTokenizer
  All Known Implementing Classes:
    AbstractFileTokenizer, AbstractTokenizer

      public interface Tokenizer

  This interface defines the information needed to complete the implementation of a parser.
  Version:
    Last modified on Fri Apr 13 20:13:47 2012 by hak
  Author:
    Hassan Aït-Kaci
  See Also:
    `Tokenizer`, `GenericParser`, `StaticParser`, `DynamicParser`
  Copyright:
    © by the author

- - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `java.io.Reader` | `getReader()`
    `int` | `lineNumber()`
    `ParseNode` | `nextToken()`
    `void` | `setReader(java.io.Reader rd)`

- - ### Method Detail

    - #### nextToken

          ParseNode nextToken()
                              throws java.io.IOException

      Throws:
        `java.io.IOException`

    - #### setReader

          void setReader(java.io.Reader rd)

    - #### getReader

          java.io.Reader getReader()

    - #### lineNumber

          int lineNumber()