LAStreamTokenizer (API documentation of hlt.language packages)
Date: 2013-01-30

hlt.language.io

## Class LAStreamTokenizer

- java.lang.Object
- - hlt.language.io.AbstractStreamTokenizer
  - - hlt.language.io.LAStreamTokenizer

-     public class LAStreamTokenizer
      extends AbstractStreamTokenizer

  This provides the exact same interface as that of the standard Java Core API's `java.io.StreamTokenizer` but with a simpler (and IMHO sounder) implementation, as well as a more complete and more general set of methods (see `AbstractStreamTokenizer`). In addition, `io.LAStreamTokenizer` provides a method for arbitrary lookaheads:

       public final String lookahead (int n) throws IOException

  which returns the string of `n` characters in the input stream including and starting with the current character.
  Version:
    Last modified on Wed Jun 20 13:57:23 2012 by hak
  Author:
    Hassan Aït-Kaci
  See Also:
    `AbstractStreamTokenizer`, `StreamTokenizer`
  Copyright:
    © by the author

- - ### Field Summary

    - ### Fields inherited from class hlt.language.io.AbstractStreamTokenizer

      `COMMENT, isInteger, NUMERIC, nval, ORDINARY, QUOTE, SPECIAL, sval, TT_EOF, TT_EOL, TT_NOTHING, TT_NUMBER, TT_SPECIAL, TT_WORD, ttype, WHITESPACE, WORD`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `LAStreamTokenizer(java.io.Reader rd)` Creates a `StreamTokenizer` for the specified `Reader` and sets the default syntax.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `commentBrackets(java.lang.String left, java.lang.String right)` Enables bracketed comments with the specified start and end strings.
    `void` | `commentChar(int c)` Makes the specified character a comment start character exclusively and enable skipping comment lines starting with this character.
    `void` | `commentLine(java.lang.String s)` Enables line comments with the specified start string.
    `void` | `disableComments()`
    `void` | `enableComments()`
    `java.lang.String` | `lookahead(int n)` Returns the string of n characters to be processed next, including and starting with the current character.
    `int` | `nextToken()` Reads a token and returns its type, which is also the value of `ttype`.
    `void` | `setEscapeChar(int c)` Sets the default escape character.
    `void` | `setTypePrecedence(int first, int second, int third, int forth)` Changes the precedence ordering among the character types to the order of the specified parameters.
    `void` | `slashSlashComments(boolean flag)` C++-style comments will be skipped iff the argument is `true`.
    `void` | `slashStarComments(boolean flag)` C-style comments will be skipped iff the argument is `true`.

    - ### Methods inherited from class hlt.language.io.AbstractStreamTokenizer

      `currentChar, eolIsSignificant, getFile, getLineNumber, ignoreNonDecimals, isNumericChar, isOrdinaryChar, isQuoteChar, isWhitespaceChar, isWordChar, lineno, location, ordinaryChar, ordinaryChars, ordinaryChars, parseDigitsOnly, parseNonDecimals, parseNonDecimals, parseNumbers, parseNumbers, parsingDigitsOnly, peek, peekBack, pushBack, quoteChar, quoteChar, quotePair, quotePair, resetSyntax, setDefaultSyntax, setOrdinaryChar, setOrdinaryChars, setQuoteChar, setQuoteChar, setQuotePair, setType, setType, setType, setWhitespaceChar, setWhitespaceChars, setWordChar, setWordChars, skipChar, skipChar, spaceIsSignificant, tokenEnd, tokenStart, toString, unsetType, unsetType, unsetType, whitespaceChar, whitespaceChars, whitespaceChars, wordChar, wordChars, wordChars`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Constructor Detail

    - #### LAStreamTokenizer

          public LAStreamTokenizer(java.io.Reader rd)

      Creates a `StreamTokenizer` for the specified `Reader` and sets the default syntax.

  - ### Method Detail

    - #### enableComments

          public final void enableComments()

      Specified by:
        `enableComments` in class `AbstractStreamTokenizer`

    - #### disableComments

          public final void disableComments()

      Specified by:
        `disableComments` in class `AbstractStreamTokenizer`

    - #### commentChar

          public final void commentChar(int c)

      Makes the specified character a comment start character exclusively and enable skipping comment lines starting with this character.
      Specified by:
        `commentChar` in class `AbstractStreamTokenizer`

    - #### slashSlashComments

          public final void slashSlashComments(boolean flag)

      C++-style comments will be skipped iff the argument is `true`.
      Specified by:
        `slashSlashComments` in class `AbstractStreamTokenizer`

    - #### slashStarComments

          public final void slashStarComments(boolean flag)

      C-style comments will be skipped iff the argument is `true`.
      Specified by:
        `slashStarComments` in class `AbstractStreamTokenizer`

    - #### nextToken

          public final int nextToken()
                              throws java.io.IOException

      Reads a token and returns its type, which is also the value of `ttype`. The possible token types are:
      `TT_NOTHING`
        if no token has been read yet;
      `TT_EOF`
        if the end of file has been read;
      `TT_EOL`
        if the end of line is significant and has been read;
      `TT_WORD`
        if a word has been read;
      `TT_NUMBER`
        if number parsing is effective and a number has been read;
      a quote character
        if a quoted word has been read;
      a white space
        if white spaces are significant and one has been read;
      an ordinary character
        otherwise.
      If the type is `TT_WORD` or a quote character, the value of the word read is in `sval`. If the type is `TT_NUMBER`, the value of the number is in `nval`. If the number read is an integer, the flag `isInteger` is set to `true`.
      Specified by:
        `nextToken` in class `AbstractStreamTokenizer`
      Throws:
        `java.io.IOException`

    - #### setEscapeChar

          public final void setEscapeChar(int c)

      Sets the default escape character.

    - #### commentBrackets

          public final void commentBrackets(java.lang.String left,
                             java.lang.String right)

      Enables bracketed comments with the specified start and end strings.

    - #### commentLine

          public final void commentLine(java.lang.String s)

      Enables line comments with the specified start string.

    - #### setTypePrecedence

          public void setTypePrecedence(int first,
                               int second,
                               int third,
                               int forth)
                                 throws java.lang.Exception

      Changes the precedence ordering among the character types to the order of the specified parameters. For example, the default ordering is set by:

      `setTypePrecedence(WORD,WHITESPACE,QUOTE,ORDINARY);`

      and this means that the `WORD` type is considered in priority, then `WHITESPACE`, etc...

      The `NUMERIC` type is not among those that can be ordered because it is relevant only when numbers are parsed. If they are, then the full Java number format is recognized independently of the character syntax setting. This means that `NUMERIC` has always highest precedence if number parsing is effective, and ignored otherwise.

      Throws:
        `java.lang.Exception`

    - #### lookahead

          public final java.lang.String lookahead(int n)
                                           throws java.io.IOException

      Returns the string of n characters to be processed next, including and starting with the current character.
      Throws:
        `java.io.IOException`