StreamTokenizer (API documentation of hlt.language packages)
Date: 2013-01-30

hlt.language.io

## Class StreamTokenizer

- java.lang.Object
- - hlt.language.io.AbstractStreamTokenizer
  - - hlt.language.io.StreamTokenizer

-     public class StreamTokenizer
      extends AbstractStreamTokenizer

  This provides the exact same interface as that of the standard Java Core API's `java.io.StreamTokenizer` but with a simpler (and IMHO sounder) implementation, as well as a more complete and more general set of methods (see `AbstractStreamTokenizer`).
  Version:
    Last modified on Mon Nov 26 18:30:39 2012 by hak
  Author:
    Hassan Aït-Kaci
  See Also:
    `AbstractStreamTokenizer`, `LAStreamTokenizer`
  Copyright:
    © by the author

- - ### Field Summary

    - ### Fields inherited from class hlt.language.io.AbstractStreamTokenizer

      `COMMENT, isInteger, NUMERIC, nval, ORDINARY, QUOTE, SPECIAL, sval, TT_EOF, TT_EOL, TT_NOTHING, TT_NUMBER, TT_SPECIAL, TT_WORD, ttype, WHITESPACE, WORD`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `StreamTokenizer(java.io.Reader rd)` Creates a `StreamTokenizer` for the specified `Reader` and sets the default syntax.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `commentChar(int c)` Makes the specified character a comment start character exclusively.
    `void` | `disableComments()`
    `void` | `enableComments()`
    `boolean` | `isCommentChar(int c)` Checks whether this character's type is comment, among other types.
    `int` | `nextToken()` Reads a token and returns its type, which is also the value of `ttype`.
    `void` | `setEscapeChar(int c)` Sets the default escape character.
    `void` | `setTypePrecedence(int first, int second, int third, int forth, int fifth)` Changes the precedence ordering among the character types to the order of the specified parameters.
    `void` | `slashSlashComments(boolean flag)` C++-style comments will be skipped iff the argument is `true`.
    `void` | `slashStarComments(boolean flag)` C-style comments will be skipped iff the argument is `true`.

    - ### Methods inherited from class hlt.language.io.AbstractStreamTokenizer

      `currentChar, eolIsSignificant, getFile, getLineNumber, ignoreNonDecimals, isNumericChar, isOrdinaryChar, isQuoteChar, isWhitespaceChar, isWordChar, lineno, location, ordinaryChar, ordinaryChars, ordinaryChars, parseDigitsOnly, parseNonDecimals, parseNonDecimals, parseNumbers, parseNumbers, parsingDigitsOnly, peek, peekBack, pushBack, quoteChar, quoteChar, quotePair, quotePair, resetSyntax, setDefaultSyntax, setOrdinaryChar, setOrdinaryChars, setQuoteChar, setQuoteChar, setQuotePair, setType, setType, setType, setWhitespaceChar, setWhitespaceChars, setWordChar, setWordChars, skipChar, skipChar, spaceIsSignificant, tokenEnd, tokenStart, toString, unsetType, unsetType, unsetType, whitespaceChar, whitespaceChars, whitespaceChars, wordChar, wordChars, wordChars`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Constructor Detail

    - #### StreamTokenizer

          public StreamTokenizer(java.io.Reader rd)

      Creates a `StreamTokenizer` for the specified `Reader` and sets the default syntax.

  - ### Method Detail

    - #### enableComments

          public final void enableComments()

      Specified by:
        `enableComments` in class `AbstractStreamTokenizer`

    - #### disableComments

          public final void disableComments()

      Specified by:
        `disableComments` in class `AbstractStreamTokenizer`

    - #### commentChar

          public final void commentChar(int c)

      Makes the specified character a comment start character exclusively.
      Specified by:
        `commentChar` in class `AbstractStreamTokenizer`

    - #### slashSlashComments

          public final void slashSlashComments(boolean flag)

      C++-style comments will be skipped iff the argument is `true`.
      Specified by:
        `slashSlashComments` in class `AbstractStreamTokenizer`

    - #### slashStarComments

          public final void slashStarComments(boolean flag)

      C-style comments will be skipped iff the argument is `true`.
      Specified by:
        `slashStarComments` in class `AbstractStreamTokenizer`

    - #### nextToken

          public final int nextToken()
                              throws java.io.IOException

      Reads a token and returns its type, which is also the value of `ttype`. The possible token types are:
      `TT_NOTHING`
        if no token has been read yet;
      `TT_EOF`
        if the end of file has been read;
      `TT_EOL`
        if the end of line is significant and has been read;
      `TT_WORD`
        if a word has been read;
      `TT_NUMBER`
        if number parsing is effective and a number has been read;
      a quote character
        if a quoted word has been read;
      a white space
        if white spaces are significant and one has been read;
      an ordinary character
        otherwise.

      If the type is `TT_WORD` or a quote character, the value of the word read is in `sval`. If the type is `TT_NUMBER`, the value of the number is in `nval`. If the number read is an integer, the flag `isInteger` is set to `true`.

      Specified by:
        `nextToken` in class `AbstractStreamTokenizer`
      Throws:
        `java.io.IOException`

    - #### isCommentChar

          public final boolean isCommentChar(int c)

      Checks whether this character's type is comment, among other types.

    - #### setEscapeChar

          public final void setEscapeChar(int c)

      Sets the default escape character.

    - #### setTypePrecedence

          public void setTypePrecedence(int first,
                               int second,
                               int third,
                               int forth,
                               int fifth)
                                 throws java.lang.Exception

      Changes the precedence ordering among the character types to the order of the specified parameters. For example, the default ordering is set by:

      `setTypePrecedence(WORD,WHITESPACE,COMMENT,QUOTE,ORDINARY);`

      and this means that the `WORD` type is considered in priority, then `WHITESPACE`, etc...

      The `NUMERIC` type is not among those that can be ordered because it is relevant only when numbers are parsed. If they are, then the full Java number format is recognized independently of the character syntax setting. This means that `NUMERIC` has always highest precedence if number parsing is effective, and ignored otherwise.

      Throws:
        `java.lang.Exception`