TokenReader

java.lang.Object
- com.jiowa.template.reader.TokenReader

```
public class TokenReader
extends java.lang.Object
```
Reads token after token from a string template according to a given template notation. Note, that simple text is also considered as so-called text token, i.e. this is the text from a 'closing' token ('>>') (or the beginning of the text) to an 'opening' token ('<<').

Author:

Robert Mencl
Copyright (C) 2012-2016 JIOWA Business Solutions GmbH, Bettinastr. 30 , 60325 Frankfurt am Main, Germany
Homepage: www.jiowa.de
All rights reserved.
Download: www.jiowa.de/download.html
License: www.jiowa.de/license.html

Field Summary

Fields
Modifier and Type	Field and Description
`protected TagStartPositionAdjuster`	`adjuster`
`protected java.lang.StringBuffer`	`buf`
`protected SecondLevelTokenBuilder`	`builder`
`protected TemplateContext`	`context`
`protected ErrorPosition`	`errorposition`
`protected boolean`	`isExternalTemplate`
`protected int`	`pos`
`protected java.util.Queue<Token>`	`queue`
`protected RedefinitionContentToken`	`redefinitionContent`
`protected RedefinitionTokenList`	`redefTokenList`
`protected TokenStateMachine`	`statemachine`
`protected TextPosition`	`textposition`
`protected java.util.Iterator<Token>`	`tokenIterator`
`protected java.util.List<Token>`	`tokenList` The tokenList will be delivered in the same way as tokens from a string.

Constructor Summary

Constructors
Constructor and Description
`TokenReader(boolean isExternalTemplate, TemplateContext context, ErrorPosition errorposition)` Constructor for a given template notation and a template as simple string.
`TokenReader(TemplateContext context, ErrorPosition errorposition)` Constructor for a given template notation and a template as simple string.
`TokenReader(TemplateContext context, java.util.List<Token> tokenList, ErrorPosition errorposition)` Special constructor.
`TokenReader(TemplateContext context, TokenStateMachine statemachineWithGivenState, ErrorPosition errorposition)` Constructor with a given statemachine (including current state and final state settings).

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`protected java.util.List<Token>`	`buildAdditionalTokens(Token nextToken, java.lang.String s)` This methods builds additional tokens that are sometimes present between two structural tokens which haven been taken as states for the tokenization process.
`protected TextToken`	`buildTextToken(java.lang.String s, TextPosition textposition)` Builds the text token from a string (and removes all quoting characters) and updates the text position.
`protected void`	`changeTemplateNotation()` Changes the template notation according to internatl redefinition content.
`protected void`	`changeTemplateNotation(TemplateNotation newNotation)` Changes the template notation according to the given new template notation.
`protected void`	`checkForEmptyString(java.lang.String s)` Checks for an empty string and logs appropriate error messages
`protected void`	`deliverRedefTokenList()` Delivers the redefinition token list to the `TemplateMapLoader` via the `TemplateContext`.
`Token`	`read()` Reads the next token from the template string (and remembers the current state of the reader).
`protected Token`	`read(java.util.SortedSet<SymbolToken> set)` Returns the next token to be read.
`java.util.List<Token>`	`readAll()` Reads all possible tokens at once.
`protected Token`	`readNextToken()` Reads the next token from the text string.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - isExternalTemplate
```
protected boolean isExternalTemplate
```
  - context
```
protected TemplateContext context
```
  - statemachine
```
protected TokenStateMachine statemachine
```
  - buf
```
protected java.lang.StringBuffer buf
```
  - pos
```
protected int pos
```
  - queue
```
protected java.util.Queue<Token> queue
```
  - textposition
```
protected TextPosition textposition
```
  - errorposition
```
protected ErrorPosition errorposition
```
  - redefinitionContent
```
protected RedefinitionContentToken redefinitionContent
```
  - builder
```
protected SecondLevelTokenBuilder builder
```
  - adjuster
```
protected TagStartPositionAdjuster adjuster
```
  - tokenList
```
protected java.util.List<Token> tokenList
```
    The tokenList will be delivered in the same way as tokens from a string.
  - tokenIterator
```
protected java.util.Iterator<Token> tokenIterator
```
  - redefTokenList
```
protected RedefinitionTokenList redefTokenList
```
- Constructor Detail
  - TokenReader
```
public TokenReader(boolean isExternalTemplate,
                   TemplateContext context,
                   ErrorPosition errorposition)
```
    Constructor for a given template notation and a template as simple string.
    
    Parameters:
    
    isExternalTemplate - true/false if this template is an external template, i.e. it is not an inline sub template
    
    context - the template context
    
    errorposition - the current text position for errors messages
  - TokenReader
```
public TokenReader(TemplateContext context,
                   ErrorPosition errorposition)
```
    Constructor for a given template notation and a template as simple string.
    
    Parameters:
    
    context - the template context
    
    errorposition - the current text position for errors messages
  - TokenReader
```
public TokenReader(TemplateContext context,
                   TokenStateMachine statemachineWithGivenState,
                   ErrorPosition errorposition)
```
    Constructor with a given statemachine (including current state and final state settings).
    
    Parameters:
    
    context - the template context
    
    statemachineWithGivenState - a TokenStateMachine with a given state for the parsing process
    
    errorposition - the current text position for errors messages
  - TokenReader
```
public TokenReader(TemplateContext context,
                   java.util.List<Token> tokenList,
                   ErrorPosition errorposition)
```
    Special constructor. Then, read() delivers the tokenList in the same way as a reader would do it with a template string.
    
    Parameters:
    
    context - the template context
    
    tokenList - the list of tokens previously read
    
    errorposition - the current text position for errors messages
- Method Detail
  - read
```
public Token read()
```
    Reads the next token from the template string (and remembers the current state of the reader).
    
    Returns:
    
    the current Token
  - readAll
```
public java.util.List<Token> readAll()
```
    Reads all possible tokens at once.
    
    Returns:
    
    list of all tokens which reprent this template
  - changeTemplateNotation
```
protected void changeTemplateNotation()
```
    Changes the template notation according to internatl redefinition content.
  - changeTemplateNotation
```
protected void changeTemplateNotation(TemplateNotation newNotation)
```
    Changes the template notation according to the given new template notation.
    
    Parameters:
    
    newNotation - the new template notation
  - readNextToken
```
protected Token readNextToken()
```
    Reads the next token from the text string.
    
    Returns:
    
    the next token
  - read
```
protected Token read(java.util.SortedSet<SymbolToken> set)
```
    Returns the next token to be read.
    
    Parameters:
    
    set - the set of possible SymbolTokens which could appear after the current text position
    
    Returns:
    
    the next token to the current text position
  - buildTextToken
```
protected TextToken buildTextToken(java.lang.String s,
                                   TextPosition textposition)
```
    Builds the text token from a string (and removes all quoting characters) and updates the text position.
    
    Parameters:
    
    s - text content
    
    textposition - current text position
    
    Returns:
    
    text token
  - buildAdditionalTokens
```
protected java.util.List<Token> buildAdditionalTokens(Token nextToken,
                                                      java.lang.String s)
```
    This methods builds additional tokens that are sometimes present between two structural tokens which haven been taken as states for the tokenization process. Example: Between a tag start token and a tag end token there is a variable token. More token variations are computed within this method.
  - deliverRedefTokenList
```
protected void deliverRedefTokenList()
```
    Delivers the redefinition token list to the TemplateMapLoader via the TemplateContext.
  - checkForEmptyString
```
protected void checkForEmptyString(java.lang.String s)
```
    Checks for an empty string and logs appropriate error messages
    
    Parameters:
    
    s - the string

Class TokenReader

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

isExternalTemplate

context

statemachine

buf

pos

queue

textposition

errorposition

redefinitionContent

builder

adjuster

tokenList

tokenIterator

redefTokenList

Constructor Detail

TokenReader

TokenReader

TokenReader

TokenReader

Method Detail

read

readAll

changeTemplateNotation

changeTemplateNotation

readNextToken

read

buildTextToken

buildAdditionalTokens

deliverRedefTokenList

checkForEmptyString