ANTLR3_TOKEN_SOURCE_struct - Online Linux Manual PageSection : 3
Updated : Tue Jan 26 2021
Source : Version 3.3.1
Note : ANTLR3C

NAMEANTLR3_TOKEN_SOURCE_struct − Definition of a token source, which has a pointer to a function that returns the next token (using a token factory if it is going to be efficient) and a pointer to an ANTLR3_INPUT_STREAM​.

SYNOPSIS
#include <antlr3tokenstream​.h>

Data FieldsANTLR3_COMMON_TOKEN eofToken
A special pre-allocated token, which signifies End Of Tokens​.
pANTLR3_STRING fileName
When the token source is constructed, it is populated with the file name from whence the tokens were produced by the lexer​.
pANTLR3_COMMON_TOKEN(* nextToken )(struct ANTLR3_TOKEN_SOURCE_struct *tokenSource)
Pointer to a function that returns the next token in the stream​.
ANTLR3_COMMON_TOKEN skipToken
A special pre-allocated token, which is returned by mTokens() if the lexer rule said to just skip the generated token altogether​.
pANTLR3_STRING_FACTORY strFactory
Whoever is providing tokens, needs to provide a string factory too​.
void * super
Whatever is supplying the token source interface, needs a pointer to itself so that this pointer can be passed to it when the nextToken function is called​.

Detailed DescriptionDefinition of a token source, which has a pointer to a function that returns the next token (using a token factory if it is going to be efficient) and a pointer to an ANTLR3_INPUT_STREAM​. This is slightly different to the Java interface because we have no way to implement multiple interfaces without defining them in the interface structure or casting (void *), which is too convoluted​.

Field Documentation

ANTLR3_COMMON_TOKEN ANTLR3_TOKEN_SOURCE_struct::eofTokenA special pre-allocated token, which signifies End Of Tokens​. Because this must be set up with the current input index and so on, we embed the structure and return the address of it​. It is marked as factoryMade, so that it is never attempted to be freed​. Referenced by antlr3LexerNew(), nextTokenStr(), setCharStream(), and tokLT()​.

pANTLR3_STRING ANTLR3_TOKEN_SOURCE_struct::fileNameWhen the token source is constructed, it is populated with the file name from whence the tokens were produced by the lexer​. This pointer is a copy of the one supplied by the CharStream (and may be NULL) so should not be manipulated other than to copy or print it​. Referenced by setCharStream()​.

pANTLR3_COMMON_TOKEN(* ANTLR3_TOKEN_SOURCE_struct::nextToken) (struct ANTLR3_TOKEN_SOURCE_struct *tokenSource)Pointer to a function that returns the next token in the stream​. Referenced by antlr3LexerNew(), and fillBuffer()​.

ANTLR3_COMMON_TOKEN ANTLR3_TOKEN_SOURCE_struct::skipTokenA special pre-allocated token, which is returned by mTokens() if the lexer rule said to just skip the generated token altogether​. Having this single token stops us wasting memory by have the token factory actually create something that we are going to SKIP(); anyway​. Referenced by antlr3LexerNew(), and nextTokenStr()​.

pANTLR3_STRING_FACTORY ANTLR3_TOKEN_SOURCE_struct::strFactoryWhoever is providing tokens, needs to provide a string factory too​. Referenced by antlr3LexerNew(), setCharStream(), and toStringSS()​.

void* ANTLR3_TOKEN_SOURCE_struct::superWhatever is supplying the token source interface, needs a pointer to itself so that this pointer can be passed to it when the nextToken function is called​. Referenced by antlr3LexerNew(), nextToken(), and nextTokenStr()​.

AuthorGenerated automatically by Doxygen for ANTLR3C from the source code​.
0
Johanes Gumabo
Data Size   :   12,781 byte
man-ANTLR3_TOKEN_SOURCE_struct.3Build   :   2024-12-05, 20:55   :  
Visitor Screen   :   x
Visitor Counter ( page / site )   :   2 / 175,795
Visitor ID   :     :  
Visitor IP   :   18.223.108.134   :  
Visitor Provider   :   AMAZON-02   :  
Provider Position ( lat x lon )   :   39.962500 x -83.006100   :   x
Provider Accuracy Radius ( km )   :   1000   :  
Provider City   :   Columbus   :  
Provider Province   :   Ohio ,   :   ,
Provider Country   :   United States   :  
Provider Continent   :   North America   :  
Visitor Recorder   :   Version   :  
Visitor Recorder   :   Library   :  
Online Linux Manual Page   :   Version   :   Online Linux Manual Page - Fedora.40 - march=x86-64 - mtune=generic - 24.12.05
Online Linux Manual Page   :   Library   :   lib_c - 24.10.03 - march=x86-64 - mtune=generic - Fedora.40
Online Linux Manual Page   :   Library   :   lib_m - 24.10.03 - march=x86-64 - mtune=generic - Fedora.40
Data Base   :   Version   :   Online Linux Manual Page Database - 24.04.13 - march=x86-64 - mtune=generic - fedora-38
Data Base   :   Library   :   lib_c - 23.02.07 - march=x86-64 - mtune=generic - fedora.36

Very long time ago, I have the best tutor, Wenzel Svojanovsky . If someone knows the email address of Wenzel Svojanovsky , please send an email to johanes_gumabo@yahoo.co.id .
If error, please print screen and send to johanes_gumabo@yahoo.co.id
Under development. Support me via PayPal.