Associate an attribute with an C-object defined in the META-HEADER …

ALC PARSER EXTENSION

The goal of the parser-extension is to associate an ATTRIBUTE with an existing META-HEADER in order to connect a C-library with a Target-Programming-Language (TPL).

The parser-extension is a part of the All-Language-Compiler (ALC).
The All-Language-Compiler (ALC) is an open-compiler-tool easy to extend with an additional feature.

The parser-extension require :

META-HEADER

The external-header-file is the source of the API information and the goal of the Programming-Language-Micro-Kernel (PLMK) in conjunction with the All-Language-Compiler (ALC) is to use this file to provide language binding for a variety of languages.

META-WRAPPER

If an external-header-file cannot be changed, it is a good idea to call this file via a wrapper and then add the changes using __parser__global__(...) .

META-PARSER

The meta-parser is a C compliant extension to the C programming language to associate an attribute with an C-object-definition (e.g. function, struct, enum, argument etc.)

META-COMPILER

The meta-compiler is the tool to generate the meta-file from the meta-header and the meta-parser.

The parser-extension create :

META-FILE

The meta-file is the text-based database with TCL syntax in which all attributes that define a library are stored and which is generated by the All-Language-Compiler (ALC) frontend and used by the All-Language-Compiler (ALC) backend.

META-LIBRARY

The meta-library is created by the meta-compiler and is a meta-code-wrapper of the native-library.

The meta-library uses only the meta-data-types and the meta-code-structure.

The native-meta-library is the link between the meta-library and the TPL, with the connection being established via the MOT.

META-HEADER

meta-code-external-header-file (META-HEADER)

The external-header-file is the source of the API information and the goal of the Programming-Language-Micro-Kernel (PLMK) in conjunction with the All-Language-Compiler (ALC) is to use this file to provide language binding for a variety of languages.

The meta-header is divided into two variants :

the programmer is the owner of the header-file and can therefore change this file.
- no META-WRAPPER is required.
the programmer is the user of the header-file and can not change this file without creating a fork.
- META-WRAPPER is required.

META-WRAPPER

meta-code-header-wrapper-file (META-WRAPPER)

If an external-header-file cannot be changed, it is a good idea to call this file via a wrapper and then add the changes using __parser__global__(...) .

Example: default-value

The default-value of a function argument is not defined in C because the C language does not support an argument-default-value.

However, many other languages support the default-value and so the task of the META-PARSER technology is to connect this attribute to a function argument.

Two scenarios need to be distinguished:

The developer "owns" the META-HEADER and can therefore modify this file to suit his needs:
extern int dummy (

char* ident __parser__(default="otto")

)

→ this is called a local-parser modification
The developer "uses" the META-HEADER and a direct modification would lead to a code fork, which should generally be avoided:
- META-HEADER : someLib.h
  extern int dummy (
  
  char* ident
  
  )
- META-WRAPPER : meta_someLib.h
  #include "someLib.h"
  
  ...
  
  __parser__global__(dummy:ident:default="otto");
→ this is called a global-parser modification

META-FILE

meta-code-library-definition-file (META-FILE)

The meta-file is the text-based database with TCL syntax in which all attributes that define a library are stored and which is generated by the All-Language-Compiler (ALC) frontend and used by the All-Language-Compiler (ALC) backend.

The meta-file is created by the meta-code-compiler (META-COMPILER) using the meta-code-parser-extension (META-PARSER), meta-code-external-header-file (META-HEADER) and optional the meta-code-header-wrapper-file (META-WRAPPER) as input.

The meta-file is …

… the meta-code-api-definition.
… a machine and human readable database-file useable to be added into the code-repository
… the stabe meta-api-reference and is suitable for checkng into the source-code-repository.
… used by all All-Language-Compiler (ALC) backend tools to extract the information needed to perform the required task.
… modified if the All-Language-Compiler (ALC) or the native-library API changes.
… the code repository that gives the programmer a good overview of the changes in the API.

The encoding of the meta-file is defined as :

utf8 text file formatted as TCL source-code with multiple TCL procs

The syntax of the meta-file is defined as :

namespace eval meta_writer {
##
##  ME_(PCANRVE)      FIRST: P=pointer, C=class, A=array, N=Native, R=reference, V=vararg, E(NE)=enum
##
##  ME_P*             The "pointer" data type
##  ME_P(VSBLFECA)    V=void, S=string, B=binary, L=list, F=format, E=exception, C=constructor, A=callback
##  ME_P.(BNX)        B=basicType, N=const, X=unrecognized
##
  array set typeDoc {
    me-name {short-name definition description} ...
  }
##
##  ME_C*             The "class" data type
##  ME_C(CX)          C=class, X=special
##  ME_C.(CN)         C=class, N=const
##
  array set typeDoc { ... }
##
##  ME_A*             The "array" data type
##  ME_A(SCX)         S=string, C=class, X=unrecognized
##  ME_A.(NX)         N=const, X=unrecognized
##
  array set typeDoc { ... }
##
##  ME_N*             The "native" data type
##  ME_N(BIF)         B=bool, I=integer, U=Unsigned, F=float
##  ME_N.(#XL)        #=#bytes, X=longlong, L=long
##
  array set typeDoc { ... }
##
##  ME_R*             The "reference" data type
##  ME_R(SCU)         S=string, C=class, U=union
##  ME_R.(NA~)        S=string, A=unrecognized, ~=unrecognized
  array set typeDoc { ... }
##
##  ME_V*             The "variable argument list" data type
##  ME_V(ACL)         A=..., C=string, L=va_list
##  ME_V.(NV)         N=const, V=unrecognized
##
  array set typeDoc { ... }
##
## additional modifer definition :
##
##   %S = struct, %U = union, %E = enum, %G = unsigned, %N/M = const, %P = ptr
##
}

# library definition
headerDEF library     value
headerDEF header      value
headerDEF doc-force   value
headerDEF doc-db      value ...
headerDEF §END§

# attribute for function, function pointer, class and enum
attributeDEF attribute value ...
attributeDEF §END§

# package definition
packageDEF default Value
packageDEF §END§

# used names definition
nameDEF name TYP/DEF definition extra ...
nameDEF §END§

# types definition
typeDEF type me-name definition extra ...
typeDEF §END§

# ignored names definition
ignoreDEF name 1 ...
ignoreDEF §END§

# enum definition
enumDEF enum vals ...
enumDEF §END§

# class definition
classDEF class me-name short-name ...
classDEF §END§

# function pointer definition
fupuDEF fupu return args ...
fupuDEF §END§

# function definition
funcDEF func return args ...
funcDEF §END§

Example: attributeDEF

The attributeDEF is part of the META-FILE and is created by the META-COMPILER using the META-PARSER.

The syntax of the attributeDEF is defined as :

attributeDEF function,attribute yes // boolean attribute

attributeDEF function,attribute value // value attribute

attributeDEF function,argument,attribute value // function argument value attribute

attributeDEF attribute,arg1,arg2 value // global value attribute with 2 additional args

To access the boolean-attribute "myattr" the following workflow is used:

add the "myattr" attribute to a function
__parser__(myattr) // add the "myattr" attribute to function "someFunc"

void someFunc (void) {

...

}

__parser__
#define __parser__(...)
in the META-FILE the attributeDEF is created
attributeDEF someFunc,myattr yes
in the backend-meta-file-loader (tcl) the attributeDEF is mapped to attributeDEF-array
set attributeDEF(someFunc,myattr) yes
in the backed-tool-library (tcl) the attribute-access-proc is defined
proc tcl::mathfunc::myattrExists { func } {

info exists ::attributeDEF($func,myattr)

}
in the backed-tool (tcl) the loop over all functions can act on the "myattr" attribute
# pseudo code

foreach name $allfunctions {

if {myattrExists($name)} {

... do something

}

}

META-COMPILER

meta-code-compiler (META-COMPILER)

The meta-compiler is the tool to generate the meta-file from the meta-header and the meta-parser.

The default meta-compiler tool is called c_meta.tcl and read the META-HEADER and the META-WRAPPER to create the META-FILE.

read more at : META-PARSER

The meta-compiler has three main components:

configuration and calling of the C-precessor
evaluation of the C-preprocessor output and construction of the data structures
saving the data structures in a META-FILE

In addition to the default meta-compiler, there are also a Target-Programming-Language (TPL) specific meta-compiler (LNG_meta.tcl) whose task is to convert the META-FILE into a language-specific META-FILE (API transformation) in order to simplify the logic in the All-Language-Compiler (ALC) backend tool(s).

META-LIBRARY

meta-code-library (META-LIBRARY)

The meta-library is created by the meta-compiler and is a meta-code-wrapper of the native-library.

The meta-library uses only the meta-data-types and the meta-code-structure.

The native-meta-library is the link between the meta-library and the TPL, with the connection being established via the MOT.

Example: the filesystem of theConfig/c :

The meta-library uses a filesystem-layout to integrate into the nhi1-development-framework.
The code-file-names are defined by the class-tool (c_Class.tcl) and used by the All-Language-Compiler (ALC).
Other development tools like: Shortcut-Tool, Label-Tool or Index-Tool also add requirements to the filesystem-lyout.

// Index-Tool
.indexignore                          // files ignored
.index.mk                             // file created
.makefile_index.bash                  // configuration
tags                                  // file created

// Label-Tool
.label.h                              // template for *.h files
.labelignore                          // files ignored
.label_inc.tcl                        // configration

// Shortcut-Tool
.shortcut                             // created "vim" navigation shortcuts
.shortcutignore                       // files ignored

// ALC compiler
compiler.mk                           // configuration of the makefile
compiler.tcl                          // configuration of the compiler
Makefile.am                           // makefile

// package LcConfig
LibLcConfig_lc.c                      // code …
LibLcConfig_lc.h                      // 
LcEnum_lc.h                           // enum definition (part of the package)
private_lc.h                          // private definitions

// meta code wrapper
config_lc.h                           // configuration
debug_lc.h                            // debugging

LcConfigC_def_lc.h                    // class LcConfigC
LcConfigC_lc.c
LcConfigC_lc.h

LcSettingC_def_lc.h                   // class LcSettingC
LcSettingC_lc.c
LcSettingC_lc.h

// libconfig documentation
libconfig.texi                        // texinfo documentation file
libconfig_2_docdb.tcl                 // texinfo documentation parser -> documentation database
gen/doc_db.dir                        // documentation database

gen                                   // directory for generated code
  nat_lcconfig.meta                   // native-meta-file
  nat_lcconfig.index

  c_lcconfig.meta                     // c-meta-file
  c_lcconfig.index
  config_inline_lc.c                  // c-meta-library inline2function code
  config_inline_lc.h
  config_overload_lc.h                // c-meta-library function-overload code (default, runtime, error etc)

  LcConfigC_lc.doc                    // library documentation
  LcSettingC_lc.doc
  LibLcConfig_lc.doc

tmpl                                  // directory for "pattern" code
  lc_misc_check_lng.h                 // generic class check code
  lc_type_S_macros.h                  // generic type check code

tests                                 // directory for tests

Example: the workflow of theConfig/py :

Process to create the META-LIBRARY and python-library from the META-HEADER and the native-library :

The C makefile                  :   theConfig/c/Makefile.am
  and the libconfig header      :   theConfig/c/libconfig.h
  and the meta-wrapper          :   theConfig/c/config_lc.h
  and the meta-compiler         :   theCompiler/src/c_Meta.tcl
→ create the nat-meta-file      :   theConfig/c/gen/nat_lcconfig.meta
→ create the meta-file          :   theConfig/c/gen/c_lcconfig.meta

The C makefile                  :   theConfig/c/Makefile.am
  and the native-library        :   ext/x86_64-suse-linux-gnu/debug/lib64/libconfig.la
  and the ALC native-backend    :   theCompiler/src/c_Native*.tcl
  and the nat-meta-file         :   theConfig/c/gen/nat_lcconfig.meta
  and the C source files        :   theConfig/c/*Lc*_lc.c/h
  and the gcc compiler          :   gcc-13
→ create the meta-library       :   build/x86_64-suse-linux-gnu/debug/theConfig/c/liblcconfig.la

The python makefile             :   theConfig/py/Makefile.am
  and the meta-library          :   build/x86_64-suse-linux-gnu/debug/theConfig/c/liblcconfig.la
  and the python-backend        :   theCompiler/src/py_XYZ.tcl
  and the meta-file             :   theConfig/c/gen/c_lcconfig.meta
  and the python source files   :   theConfig/py/*Lc*_py.c/h
  and the gcc compiler          :   gcc-13
→ create the TPL library        :   build/x86_64-suse-linux-gnu/debug/theConfig/py/pylcconfig.la

META-PARSER

meta-code-parser-extension (META-PARSER)

The meta-parser is a C compliant extension to the C programming language to associate an attribute with an C-object-definition (e.g. function, struct, enum, argument etc.)

The meta-parser uses the C-preprocessor-macro to modify a given C-header-file without changing the C syntax.

All macros start with __parser__ and are inactive during C compilation with #define __parser__xyz(...).
The macros only become active when the C preprocessor macro __has_parser__ has been defined.

The technology behind it is simple.

The META-COMPILER uses the C-preprocessor with active macro __has_parser__ to compile the C-header-file, with the result then being a simplified form of the original C-header-file.
This input is then further processed by the META-COMPILER, with the __parser__xyz being read and processed.

The meta-parser-extension is defined as :

__parser__(...)
__parser__global__(...)
__parser__push__(...)
__parser__define__(...)

The parser-code-extension is …

a c-code-extension and a c-preprocessor-extension.
used as the __parser__(...) and __parser__global__(...)
used by the All-Language-Compiler (ALC) to create the META-FILE.

The c-code-extension is …

full-compatible with the c-language-definition defined at https://www.iso-9899.info/wiki/The_Standard.
a c-preprocessor-definition without affecting the c-language.
an attribute-syntax for the META-HEADER to associate a meta-parser-attribute with a c-language-definition.
used by the All-Language-Compiler (ALC) to create the META-FILE .

The c-preprocessor-extension is …

a set of macros defined as __parser__???(…) and evaluate to nothing if the source is compiled without the __has_parser__ compile flag defined.
#ifndef __has_parser__

# define __parser__(...)

# define __parser__define__(...)

# define __parser__push__(...)

# define __parser__pop__

// END: local attributes

# define __parser__global__(...)

# define __parser__global__define__(...)

# define __parser__global__push__(...)

# define __parser__global__pop__

// END: global attributes

#else

# include "parser_mk.h"

#endif

// END: __has_parser__
used by the All-Language-Compiler (ALC) tool c_meta.tcl if the c-preprocessor flag __has_parser__ is set.

parser(...)

parser: add an local attribute …

The local-code-extension is related to the local-attribute :

The __parser__ feature is defined at : META-PARSER and META-FILE

The following __parser__ definitions are supported :

__parser__global__(ATTRIBUTE=VALUE?@VALUE?…);
__parser__global__(OBJECT:ATTRIBUTE=VALUE?@VALUE?…);
__parser__global__(FUNCTION:ARGUMENT:ATTRIBUTE=VALUE?@VALUE?…);
__parser__(ATTRIBUTE?=VALUE?, ATTRIBUTE?=VALUE?, …) OBJECT …
FUNCTION ( type ARGUMENT __parser__(ATTRIBUTE?=VALUE?, ATTRIBUTE?=VALUE?, …), ... );

The OBJECT can be a FUNCTION, an ENUM, a STRUCT or an other unique named OBJECT.
Every VALUE not added is set to yes by default.
- same: myfunc( type1 arg1 __parser__(internal), ...)
- same: myfunc( type1 arg1 __parser__(internal=yes), ...)
Every local-attribute can be added using the __parser__global__ if the object is added in front:
- local: __parser__(alias=XYZ) myfunc(...)
- global: __parser__global__(myfunc:alias=XYZ);
Every global-attribute can only be added using the __parser__global__ definition:
- global: __parser__global__define__(pattern2prefix:PATTERN=PREFIX);
Every parser__global have to end with a ";" character like normal C code.
- _parser__global__(prefix2class:MkRuntime=MkRuntimeC);
Multiple local-attributes can be put into one parser definition using the "," character as seperator.
- multiple: myfunc( type1 arg1 __parser__(internal=yes,default=NULL), ...)
An attribute may contain a ":" character to support a path-like depth.
- _parser__global__(prefix2class:MkRuntime=MkRuntimeC);
Multiple VALUES are linked together with the "@" character.
- __parser__(library=Sq3Lite@native@Sq3@SqLite3)

parserglobal__(...)

parser: add an global attribute …

The global-code-extension is related to the global-attribute :

The __parser__global__ feature is defined at : META-PARSER and META-FILE

The following __parser__global__ definitions are supported :

__parser__global__(ATTRIBUTE=VALUE?@VALUE?…);
__parser__global__(OBJECT:ATTRIBUTE=VALUE?@VALUE?…);
__parser__global__(FUNCTION:ARGUMENT:ATTRIBUTE=VALUE?@VALUE?…);

The OBJECT can be a FUNCTION, a FUPU, an ENUM, a STRUCT or an other unique named OBJECT.
Every __parser__global__(…) have to end with a ";" character like normal "C" code.
Every VALUE not added is set to yes by default.
Only one definition can be put into one __parser__global__ definition.
A Definition may contain a ":" character to support a tree-like depth.
A List of values are linked together with the "@" character.

parserpush__(...)

The special keywords:

__parser__push__(ATTRIBUTE?=VALUE?, ATTRIBUTE?=VALUE?, …); and
__parser__pop__;

are used to add a ATTRIBUTE to a block-of-code.

attention: The __parser__push__ and __parser__pop__ must not be nested

Example

Ignore the doc-index from the __parser__push__ block definition

__parser__push__(doc-index=BlaBla);
 ... some functions
   __parser__(doc-index=_ignore_)    // ignore the 'BlaBla' for function 'my_special_func'
   void my_special_func(…)
 ... some functions
__parser__pop__;

parserdefine__(...)

An ATTRIBUTE is defined with __parser__define__ or __parser__global__define__

A list of predifined PARSER Attributes is available at theKernel/c/LibMkKernel_mk.h
Every ATTRIBUTE used and not defined generate a warning in META-COMPILER
Using an ATTRIBUTE without a definition is possible during development, but is not recommended for production code due to the lack of documentation of the ATTRIBUTE.
It is always possible to add an additional ATTRIBUTE in the local C-header-file .

The following syntay is defined :

The VALUE is defined as plain string without ":" character, the "@" character is used to join multiple VALUES into one string.
A PATH and a ATTRIBUTE is connected using the ":" character.
The ATTRIBUTE is always behind the PATH.
The ATTRIBUTE and a VALUE are connected using the "=" character.
If no VALUE is defined the "=" character can be skipped and the default VALUE is yes.
The special VALUE _ignore_ is used to ignore the ATTRIBUTE.

Example: A ATTRIBUTE with multiple VALUEs -
__parser__global__(PATH1:PATH2:...:ATTRIBUTE=VALUE1@VALUE2@...);

EXAMPLE: parser

This is an example from the theKernel/c/kernel_mk.h file and define the BufferLog slot of the MkBufferC class.

__parser__(const,flags=new)
MK_EXTERN void MK_DECL MkBufferLog (
  MK_BUFN const buf,
  MK_OBJN       fmtobj    __parser__(default=NULL),
  MK_DBG const  debug     __parser__(default=0),
  MK_STRN const callfunc  __parser__(default=F#FUNC),
  MK_I32 const  lvl       __parser__(default=0)
);

1. map to class MkBufferC: The prefix MkBuffer together with the first parameter MK_BUFN indicate the class MkBufferC.
If the indication is not possible the meta-code-parser-extension (META-PARSER) attribute "class=MkBufferC" force the indication

2. analyse the function-attribute: The function-attributes const,flags=new define two attributes separated by ",". The first attribute const is a boolean-attribute and the second attribute flags=new is a value-attribute.

3. analyse the arguments-attribute(s): The first argument buf is the class-hdl. All other arguments have a default-value defined with the default=VALUE attribute.

4. META-PARSER and attribute documentation

The usage of the meta-code-parser-extension (META-PARSER) extension und the already defined parser-attribute are documented at the PARSER API

example: const , flags , default

back

theCompiler