MkKernel_Storage_C_API

MkKernel PACKAGE - Storage Management …

Collaboration diagram for MkKernel_Storage_C_API:

MkKernel PACKAGE - Storage Management …

Storage management is used in ccmkkernel to provide temporary storage. It is a common design pattern that ccmkkernel only returns a reference to the Internal-Temporary-Storage (ITS), so the Internal-Active-Storage (IAS) is not returned to the external end user. The ITS is a storage that is only used as a return value and nothing else. The temporary in ITS refers exclusively to the current state of the storage and not to the lifespan of the storage, the ITS is only allocated once at startup and then used again and again, similar to the static storage in C.

Internal ccmkkernel distinguishes three different storage sources:

Context-Local-Storage (CLS)

CLS is tied to a specific MqContextC.
Example: the MqReadBUF returns a reference to an internal MkBufferC.

Funktion-Local-Storage (FLS)

FLS is used as the local temporary storage, usually as thread-local-storage, of a function-return-value.
Example: the MqReadBFL returns a MkBufferListC which is filled with multiple MkBufferC.

Runtime-Local-Storage (RLS)

RLS is used as global storage per RunTime instance.
Example: the MkErrorC only exists ONCE per runtime.

The CLS and FLS have the same visibility to the end user and are explained together as FLS.
The RLS is not mentioned in this documentation section because the RLS is more internal than CLS and FLS.

The end-user uses a FLS reference like a normal local C++ variable but with the following restriction:

1. The value of the variable is a reference to the FLS storage belonging to the method that returned the reference.
2. A FLS storage only ever exists once in a thread, which means that the FLS storage of a reference is overwritten if the FLS storage is used a second time in the same context.
3. A context is, for example, a coherent block of code such as in a "service callback". A coherent context is broken if the same method that returned the original FLS as a result is called a second time or if a method is called that uses the "event loop".
4. FLS storage must NOT be released by the end user, the Programming-Language-Micro-Kernel (PLMK) always ensures that the storage management of ccmkkernel and the target-language is synchronized.
5. If a FLS reference is added to another reference and this reference is also managed by the Programming-Language-Micro-Kernel (PLMK), the Programming-Language-Micro-Kernel (PLMK) automatically ensures that the storage management is coherent, which means that the end user does not have to do anything.
6. The FLS reference can be updated. This means that the FLS storage is being updated because the reference owner (usually a local variable) temporarily owns the FLS storage.

The "Dup" (duplicate) function is used to convert a temporary FLS variable into a global storage. The global storage is managed by the end user and may have to be released depending on the target programming language.

Example from server.cc → "ReadBFL" overwrite previous "ReadBFL"

      void BFL2() {
        auto tmp1 = ReadBFL()                     ; // "tmp1" is now a reference to the FLS storage of "ReadBFL"
        auto tmp2 = ReadBFL()                     ; // ERROR: the "tmp2" is using a SHARED reference with "tmp1"
        Send("R","LL",tmp1,tmp2)                ; // ERROR: "tmp1" and "tmp2" are the SAME values
      }

Example from server.cc → "ReadBFL" overwrite previous "ReadBFL" even in an "Event-Loop"

      void pBFL3() {
        /* tmp2 = */ ReadBFL()                    ; // ERROR: the "tmp2" is using a SHARED reference with "tmp1"
      }
      void BFL3() {
        auto tmp1 = ReadBFL()                     ; // "tmp1" is now a reference to the FLS storage of "ReadBFL"
        Send("C",MqTokenICB(&Server::pBFL3),"ECOL:[III]",4,5,6)      ; // ATTENTION: callback "pBFL3" using "ReadBFL"
        ProcessEvent(MQ_WAIT_OWN)               ; // ERROR: enter event-loop, callback "pBFL3" is called
        Send("R","L",tmp1)                      ; // ERROR: "tmp1" has now the value from "tmp2"
      }

Example from server.cc → convert "ReadBFL" result into global storage using "Dup" and free later

      void BFL4() {
        auto tmp1 = ReadBFL()                     ; // "tmp1" is now a reference to the FLS storage of "ReadBFL"
        auto glb1 = tmp1->Dup()                 ; // OK: "glb1" is now a UNSHARED reference to the global memory
        auto tmp2 = ReadBFL()                     ; // "tmp2" is now a reference to the FLS storage of "ReadBFL"
        Send("R","LL",glb1,tmp2)                ; // OK: "glb1" (alias tmp1) and "tmp2" are separate references
      }

TLS storage used by the *CreateTLS style of functions

In the C language the TLS (Thread-Local-Storage) is unique per definition and the name is used to distinguish the storage.

‍The Problem is to create a TLS interface useable in all Target-Programming-Language (TPL) supported by the Programming-Language-Micro-Kernel (PLMK).

The *CreateTLS style function return a TLS that is a global storage. global mean unique per runtime and not unique per definition. The tlsid (0,1,2,3...) is used to distinguish the storage on the global level.

‍Every *CreateTLS style function with the same tlsid return the same memory in the same thread.

There is always a risk that the memory used by the *CreateTLS style of functions will also be used by another component of the software in the same thread.

Attention

1. Use the *CreateTLS style function with caution in a local (controlled) context.
2. It is a good-practice to manage the tlsid on a global level like an enum
3. If the tlsid is not managed, it will be a problem when the event-loop is invoked or an asynchronous-service-call is received or made and other code uses the same tlsid

Example from perfserver.cc → performance test with TLS storage in a local (controlled) context

    void BUST () {
      auto bus = MkBufferStreamC::CreateTLS( "perfserver-BUST" );
      while (ReadItemExists()) {
        bus->WriteBUF(ReadBUF());
      }
      bus->PosToStart();
      SendSTART();
      while (bus->ReadItemExists()) {
        SendBUF(bus->ReadBUF());
      }
      SendRETURN();
    }

Example from LibSq3LiteRpcClient.tcl → callback dealing the temporary TLS data

# Intro     : Example from tcl-rpc-client of using a CreateTLS-like function (here for MkBufferListC) 
#             to improve code speed and readability.
#
# Problem   : This function is used to invoke a callback (myCb). The arguments come from the argument 
#             list args *and* from a service call (ReadBFL).
#             The problem is that ReadBFL is called *twice* and the *second* call overwrites the value 
#             of the *first* call because CreateTLS always returns *the same* MkBufferListC, just 
#             replaced with a new set of values.
#
# Solution  : The MkBufferListC instance returned by ReadBFL is copied into another MkBufferListC 
#             instance returned by CreateTLS.
#             The "CreateTLS" instance is only created *once* and reused, *but* now we can create as 
#             many MkBufferListC instances as we want, because "CreateTLS" distinguishes the returned 
#             instances by the string identifier. 
#             WITHOUT "CreateTLS" a copy would have to be created (Dup) which would then be destroyed 
#             *after* the callback is called (Delete)

proc Sq3LiteRpcClientExecV2CB {rpc myCb args} {
  set valL [MkBufferListC CreateTLS "Sq3LiteRpcClientExecV2CB→valL"]
  set colL [MkBufferListC CreateTLS "Sq3LiteRpcClientExecV2CB→colL"]

  $valL Copy [$rpc ReadBFL]
  $colL Copy [$rpc ReadBFL]

  $myCb {*}$args $valL $colL
}