theLink 10.0
Loading...
Searching...
No Matches
MqContextC_Route_ATL_API

MqContextC - setup and manage a routing-linkMore...

+ Collaboration diagram for MqContextC_Route_ATL_API:

Functions

static OT_ProcRet atlmqmsgque_MqContextC_RouteGetTree (MqContextC_ARGS)
  Atl: MkBufferListC [MqContextC::RouteGetTree $ctx] C-API
create an overview about all available routing-target and services …
 
static OT_ProcRet atlmqmsgque_MqContextC_RouteCreate (MqContextC_ARGS)
  Atl: MqContextC::RouteCreate $ctx route:string service:string ?overwrite:bool=false? C-API
create/delete a routing-link between context an a service using route
 
static OT_ProcRet atlmqmsgque_MqContextC_RouteDelete (MqContextC_ARGS)
  Atl: MqContextC::RouteDelete $ctx route:string service:string ?overwrite:bool=false? C-API
delete a routing-link created with MqRouteCreate
 
static OT_ProcRet atlmqmsgque_MqContextC_RouteResolve (MqContextC_ARGS)
  Atl: [MqContextC...] [MqContextC::RouteResolve $ctx ident:string ?retnum:int32=-1?] C-API
return a list of all context belonging to ident
 
static OT_ProcRet atlmqmsgque_MqContextC_RouteTraverse (MqContextC_ARGS)
  Atl: MqContextC::RouteTraverse $ctx service:string ?args:MkBufferListC...=NULL? C-API
traverse a tree down and call service if available.
 
static OT_ProcRet atlmqmsgque_MqContextC_RouteGetPath (MqContextC_ARGS)
  Atl: string [MqContextC::RouteGetPath $ctx] C-API
return the absolut route-connection-string up to the current ctx
 

Detailed Description

MqContextC - setup and manage a routing-link

A routing-link is the connection of two context, the route-source and the route-target, with a unspecifiend number of hub-context in between using a specific service-token.

A single context is identified by the context-identifier as returned by MqClassIdentGet.

A routing-link can be created using Service-Level-Routing or Package-Level-Routing.

The difference between Service-Level-Routing and Package-Level-Routing is the public versa private behaviour.

Summary:

TCP/IP layer model

The TCP/IP model define the following layers (from: https://www.geeksforgeeks.org/tcp-ip-model/) :

Network Access Layer
This layer corresponds to the combination of Data Link Layer and Physical Layer of the OSI model. It looks out for hardware addressing and the protocols present in this layer allows for the physical transmission of data.
Internet Layer
This layer parallels the functions of OSI’s Network layer. It defines the protocols which are responsible for logical transmission of data over the entire network
The main protocols residing at this layer are :
  1. IP – stands for Internet Protocol and it is responsible for delivering packages from the source host to the destination host by looking at the IP addresses in the package headers. IP has 2 versions: IPv4 and IPv6. IPv4 is the one that most of the websites are using currently. But IPv6 is growing as the number of IPv4 addresses are limited in number when compared to the number of users.
  2. ICMP – stands for Internet Control Message Protocol. It is encapsulated within IP datagrams and is responsible for providing hosts with information about network problems.
  3. ARP – stands for Address Resolution Protocol. Its job is to find the hardware address of a host from a known IP address. ARP has several types: Reverse ARP, Proxy ARP, Gratuitous ARP and Inverse ARP.
Host-to-Host Layer
This layer is analogous to the transport layer of the OSI model. It is responsible for end-to-end communication and error-free delivery of data. It shields the upper-layer applications from the complexities of data.
The two main protocols present in this layer are :
  1. Transmission Control Protocol (TCP) – It is known to provide reliable and error-free communication between end systems. It performs sequencing and segmentation of data. It also has acknowledgment feature and controls the flow of the data through flow control mechanism. It is a very effective protocol but has a lot of overhead due to such features. Increased overhead leads to increased cost.
  2. User Datagram Protocol (UDP) – On the other hand does not provide any such features. It is the go-to protocol if your application does not require reliable transport as it is very cost-effective. Unlike TCP, which is connection-oriented protocol, UDP is connectionless.
Application Layer
This layer performs the functions of top three layers of the OSI model: Application, Presentation and Session Layer. It is responsible for node-to-node communication and controls user-interface specifications. Some of the protocols present in this layer are: HTTP, HTTPS, FTP, TFTP, Telnet, SSH, SMTP, SNMP, NTP, DNS, DHCP, NFS, X Window, LPD. Have a look at Protocols in Application Layer for some information about these protocols.
Protocols other than those present in the linked article are :
  1. HTTP and HTTPS – HTTP stands for Hypertext transfer protocol. It is used by the World Wide Web to manage communications between web browsers and servers. HTTPS stands for HTTP-Secure. It is a combination of HTTP with SSL(Secure Socket Layer). It is efficient in cases where the browser need to fill out forms, sign in, authenticate and carry out bank transactions.
  2. SSH – SSH stands for Secure Shell. It is a terminal emulations software similar to Telnet. The reason SSH is more preferred is because of its ability to maintain the encrypted connection. It sets up a secure session over a TCP/IP connection.
  3. NTP – NTP stands for Network Time Protocol. It is used to synchronize the clocks on our computer to one standard time source. It is very useful in situations like bank transactions. Assume the following situation without the presence of NTP. Suppose you carry out a transaction, where your computer reads the time at 2:30 PM while the server records it at 2:28 PM. The server can crash very badly if it’s out of sync.

LibMqMsgque layer model

The libmqmsgque layer model is an extension to the TCP/IP layer model.

TCP/IP Application Layer
This layer is used for the libmqmsgque protocol - The protocol is the "language" of the libmqmsgque
library. The layer is defined as protocol-message-format and as protocol-flow-format.
  1. protocol-message-format - Example from protocoll_mq.h this is the "syntax" of the protocol.
    struct HdrS {
    MK_STRB ID[4] ;
    MK_STRB native ;
    MK_STRB charA ;
    union HdrIU ctxId ;
    MK_STRB charS ;
    union HdrIU bdySize ;
    MK_STRB charO ;
    MK_STRB tok[HDR_TOK_LEN] ;
    MK_STRB charT ;
    union HdrIU transSId ;
    MK_STRB charR ;
    union HdrIU routeId ;
    MK_STRB charC ;
    MK_STRB hs ;
    MK_STRB charE ;
    MK_STRB tt ;
    MK_STRB charF ;
    };
    struct BdyS {
    MK_STRB ID[4] ;
    //MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>
    union HdrIU numItems;
    MK_STRB charE ;
    };
    struct LtrS {
    MK_STRB ID[4] ;
    //MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>
    MK_BINB ltrP[HDR_ID_SIZE] ;
    MK_STRB charE ;
    };
    #define LTR_SIZE (3 + 1 + HDR_ID_SIZE + 1)
    struct RouS {
    MK_STRB ID[4] ;
    //MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>
    MK_BINB rouP[HDR_ID_SIZE] ;
    MK_STRB charE ;
    };
    #define ROU_SIZE (3 + 1 + HDR_ID_SIZE + 1)
    struct PackageS {
    struct HdrS hdr ;
    struct BdyS bdy ;
    };
  2. protocol-flow-format - this is the "grammatic" of the protocol.
LibMqMsgque Service Layer

This layer provide the MqContextC-ServiceApi ontop of the Application Layer.

The master-slave-link is used to create a mesh of nodes defined by different parent-context. The master control the slave.

The master-slave-link is used to perform the following tasks:

  • report error messages from the slave-context to the master-context
  • to create a slave-child-context if a master-child-context is created
  • to delete a slave-context if a master-context is deleted

In difference to the client-server-link the master-slave-link connect two independent parent-context in the same process or thread (e.g. node). This leads to the restriction that only the master-context can be a server-context because only one server-context per node is possible.

   node-0   |           node-1/2        |   node-3/4/5
===================================================================

| <- client/server link -> | <- client/server link -> |

             | <-- master/slave link --> |

                           |- client1-0 -|- server3 ...
             |-  server1  -|
             |             |- client1-1 -|- server4 ...
  client0-0 -|
             |-  server2  -|- client1-2 -|- server5 ...

Definition of the "master-context"

  • the master-context is a parent-context without a child-context available.
  • the master-context is a client-context or a server-context.
  • the link between the master-context and the slave-context is done using MqSlaveWorker or MqSlaveCreate

Definition of the "slave-context"

  • the slave-context is a parent-context without a child-context available.
  • the slave-context is a client-context.
  • the slave-context lifetime is controlled by the master-context.
  • the slave-context report all error-messages to the master-context.
  • the slave-context is identified by a unique-slave-id starting with 0.
  • a special form of a slave-context is a worker-context

Definition of the "worker-context"

  • the worker-context is a slave-context using the image of the master-context self.
  • the master-context can be a server-context or a client-context.
  • the worker-context is created using MqSlaveWorker
  • the worker-context is identified by a unique-slave-id starting with 0.

Definition of the "slave-id"

  • the slave-id is defined at enum MqSlaveE
  • a slave is identified in his master-context by a slave-id
  • the slave-id work like a defined well-known-port-number in /etc/services
  • the slave-id is an integer value with valid values > 0
  • it is a good practice to plan the usage of your slave-id(s)
  • slave
    slave-id value definition
    MQ_SLAVE_MAX 1024 internal: the maximum slave-id … .
    MQ_SLAVE_USER 10 internal: start of user-defined-slave-id .
    MQ_SLAVE_LOOPBACK 0 internal: the loopback-slave-id, (call my own services) .
    MQ_SLAVE_FILTER 1 internal: the filter-slave-id, (on a master get the filter-slave) .
    MQ_SLAVE_MASTER 1 internal: the master-slave-id, (on a slave get the master) .
    MQ_SLAVE_OTHER 1 internal: on the master-ctx get the slave-ctx and on the slave-ctx get the master-ctx .
  • range
    range definition
    0 <= slave-id < MQ_SLAVE_MAX range of valid slave-id's
    0 <= slave-id < MQ_SLAVE_USER internale usage
    MQ_SLAVE_USER <= slave-id < MQ_SLAVE_MAX external usage

Definition of the "LOOPBACK" (0) slave

  •    client   |           server            |
    ===========================================
    
    | <--- client/server --->  | <-- loop --> |
    
                | <------ master/slave -----> |
    
      client -- | -- server -- | -- client -- #
                       ==             ==      #
                     server -- | -- client -- #
  • the loopback has always the slave-id = 0 .
  • the loopback has the same class as the parent, if reflection is not available the MqFactoryInitial is used.
  • the loopback is used to call a service on the same process or thread.
  • the loopback is a special filter without an additional process or thread to be started.
  • the loopback is only internal accessible
  • the service called by the loopback need the same attention as the service called by the filter, the context of the service is the loopback-context.
  • the loopback can call MqServiceCreate to create a new link between a service-token and a service-method, by default all services from the master-context (the owner of the loopback) are also accessible by the loopback.
  • in the service use MqSlaveGetMaster to get the master-context from the loopback-context.
  • Example from MyLoopServer.atl create a new loop-server
    package require lib_85
    package require atlmqmsgque
    
    ::myooX::ClassN ::MyLoopServer {
      SuperI ::MqContextC
    
      # factory startup
      proc MyLoopServer {myNs {tmpl ""}} {
        namespace upvar $myNs my(mydata) mydata;
        MqContextC $myNs $tmpl
        ConfigSetServerSetup $myNs ServerSetup
        # set the "mydata" attribute to the master-context
        set mydata  "Hello World";
      }
    
      # service to serve all EXTERNAL requests for token "HLWO"
      proc HLWO_srv { myNs } {
        # get the "loopback" context
        set loop  [SlaveGet $myNs LOOPBACK]
        # call the LOOP service on the SAME server
        MqContextC::Send $loop "W" "LOOP"
        # answer HLWO with string-return from LOOP
        Send $myNs "R" "C" [MqContextC::ReadSTR $loop]
      }
    
      # service to serve all INTERNAL requests for token "LOOP"
      proc LOOP_srv { myNs } {
        # get the "master" context
        set master  [SlaveGetMaster $myNs]
        # answer LOOP with data from MASTER->mydata attribute
        namespace upvar $master my(mydata) mydata;
        Send $myNs "R" "C" $mydata
      }
    
      # setup a server-context - called for every new server-context
      proc ServerSetup { myNs } {
        # EXTERNAL: link the "HLWO" service with "HLWO_srv"
        ServiceCreate $myNs "HLWO" HLWO_srv
        # INTERNAL: link the "LOOP" service with "LOOP_srv"
        ServiceCreate [SlaveGet $myNs MQ_SLAVE_LOOPBACK] "LOOP" LOOP_srv
      }
    }
    
    MqMsgque::Main {
    
      # create "MyLoopServer" factory... and the instance
      set srv   [MqFactoryC::New [MqFactoryC::Add ::MyLoopServer]]
    
      try {
        MqContextC::LinkCreate $srv $argv
        MqContextC::ProcessEvent $srv MQ_WAIT_FOREVER
      } on error {} {
        MqContextC::ErrorCatch $srv
      } finally {
        MqContextC::Exit $srv
      }
    }
    

Performance analyse

The performance-test is created with:
  • pipe
    • Nhi1Exec perfclient.c --parent --wrk ? @ perfserver.c
  • spawn, fork, thread
    • Nhi1Exec -r=uds perfserver.c --spawn|fork|thread
    • Nhi1Exec -r=uds perfclient.c --parent --wrk ?
  • the number of workers are set with the –wrk option
  • the cpu is a xeon with 4/8
  • the performance is calculated as worker-context-created / time-in-sec with a ~2sec (default) measurement period.
performance-test code:
  • The test-setup is done as:
      perfclient                                worker                            perfserver
      ==========                                ======                            ==========
      |
      |- loop --wrk x
        |- MqSlaveWorker(...)               ->  worker[1] 
        |- MqSend(worker[1],"E","STR0..")   ->  PerfWorker_I160(...)
                                                |- loop endless
                                                  |- MqContextCreate(...)
                                                  |- MqLinkCreate(...)      <->   MqContextCreate(...)
                                                  |- MqContextDelete(...)   <->   MqContextDelete(...)
      |- sleep x sec
      |- loop --wrk x
        |- MqSend(worker[1],"C"..,"END0")   ->  PerfWorker_END0(...)
        |                                       |- stop loop
        |- "callback" - add number to all   <-  |- return #context 
performance-test results:
  • results generated using the debug environment
    setup –wrk # worker-context performance info
    pipe 1 2500 1000 the pipe start a new worker-context with spawn
    spawn 1 2500 <1000 same as pipe but use network-protocoll
    fork 1 3800 4000 the fork is faster than spawn
    thread 1 16500 9000 the thread is faster than fork
    pipe 4 8000 4500 the worker scale linear up to number of processors
    spawn 4 7600 <4500 -
    fork 4 23200 11500 -
    thread 4 55500 27500 -
    pipe 8 10000 5500 the additional scaling up to the max hyper-threading does not really help
    spawn 8 9100 <5500 -
    fork 8 23200 11500 -
    thread 8 55500 27500 -

example (C++ syntax)

Route-Connection-String

A client-server connection is defined with the route-connection-string as a composition of "identifier" and is build like a UNIX directory tree with the initial client as root.

Example: GUI/Data Frontend

*
|-header
|-GUI-|
frontend-| |-footer
|
|-DB
*

With --ident-from prefix|factory the identifier is defined as prefix-identifier or factory-identifer.

A route-connection-string is the path between TWO locations in the tree using the UNIX syntax:

short syntax direction
dot . current-context
double-dot .. previous-context
slash / root-context (the client)
"string" xxx next-context with name xxx

path between "DB" and "header"

  • relative: ctx.RouteCreate("../GUI/header",service)
  • absolute: ctx.RouteCreate("/frontend/GUI/header",service)

path between "footer" and "DB"

  • relative: ctx.RouteCreate("../../DB",service)
  • absolute: ctx.RouteCreate("/frontend/DB",service)

path between "frontend" and "footer"

  • relative: ctx.RouteCreate("GUI/footer",service)
  • absolute: ctx.RouteCreate("/frontend/GUI/footer",service)

Service-Layer-Routing

The LibMqMsgque-Service-Layer-Routing create proxy-services (MqServiceProxyRoundRobin ...) between frontend and footer. (example c++)

// 1. on "frontend" create a route to the service "MYSV" on the "footer" passing "GUI"
DOING: frontendCtx.RouteCreate("GUI/footer", "MYSV");
// 2. on "frontend" call the route using the "loopback" slave
DOING: frontendCtx.SlaveGet(MQ_SLAVE_LOOPBACK).Send("W", "MYSV:..@..", ..);
// 3. on "frontend/GUI" proxy "footer"
INTERNAL: frontendCtx.ServiceProxyRoundRobin("MYSV", GUICtx);
INTERNAL: GUICtx.ServiceProxyRoundRobin("MYSV", footerCtx);
// 4. on "footer" call the service "MYSV"
SETUP: footerCtx.ServiceCreate("MYSV", ...);
@ MQ_SLAVE_LOOPBACK
internal: the loopback-slave-id, (call my own services)

The MqRouteCreate will take the following action:

  1. create a round-robin-proxy on frontend and on GUI using service MYSV
  2. check if service MYSV is available on footer

Attention:

  1. With the proxy on frontend/GUI no other service with the name MYSV on frontend/GUI is possible and every service-call to frontend or GUI using the service MYSV will be redirected to the service MYSV on footer.
  2. The live-time of the Service-Layer-Routing is up to the next MqRouteDelete call using the same arguments. If the proxy-to-footer becomes invalid on a the GUI-context the routing-link will be recreated.
  3. The MqRouteCreate can be used multiple times with the same Route-Connection-String and service-token and overwrite=true to guarantee that a route is available.

Package-Layer-Routing

...

Function Documentation

◆ atlmqmsgque_MqContextC_RouteCreate()

static OT_ProcRet atlmqmsgque_MqContextC_RouteCreate ( MqContextC_ARGS )
static

Atl: MqContextC::RouteCreate $ctx route:string service:string ?overwrite:bool=false? C-API
create/delete a routing-link between context an a service using route

Definition at line 2390 of file MqContextC_atl.c.

2390 {
2393 MK_STRN route = 0;
2395 MK_STRN service = 0;
2397 MK_BOOL overwrite = false;
2398 OT_CHECK_OPTIONAL(OT_CHECK_bool (overwrite))
2400 MkErrorC_Check(hdl,MqRouteCreate (hdl, route, service, overwrite));
2402 goto end;
2403 error:
2405 end:
2407}
#define OT_CHECK_bool(val)
#define OT_SETUP_VARARGS(min, max, d)
#define OT_retObj_SET_Error
#define OT_CHECK_REQUIRED(val)
#define OT_retObj_SET_None
#define OT_retObj_RETURN
#define OT_CHECK_OPTIONAL(val)
#define OT_CHECK_NOARGS
#define MkErrorC_Check(mng, PROC)
#define OT_CHECK_STRN(val)
#define RouteCreate_doc
#define OT_SETUP_hdl
#define error
Definition high_lng.h:339
const MK_STRB * MK_STRN
bool MK_BOOL
#define MqRouteCreate(...)

◆ atlmqmsgque_MqContextC_RouteDelete()

static OT_ProcRet atlmqmsgque_MqContextC_RouteDelete ( MqContextC_ARGS )
static

Atl: MqContextC::RouteDelete $ctx route:string service:string ?overwrite:bool=false? C-API
delete a routing-link created with MqRouteCreate

Definition at line 2410 of file MqContextC_atl.c.

2410 {
2413 MK_STRN route = 0;
2415 MK_STRN service = 0;
2417 MK_BOOL overwrite = false;
2418 OT_CHECK_OPTIONAL(OT_CHECK_bool (overwrite))
2420 MkErrorC_Check(hdl,MqRouteDelete (hdl, route, service, overwrite));
2422 goto end;
2423 error:
2425 end:
2427}
#define RouteDelete_doc
#define MqRouteDelete(...)

◆ atlmqmsgque_MqContextC_RouteGetPath()

static OT_ProcRet atlmqmsgque_MqContextC_RouteGetPath ( MqContextC_ARGS )
static

Atl: string [MqContextC::RouteGetPath $ctx] C-API
return the absolut route-connection-string up to the current ctx

Definition at line 2471 of file MqContextC_atl.c.

2471 {
2475 MK_STRN path_out;
2476 MkErrorC_Check(hdl,MqRouteGetPath (hdl, &path_out));
2477 OT_retObj_SET_STR(path_out);
2478 goto end;
2479 error:
2481 end:
2483}
#define OT_SETUP_NOARG(d)
#define OT_retObj_SET_STR(nat)
#define RouteGetPath_doc
#define MqRouteGetPath(...)

◆ atlmqmsgque_MqContextC_RouteGetTree()

static OT_ProcRet atlmqmsgque_MqContextC_RouteGetTree ( MqContextC_ARGS )
static

Atl: MkBufferListC [MqContextC::RouteGetTree $ctx] C-API
create an overview about all available routing-target and services …

Definition at line 2373 of file MqContextC_atl.c.

2373 {
2377 MK_BFL treeP_out;
2378 MkErrorC_Check(hdl,MqRouteGetTree (hdl, &treeP_out));
2379 OT_retObj_SET_BFL(treeP_out);
2380 goto end;
2381 error:
2383 end:
2385}
#define OT_retObj_SET_BFL(nat)
#define RouteGetTree_doc
#define MqRouteGetTree(...)

◆ atlmqmsgque_MqContextC_RouteResolve()

static OT_ProcRet atlmqmsgque_MqContextC_RouteResolve ( MqContextC_ARGS )
static

Atl: [MqContextC...] [MqContextC::RouteResolve $ctx ident:string ?retnum:int32=-1?] C-API
return a list of all context belonging to ident

Definition at line 2430 of file MqContextC_atl.c.

2430 {
2433 MK_STRN ident = 0;
2435 MK_NUM retnum = -1;
2438 MQ_CTX_A retVal = MqRouteResolve (hdl, ident, retnum);
2440 for (int i=0; i<retVal.size; i++) {
2442 }
2443 goto end;
2444 error:
2446 end:
2448}
#define OT_CHECK_NI4(val)
#define OT_retObj_SET_List
#define OT_retObj_APPEND(var)
#define OT_TMP_CTX_OBJ(nat)
#define RouteResolve_doc
int32_t MK_NUM
#define MqRouteResolve(...)
array of MqContextC instances
MQ_CTX * data

◆ atlmqmsgque_MqContextC_RouteTraverse()

static OT_ProcRet atlmqmsgque_MqContextC_RouteTraverse ( MqContextC_ARGS )
static

Atl: MqContextC::RouteTraverse $ctx service:string ?args:MkBufferListC...=NULL? C-API
traverse a tree down and call service if available.

Definition at line 2451 of file MqContextC_atl.c.

2451 {
2454 MK_STRN service = 0;
2456 MkBufferListCreateTLS_T(args,10);
2459 MkErrorC_Check(hdl,MqRouteTraverse (hdl, service, args));
2461 goto end;
2462 error:
2464 end:
2466}
#define RouteTraverse_doc
#define MkBufferListCreateTLS_T(name, num)
#define MqRouteTraverse(...)
#define OT_CHECK_BAC(val)