Loading...
Searching...
No Matches
MqContextC_Route_CC_API

MqContextC - setup and manage a routing-linkMore...

+ Collaboration diagram for MqContextC_Route_CC_API:

Functions

MkBufferListCccmqmsgque::MqContextC::RouteGetTree ()
  C++: MkBufferListC* ctx.RouteGetTree() C-API
create an overview about all available routing-target and services …
 
void ccmqmsgque::MqContextC::RouteCreate (MK_STRN route, MK_STRN service, MK_BOOL overwrite=false)
  C++: ctx.RouteCreate(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) C-API
create/delete a routing-link between context an a service using route
 
void ccmqmsgque::MqContextC::RouteCreate (const std::string &route, const std::string &service, MK_BOOL overwrite=false)
  C++: ctx.RouteCreate(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) C-API
create/delete a routing-link between context an a service using route
 
void ccmqmsgque::MqContextC::RouteDelete (MK_STRN route, MK_STRN service, MK_BOOL overwrite=false)
  C++: ctx.RouteDelete(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) C-API
delete a routing-link created with MqRouteCreate
 
void ccmqmsgque::MqContextC::RouteDelete (const std::string &route, const std::string &service, MK_BOOL overwrite=false)
  C++: ctx.RouteDelete(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) C-API
delete a routing-link created with MqRouteCreate
 
MqContextC_A ccmqmsgque::MqContextC::RouteResolve (MK_STRN ident, MK_NUM retnum=-1)
  C++: MqContextC_A ctx.RouteResolve(MK_STRN ident, MK_NUM retnum = -1) C-API
return a list of all context belonging to ident
 
MqContextC_A ccmqmsgque::MqContextC::RouteResolve (const std::string &ident, MK_NUM retnum=-1)
  C++: MqContextC_A ctx.RouteResolve(MK_STRN ident, MK_NUM retnum = -1) C-API
return a list of all context belonging to ident
 
void ccmqmsgque::MqContextC::RouteTraverse (MK_STRN service, MkBufferListC *args=NULL)
  C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL) C-API
traverse a tree down and call service if available.
 
void ccmqmsgque::MqContextC::RouteTraverse (MK_STRN service, const MkBufferListC &args)
  C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL) C-API
traverse a tree down and call service if available.
 
void ccmqmsgque::MqContextC::RouteTraverse (const std::string &service, MkBufferListC *args=NULL)
  C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL) C-API
traverse a tree down and call service if available.
 
void ccmqmsgque::MqContextC::RouteTraverse (const std::string &service, const MkBufferListC &args)
  C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL) C-API
traverse a tree down and call service if available.
 
MK_STRN ccmqmsgque::MqContextC::RouteGetPath ()
  C++: MK_STRN ctx.RouteGetPath() C-API
return the absolut route-connection-string up to the current ctx
 

Detailed Description

MqContextC - setup and manage a routing-link

A routing-link is the connection of two context, the route-source and the route-target, with a unspecifiend number of hub-context in between using a specific service-token.

A single context is identified by the context-identifier as returned by MqClassIdentGet.

A routing-link can be created using Service-Level-Routing or Package-Level-Routing.

The difference between Service-Level-Routing and Package-Level-Routing is the public versa private behaviour.

Summary:

TCP/IP layer model

The TCP/IP model define the following layers (from: https://www.geeksforgeeks.org/tcp-ip-model/) :

Network Access Layer
This layer corresponds to the combination of Data Link Layer and Physical Layer of the OSI model. It looks out for hardware addressing and the protocols present in this layer allows for the physical transmission of data.
Internet Layer
This layer parallels the functions of OSI’s Network layer. It defines the protocols which are responsible for logical transmission of data over the entire network
The main protocols residing at this layer are :
  1. IP – stands for Internet Protocol and it is responsible for delivering packages from the source host to the destination host by looking at the IP addresses in the package headers. IP has 2 versions: IPv4 and IPv6. IPv4 is the one that most of the websites are using currently. But IPv6 is growing as the number of IPv4 addresses are limited in number when compared to the number of users.
  2. ICMP – stands for Internet Control Message Protocol. It is encapsulated within IP datagrams and is responsible for providing hosts with information about network problems.
  3. ARP – stands for Address Resolution Protocol. Its job is to find the hardware address of a host from a known IP address. ARP has several types: Reverse ARP, Proxy ARP, Gratuitous ARP and Inverse ARP.
Host-to-Host Layer
This layer is analogous to the transport layer of the OSI model. It is responsible for end-to-end communication and error-free delivery of data. It shields the upper-layer applications from the complexities of data.
The two main protocols present in this layer are :
  1. Transmission Control Protocol (TCP) – It is known to provide reliable and error-free communication between end systems. It performs sequencing and segmentation of data. It also has acknowledgment feature and controls the flow of the data through flow control mechanism. It is a very effective protocol but has a lot of overhead due to such features. Increased overhead leads to increased cost.
  2. User Datagram Protocol (UDP) – On the other hand does not provide any such features. It is the go-to protocol if your application does not require reliable transport as it is very cost-effective. Unlike TCP, which is connection-oriented protocol, UDP is connectionless.
Application Layer
This layer performs the functions of top three layers of the OSI model: Application, Presentation and Session Layer. It is responsible for node-to-node communication and controls user-interface specifications. Some of the protocols present in this layer are: HTTP, HTTPS, FTP, TFTP, Telnet, SSH, SMTP, SNMP, NTP, DNS, DHCP, NFS, X Window, LPD. Have a look at Protocols in Application Layer for some information about these protocols.
Protocols other than those present in the linked article are :
  1. HTTP and HTTPS – HTTP stands for Hypertext transfer protocol. It is used by the World Wide Web to manage communications between web browsers and servers. HTTPS stands for HTTP-Secure. It is a combination of HTTP with SSL(Secure Socket Layer). It is efficient in cases where the browser need to fill out forms, sign in, authenticate and carry out bank transactions.
  2. SSH – SSH stands for Secure Shell. It is a terminal emulations software similar to Telnet. The reason SSH is more preferred is because of its ability to maintain the encrypted connection. It sets up a secure session over a TCP/IP connection.
  3. NTP – NTP stands for Network Time Protocol. It is used to synchronize the clocks on our computer to one standard time source. It is very useful in situations like bank transactions. Assume the following situation without the presence of NTP. Suppose you carry out a transaction, where your computer reads the time at 2:30 PM while the server records it at 2:28 PM. The server can crash very badly if it’s out of sync.

LibMqMsgque layer model

The libmqmsgque layer model is an extension to the TCP/IP layer model.

TCP/IP Application Layer
This layer is used for the libmqmsgque protocol - The protocol is the "language" of the libmqmsgque
library. The layer is defined as protocol-message-format and as protocol-flow-format.
  1. protocol-message-format - Example from protocoll_mq.h this is the "syntax" of the protocol.
    struct HdrS {
    MK_STRB ID[4] ;
    MK_STRB native ;
    MK_STRB charA ;
    union HdrIU ctxId ;
    MK_STRB charS ;
    union HdrIU bdySize ;
    MK_STRB charO ;
    MK_STRB tok[HDR_TOK_LEN] ;
    MK_STRB charT ;
    union HdrIU transSId ;
    MK_STRB charR ;
    union HdrIU routeId ;
    MK_STRB charC ;
    MK_STRB hs ;
    MK_STRB charE ;
    MK_STRB tt ;
    MK_STRB charF ;
    };
    struct BdyS {
    MK_STRB ID[4] ;
    //MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>
    union HdrIU numItems;
    MK_STRB charE ;
    };
    struct LtrS {
    MK_STRB ID[4] ;
    //MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>
    MK_BINB ltrP[HDR_ID_SIZE] ;
    MK_STRB charE ;
    };
    #define LTR_SIZE (3 + 1 + HDR_ID_SIZE + 1)
    struct RouS {
    MK_STRB ID[4] ;
    //MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>
    MK_BINB rouP[HDR_ID_SIZE] ;
    MK_STRB charE ;
    };
    #define ROU_SIZE (3 + 1 + HDR_ID_SIZE + 1)
    struct PackageS {
    struct HdrS hdr ;
    struct BdyS bdy ;
    };
  2. protocol-flow-format - this is the "grammatic" of the protocol.
LibMqMsgque Service Layer

This layer provide the MqContextC-ServiceApi ontop of the Application Layer.

The master-slave-link is used to create a mesh of nodes defined by different parent-context. The master control the slave.

The master-slave-link is used to perform the following tasks:

  • report error messages from the slave-context to the master-context
  • to create a slave-child-context if a master-child-context is created
  • to delete a slave-context if a master-context is deleted

In difference to the client-server-link the master-slave-link connect two independent parent-context in the same process or thread (e.g. node). This leads to the restriction that only the master-context can be a server-context because only one server-context per node is possible.

   node-0   |           node-1/2        |   node-3/4/5
===================================================================

| <- client/server link -> | <- client/server link -> |

             | <-- master/slave link --> |

                           |- client1-0 -|- server3 ...
             |-  server1  -|
             |             |- client1-1 -|- server4 ...
  client0-0 -|
             |-  server2  -|- client1-2 -|- server5 ...

Definition of the "master-context"

  • the master-context is a parent-context without a child-context available.
  • the master-context is a client-context or a server-context.
  • the link between the master-context and the slave-context is done using MqSlaveWorker or MqSlaveCreate

Definition of the "slave-context"

  • the slave-context is a parent-context without a child-context available.
  • the slave-context is a client-context.
  • the slave-context lifetime is controlled by the master-context.
  • the slave-context report all error-messages to the master-context.
  • the slave-context is identified by a unique-slave-id starting with 0.
  • a special form of a slave-context is a worker-context

Definition of the "worker-context"

  • the worker-context is a slave-context using the image of the master-context self.
  • the master-context can be a server-context or a client-context.
  • the worker-context is created using MqSlaveWorker
  • the worker-context is identified by a unique-slave-id starting with 0.

Definition of the "slave-id"

Definition of the "LOOPBACK" (0) slave

  •    client   |           server            |
    ===========================================
    
    | <--- client/server --->  | <-- loop --> |
    
                | <------ master/slave -----> |
    
      client -- | -- server -- | -- client -- #
                       ==             ==      #
                     server -- | -- client -- #
  • the loopback has always the slave-id = 0 .
  • the loopback has the same class as the parent, if reflection is not available the MqFactoryInitial is used.
  • the loopback is used to call a service on the same process or thread.
  • the loopback is a special filter without an additional process or thread to be started.
  • the loopback is only internal accessible
  • the service called by the loopback need the same attention as the service called by the filter, the context of the service is the loopback-context.
  • the loopback can call MqServiceCreate to create a new link between a service-token and a service-method, by default all services from the master-context (the owner of the loopback) are also accessible by the loopback.
  • in the service use MqSlaveGetMaster to get the master-context from the loopback-context.
  • Example from MyLoopServer.cc create a new loop-server
    #include  "LibMqMsgque_cc.hh"
    
    using namespace ccmqmsgque;
    
    // package-item
    class MyLoopServer : public MqContextC, public IServerSetup {
      friend class MqFactoryCT<MyLoopServer>;
    
      // set the "mydata" attribute to the master-context
      MK_STRN mydata = "Hello World";
    
      // define the factory constructor
      MyLoopServer(MK_TYP const typ, MqContextC* tmpl=NULL) : MqContextC(typ, tmpl) {};
    
      private:
        // service to serve all EXTERNAL requests for token "HLWO"
        void HLWO_srv () {
          // get the "loopback" context
          auto loop = SlaveGet(MQ_SLAVE_LOOPBACK);
          // call the LOOP service on the SAME server
          loop->Send("W", "LOOP");
          // answer HLWO with string-return from LOOP
          Send("R", "C", loop->ReadSTR());
        }
    
        // service to serve all INTERNAL requests for token "LOOP"
        void LOOP_srv () {
          // get the "master" context 
          auto master = static_cast<MyLoopServer*>(SlaveGetMaster());
          // answer LOOP with data from MASTER->mydata attribute
          Send("R", "C", master->mydata);
        }
    
        // define a service as link between the token "HLWO" and the callback "MyFirstService"
        void ServerSetup() {
          // EXTERNAL: link the "HLWO" service with "HLWO_srv"
          ServiceCreate("HLWO", MqServiceICB(&MyLoopServer::HLWO_srv));
          // INTERNAL: link the "LOOP" service with "LOOP_srv"
          SlaveGet(MQ_SLAVE_LOOPBACK)->ServiceCreate("LOOP", MqServiceICB(&MyLoopServer::LOOP_srv));
        }
    };
    
    // package-main
    int MK_CDECL main(int argc, MK_STRN argv[]) {
      MqMsgque::Setup();
    
      // create "MyLoopServer" factory… and the instance
      auto srv = MqFactoryCT<MyLoopServer>::Add("MyLoopServer")->New();
    
      try {
        srv->LinkCreate( MkBufferListC {argc, argv} );
        srv->ProcessEvent (MQ_WAIT_FOREVER);
      } catch (const std::exception& e) {
        srv->ErrorCatch(e);
      }
      srv->Exit();
    }
    

Performance analyse

The performance-test is created with:
  • pipe
    • Nhi1Exec perfclient.c --parent --wrk ? @ perfserver.c
  • spawn, fork, thread
    • Nhi1Exec -r=uds perfserver.c --spawn|fork|thread
    • Nhi1Exec -r=uds perfclient.c --parent --wrk ?
  • the number of workers are set with the –wrk option
  • the cpu is a xeon with 4/8
  • the performance is calculated as worker-context-created / time-in-sec with a ~2sec (default) measurement period.
performance-test code:
  • The test-setup is done as:
      perfclient                                worker                            perfserver
      ==========                                ======                            ==========
      |
      |- loop --wrk x
        |- MqSlaveWorker(...)               ->  worker[1] 
        |- MqSend(worker[1],"E","STR0..")   ->  PerfWorker_I160(...)
                                                |- loop endless
                                                  |- MqContextCreate(...)
                                                  |- MqLinkCreate(...)      <->   MqContextCreate(...)
                                                  |- MqContextDelete(...)   <->   MqContextDelete(...)
      |- sleep x sec
      |- loop --wrk x
        |- MqSend(worker[1],"C"..,"END0")   ->  PerfWorker_END0(...)
        |                                       |- stop loop
        |- "callback" - add number to all   <-  |- return #context 
performance-test results:
  • results generated using the debug environment
    setup –wrk # worker-context performance info
    pipe 1 2500 1000 the pipe start a new worker-context with spawn
    spawn 1 2500 <1000 same as pipe but use network-protocoll
    fork 1 3800 4000 the fork is faster than spawn
    thread 1 16500 9000 the thread is faster than fork
    pipe 4 8000 4500 the worker scale linear up to number of processors
    spawn 4 7600 <4500 -
    fork 4 23200 11500 -
    thread 4 55500 27500 -
    pipe 8 10000 5500 the additional scaling up to the max hyper-threading does not really help
    spawn 8 9100 <5500 -
    fork 8 23200 11500 -
    thread 8 55500 27500 -

example (C++ syntax)

Route-Connection-String

A client-server connection is defined with the route-connection-string as a composition of "identifier" and is build like a UNIX directory tree with the initial client as root.

Example: GUI/Data Frontend

*
|-header
|-GUI-|
frontend-| |-footer
|
|-DB
*

With --ident-from prefix|factory the identifier is defined as prefix-identifier or factory-identifer.

A route-connection-string is the path between TWO locations in the tree using the UNIX syntax:

short syntax direction
dot . current-context
double-dot .. previous-context
slash / root-context (the client)
"string" xxx next-context with name xxx

path between "DB" and "header"

  • relative: ctx.RouteCreate("../GUI/header",service)
  • absolute: ctx.RouteCreate("/frontend/GUI/header",service)

path between "footer" and "DB"

  • relative: ctx.RouteCreate("../../DB",service)
  • absolute: ctx.RouteCreate("/frontend/DB",service)

path between "frontend" and "footer"

  • relative: ctx.RouteCreate("GUI/footer",service)
  • absolute: ctx.RouteCreate("/frontend/GUI/footer",service)

Service-Layer-Routing

The LibMqMsgque-Service-Layer-Routing create proxy-services (MqServiceProxyRoundRobin ...) between frontend and footer. (example c++)

// 1. on "frontend" create a route to the service "MYSV" on the "footer" passing "GUI"
DOING: frontendCtx.RouteCreate("GUI/footer", "MYSV");
// 2. on "frontend" call the route using the "loopback" slave
DOING: frontendCtx.SlaveGet(MQ_SLAVE_LOOPBACK).Send("W", "MYSV:..@..", ..);
// 3. on "frontend/GUI" proxy "footer"
INTERNAL: frontendCtx.ServiceProxyRoundRobin("MYSV", GUICtx);
INTERNAL: GUICtx.ServiceProxyRoundRobin("MYSV", footerCtx);
// 4. on "footer" call the service "MYSV"
SETUP: footerCtx.ServiceCreate("MYSV", ...);
@ MQ_SLAVE_LOOPBACK
internal: the loopback-slave-id, (call my own services)

The MqRouteCreate will take the following action:

  1. create a round-robin-proxy on frontend and on GUI using service MYSV
  2. check if service MYSV is available on footer

Attention:

  1. With the proxy on frontend/GUI no other service with the name MYSV on frontend/GUI is possible and every service-call to frontend or GUI using the service MYSV will be redirected to the service MYSV on footer.
  2. The live-time of the Service-Layer-Routing is up to the next MqRouteDelete call using the same arguments. If the proxy-to-footer becomes invalid on a the GUI-context the routing-link will be recreated.
  3. The MqRouteCreate can be used multiple times with the same Route-Connection-String and service-token and overwrite=true to guarantee that a route is available.

Package-Layer-Routing

...

A routing-link is the connection of two context, the route-source and the route-target, with a unspecifiend number of hub-context in between using a specific service-token.

A single context is identified by the context-identifier as returned by MqClassIdentGet.

A routing-link can be created using Service-Level-Routing or Package-Level-Routing.

The difference between Service-Level-Routing and Package-Level-Routing is the public versa private behaviour.

Summary:

TCP/IP layer model

The TCP/IP model define the following layers (from: https://www.geeksforgeeks.org/tcp-ip-model/) :

Network Access Layer
This layer corresponds to the combination of Data Link Layer and Physical Layer of the OSI model. It looks out for hardware addressing and the protocols present in this layer allows for the physical transmission of data.
Internet Layer
This layer parallels the functions of OSI’s Network layer. It defines the protocols which are responsible for logical transmission of data over the entire network
The main protocols residing at this layer are :
  1. IP – stands for Internet Protocol and it is responsible for delivering packages from the source host to the destination host by looking at the IP addresses in the package headers. IP has 2 versions: IPv4 and IPv6. IPv4 is the one that most of the websites are using currently. But IPv6 is growing as the number of IPv4 addresses are limited in number when compared to the number of users.
  2. ICMP – stands for Internet Control Message Protocol. It is encapsulated within IP datagrams and is responsible for providing hosts with information about network problems.
  3. ARP – stands for Address Resolution Protocol. Its job is to find the hardware address of a host from a known IP address. ARP has several types: Reverse ARP, Proxy ARP, Gratuitous ARP and Inverse ARP.
Host-to-Host Layer
This layer is analogous to the transport layer of the OSI model. It is responsible for end-to-end communication and error-free delivery of data. It shields the upper-layer applications from the complexities of data.
The two main protocols present in this layer are :
  1. Transmission Control Protocol (TCP) – It is known to provide reliable and error-free communication between end systems. It performs sequencing and segmentation of data. It also has acknowledgment feature and controls the flow of the data through flow control mechanism. It is a very effective protocol but has a lot of overhead due to such features. Increased overhead leads to increased cost.
  2. User Datagram Protocol (UDP) – On the other hand does not provide any such features. It is the go-to protocol if your application does not require reliable transport as it is very cost-effective. Unlike TCP, which is connection-oriented protocol, UDP is connectionless.
Application Layer
This layer performs the functions of top three layers of the OSI model: Application, Presentation and Session Layer. It is responsible for node-to-node communication and controls user-interface specifications. Some of the protocols present in this layer are: HTTP, HTTPS, FTP, TFTP, Telnet, SSH, SMTP, SNMP, NTP, DNS, DHCP, NFS, X Window, LPD. Have a look at Protocols in Application Layer for some information about these protocols.
Protocols other than those present in the linked article are :
  1. HTTP and HTTPS – HTTP stands for Hypertext transfer protocol. It is used by the World Wide Web to manage communications between web browsers and servers. HTTPS stands for HTTP-Secure. It is a combination of HTTP with SSL(Secure Socket Layer). It is efficient in cases where the browser need to fill out forms, sign in, authenticate and carry out bank transactions.
  2. SSH – SSH stands for Secure Shell. It is a terminal emulations software similar to Telnet. The reason SSH is more preferred is because of its ability to maintain the encrypted connection. It sets up a secure session over a TCP/IP connection.
  3. NTP – NTP stands for Network Time Protocol. It is used to synchronize the clocks on our computer to one standard time source. It is very useful in situations like bank transactions. Assume the following situation without the presence of NTP. Suppose you carry out a transaction, where your computer reads the time at 2:30 PM while the server records it at 2:28 PM. The server can crash very badly if it’s out of sync.

LibMqMsgque layer model

The libmqmsgque layer model is an extension to the TCP/IP layer model.

TCP/IP Application Layer
This layer is used for the libmqmsgque protocol - The protocol is the "language" of the libmqmsgque
library. The layer is defined as protocol-message-format and as protocol-flow-format.
  1. protocol-message-format - Example from protocoll_mq.h this is the "syntax" of the protocol.
    struct HdrS {
    MK_STRB ID[4] ;
    MK_STRB native ;
    MK_STRB charA ;
    union HdrIU ctxId ;
    MK_STRB charS ;
    union HdrIU bdySize ;
    MK_STRB charO ;
    MK_STRB tok[HDR_TOK_LEN] ;
    MK_STRB charT ;
    union HdrIU transSId ;
    MK_STRB charR ;
    union HdrIU routeId ;
    MK_STRB charC ;
    MK_STRB hs ;
    MK_STRB charE ;
    MK_STRB tt ;
    MK_STRB charF ;
    };
    struct BdyS {
    MK_STRB ID[4] ;
    //MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>
    union HdrIU numItems;
    MK_STRB charE ;
    };
    struct LtrS {
    MK_STRB ID[4] ;
    //MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>
    MK_BINB ltrP[HDR_ID_SIZE] ;
    MK_STRB charE ;
    };
    #define LTR_SIZE (3 + 1 + HDR_ID_SIZE + 1)
    struct RouS {
    MK_STRB ID[4] ;
    //MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>
    MK_BINB rouP[HDR_ID_SIZE] ;
    MK_STRB charE ;
    };
    #define ROU_SIZE (3 + 1 + HDR_ID_SIZE + 1)
    struct PackageS {
    struct HdrS hdr ;
    struct BdyS bdy ;
    };
  2. protocol-flow-format - this is the "grammatic" of the protocol.
LibMqMsgque Service Layer

This layer provide the MqContextC-ServiceApi ontop of the Application Layer.

The master-slave-link is used to create a mesh of nodes defined by different parent-context. The master control the slave.

The master-slave-link is used to perform the following tasks:

  • report error messages from the slave-context to the master-context
  • to create a slave-child-context if a master-child-context is created
  • to delete a slave-context if a master-context is deleted

In difference to the client-server-link the master-slave-link connect two independent parent-context in the same process or thread (e.g. node). This leads to the restriction that only the master-context can be a server-context because only one server-context per node is possible.

   node-0   |           node-1/2        |   node-3/4/5
===================================================================

| <- client/server link -> | <- client/server link -> |

             | <-- master/slave link --> |

                           |- client1-0 -|- server3 ...
             |-  server1  -|
             |             |- client1-1 -|- server4 ...
  client0-0 -|
             |-  server2  -|- client1-2 -|- server5 ...

Definition of the "master-context"

  • the master-context is a parent-context without a child-context available.
  • the master-context is a client-context or a server-context.
  • the link between the master-context and the slave-context is done using MqSlaveWorker or MqSlaveCreate

Definition of the "slave-context"

  • the slave-context is a parent-context without a child-context available.
  • the slave-context is a client-context.
  • the slave-context lifetime is controlled by the master-context.
  • the slave-context report all error-messages to the master-context.
  • the slave-context is identified by a unique-slave-id starting with 0.
  • a special form of a slave-context is a worker-context

Definition of the "worker-context"

  • the worker-context is a slave-context using the image of the master-context self.
  • the master-context can be a server-context or a client-context.
  • the worker-context is created using MqSlaveWorker
  • the worker-context is identified by a unique-slave-id starting with 0.

Definition of the "slave-id"

Definition of the "LOOPBACK" (0) slave

  •    client   |           server            |
    ===========================================
    
    | <--- client/server --->  | <-- loop --> |
    
                | <------ master/slave -----> |
    
      client -- | -- server -- | -- client -- #
                       ==             ==      #
                     server -- | -- client -- #
  • the loopback has always the slave-id = 0 .
  • the loopback has the same class as the parent, if reflection is not available the MqFactoryInitial is used.
  • the loopback is used to call a service on the same process or thread.
  • the loopback is a special filter without an additional process or thread to be started.
  • the loopback is only internal accessible
  • the service called by the loopback need the same attention as the service called by the filter, the context of the service is the loopback-context.
  • the loopback can call MqServiceCreate to create a new link between a service-token and a service-method, by default all services from the master-context (the owner of the loopback) are also accessible by the loopback.
  • in the service use MqSlaveGetMaster to get the master-context from the loopback-context.
  • Example from MyLoopServer.cc create a new loop-server
    #include  "LibMqMsgque_cc.hh"
    
    using namespace ccmqmsgque;
    
    // package-item
    class MyLoopServer : public MqContextC, public IServerSetup {
      friend class MqFactoryCT<MyLoopServer>;
    
      // set the "mydata" attribute to the master-context
      MK_STRN mydata = "Hello World";
    
      // define the factory constructor
      MyLoopServer(MK_TYP const typ, MqContextC* tmpl=NULL) : MqContextC(typ, tmpl) {};
    
      private:
        // service to serve all EXTERNAL requests for token "HLWO"
        void HLWO_srv () {
          // get the "loopback" context
          auto loop = SlaveGet(MQ_SLAVE_LOOPBACK);
          // call the LOOP service on the SAME server
          loop->Send("W", "LOOP");
          // answer HLWO with string-return from LOOP
          Send("R", "C", loop->ReadSTR());
        }
    
        // service to serve all INTERNAL requests for token "LOOP"
        void LOOP_srv () {
          // get the "master" context 
          auto master = static_cast<MyLoopServer*>(SlaveGetMaster());
          // answer LOOP with data from MASTER->mydata attribute
          Send("R", "C", master->mydata);
        }
    
        // define a service as link between the token "HLWO" and the callback "MyFirstService"
        void ServerSetup() {
          // EXTERNAL: link the "HLWO" service with "HLWO_srv"
          ServiceCreate("HLWO", MqServiceICB(&MyLoopServer::HLWO_srv));
          // INTERNAL: link the "LOOP" service with "LOOP_srv"
          SlaveGet(MQ_SLAVE_LOOPBACK)->ServiceCreate("LOOP", MqServiceICB(&MyLoopServer::LOOP_srv));
        }
    };
    
    // package-main
    int MK_CDECL main(int argc, MK_STRN argv[]) {
      MqMsgque::Setup();
    
      // create "MyLoopServer" factory… and the instance
      auto srv = MqFactoryCT<MyLoopServer>::Add("MyLoopServer")->New();
    
      try {
        srv->LinkCreate( MkBufferListC {argc, argv} );
        srv->ProcessEvent (MQ_WAIT_FOREVER);
      } catch (const std::exception& e) {
        srv->ErrorCatch(e);
      }
      srv->Exit();
    }
    

Performance analyse

The performance-test is created with:
  • pipe
    • Nhi1Exec perfclient.c --parent --wrk ? @ perfserver.c
  • spawn, fork, thread
    • Nhi1Exec -r=uds perfserver.c --spawn|fork|thread
    • Nhi1Exec -r=uds perfclient.c --parent --wrk ?
  • the number of workers are set with the –wrk option
  • the cpu is a xeon with 4/8
  • the performance is calculated as worker-context-created / time-in-sec with a ~2sec (default) measurement period.
performance-test code:
  • The test-setup is done as:
      perfclient                                worker                            perfserver
      ==========                                ======                            ==========
      |
      |- loop --wrk x
        |- MqSlaveWorker(...)               ->  worker[1] 
        |- MqSend(worker[1],"E","STR0..")   ->  PerfWorker_I160(...)
                                                |- loop endless
                                                  |- MqContextCreate(...)
                                                  |- MqLinkCreate(...)      <->   MqContextCreate(...)
                                                  |- MqContextDelete(...)   <->   MqContextDelete(...)
      |- sleep x sec
      |- loop --wrk x
        |- MqSend(worker[1],"C"..,"END0")   ->  PerfWorker_END0(...)
        |                                       |- stop loop
        |- "callback" - add number to all   <-  |- return #context 
performance-test results:
  • results generated using the debug environment
    setup –wrk # worker-context performance info
    pipe 1 2500 1000 the pipe start a new worker-context with spawn
    spawn 1 2500 <1000 same as pipe but use network-protocoll
    fork 1 3800 4000 the fork is faster than spawn
    thread 1 16500 9000 the thread is faster than fork
    pipe 4 8000 4500 the worker scale linear up to number of processors
    spawn 4 7600 <4500 -
    fork 4 23200 11500 -
    thread 4 55500 27500 -
    pipe 8 10000 5500 the additional scaling up to the max hyper-threading does not really help
    spawn 8 9100 <5500 -
    fork 8 23200 11500 -
    thread 8 55500 27500 -

example (C++ syntax)

Route-Connection-String

A client-server connection is defined with the route-connection-string as a composition of "identifier" and is build like a UNIX directory tree with the initial client as root.

Example: GUI/Data Frontend

*
|-header
|-GUI-|
frontend-| |-footer
|
|-DB
*

With --ident-from prefix|factory the identifier is defined as prefix-identifier or factory-identifer.

A route-connection-string is the path between TWO locations in the tree using the UNIX syntax:

short syntax direction
dot . current-context
double-dot .. previous-context
slash / root-context (the client)
"string" xxx next-context with name xxx

path between "DB" and "header"

  • relative: ctx.RouteCreate("../GUI/header",service)
  • absolute: ctx.RouteCreate("/frontend/GUI/header",service)

path between "footer" and "DB"

  • relative: ctx.RouteCreate("../../DB",service)
  • absolute: ctx.RouteCreate("/frontend/DB",service)

path between "frontend" and "footer"

  • relative: ctx.RouteCreate("GUI/footer",service)
  • absolute: ctx.RouteCreate("/frontend/GUI/footer",service)

Service-Layer-Routing

The LibMqMsgque-Service-Layer-Routing create proxy-services (MqServiceProxyRoundRobin ...) between frontend and footer. (example c++)

// 1. on "frontend" create a route to the service "MYSV" on the "footer" passing "GUI"
DOING: frontendCtx.RouteCreate("GUI/footer", "MYSV");
// 2. on "frontend" call the route using the "loopback" slave
DOING: frontendCtx.SlaveGet(MQ_SLAVE_LOOPBACK).Send("W", "MYSV:..@..", ..);
// 3. on "frontend/GUI" proxy "footer"
INTERNAL: frontendCtx.ServiceProxyRoundRobin("MYSV", GUICtx);
INTERNAL: GUICtx.ServiceProxyRoundRobin("MYSV", footerCtx);
// 4. on "footer" call the service "MYSV"
SETUP: footerCtx.ServiceCreate("MYSV", ...);

The MqRouteCreate will take the following action:

  1. create a round-robin-proxy on frontend and on GUI using service MYSV
  2. check if service MYSV is available on footer

Attention:

  1. With the proxy on frontend/GUI no other service with the name MYSV on frontend/GUI is possible and every service-call to frontend or GUI using the service MYSV will be redirected to the service MYSV on footer.
  2. The live-time of the Service-Layer-Routing is up to the next MqRouteDelete call using the same arguments. If the proxy-to-footer becomes invalid on a the GUI-context the routing-link will be recreated.
  3. The MqRouteCreate can be used multiple times with the same Route-Connection-String and service-token and overwrite=true to guarantee that a route is available.

Package-Layer-Routing

...

Function Documentation

◆ RouteCreate() [1/2]

void ccmqmsgque::MqContextC::RouteCreate ( const std::string & route,
const std::string & service,
MK_BOOL overwrite = false )
inline

C++: ctx.RouteCreate(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) C-API
create/delete a routing-link between context an a service using route

Definition at line 1506 of file MqContextC_inline_cc.hh.

1506 {
1507 MK_UNUSED auto ctx = getCTX(MK_NULL_NO);
1508 MkRtSetup_X(ctx);
1509 const MK_STRN route_hdl = route.c_str();
1510 const MK_STRN service_hdl = service.c_str();
1511 enum MkErrorE errVal = MqRouteCreate(ctx, route_hdl, service_hdl, overwrite);
1512 MkErrorC_Check(ctx, errVal);
1513 }
#define MK_NULL_NO
#define MK_UNUSED
const MK_STRB * MK_STRN
#define MkRtSetup_X(x)
MQ_CTX getCTX(bool nullB=MK_NULL_NO) const
return the LibMsgqueObject from current MqContextC instance
#define MqRouteCreate(...)

◆ RouteCreate() [2/2]

void ccmqmsgque::MqContextC::RouteCreate ( MK_STRN route,
MK_STRN service,
MK_BOOL overwrite = false )
inline

C++: ctx.RouteCreate(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) C-API
create/delete a routing-link between context an a service using route

Definition at line 1498 of file MqContextC_inline_cc.hh.

1498 {
1499 MK_UNUSED auto ctx = getCTX(MK_NULL_NO);
1500 MkRtSetup_X(ctx);
1501 enum MkErrorE errVal = MqRouteCreate(ctx, route, service, overwrite);
1502 MkErrorC_Check(ctx, errVal);
1503 }

◆ RouteDelete() [1/2]

void ccmqmsgque::MqContextC::RouteDelete ( const std::string & route,
const std::string & service,
MK_BOOL overwrite = false )
inline

C++: ctx.RouteDelete(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) C-API
delete a routing-link created with MqRouteCreate

Definition at line 1524 of file MqContextC_inline_cc.hh.

1524 {
1525 MK_UNUSED auto ctx = getCTX(MK_NULL_NO);
1526 MkRtSetup_X(ctx);
1527 const MK_STRN route_hdl = route.c_str();
1528 const MK_STRN service_hdl = service.c_str();
1529 enum MkErrorE errVal = MqRouteDelete(ctx, route_hdl, service_hdl, overwrite);
1530 MkErrorC_Check(ctx, errVal);
1531 }
#define MqRouteDelete(...)

◆ RouteDelete() [2/2]

void ccmqmsgque::MqContextC::RouteDelete ( MK_STRN route,
MK_STRN service,
MK_BOOL overwrite = false )
inline

C++: ctx.RouteDelete(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) C-API
delete a routing-link created with MqRouteCreate

Definition at line 1516 of file MqContextC_inline_cc.hh.

1516 {
1517 MK_UNUSED auto ctx = getCTX(MK_NULL_NO);
1518 MkRtSetup_X(ctx);
1519 enum MkErrorE errVal = MqRouteDelete(ctx, route, service, overwrite);
1520 MkErrorC_Check(ctx, errVal);
1521 }

◆ RouteGetPath()

MK_STRN ccmqmsgque::MqContextC::RouteGetPath ( )
inline

C++: MK_STRN ctx.RouteGetPath() C-API
return the absolut route-connection-string up to the current ctx

Definition at line 1613 of file MqContextC_inline_cc.hh.

1613 {
1614 MK_UNUSED auto ctx = getCTX(MK_NULL_NO);
1615 MkRtSetup_X(ctx);
1616 MK_STRN path_out;
1617 enum MkErrorE errVal = MqRouteGetPath(ctx, &path_out);
1618 MkErrorC_Check(ctx, errVal);
1619 return path_out;
1620 }
#define MqRouteGetPath(...)

◆ RouteGetTree()

MkBufferListC * ccmqmsgque::MqContextC::RouteGetTree ( )
inline

C++: MkBufferListC* ctx.RouteGetTree() C-API
create an overview about all available routing-target and services …

Definition at line 1486 of file MqContextC_inline_cc.hh.

1486 {
1487 MK_UNUSED auto ctx = getCTX(MK_NULL_NO);
1488 MkRtSetup_X(ctx);
1489 MK_BFL treeP_out;
1490 enum MkErrorE errVal = MqRouteGetTree(ctx, &treeP_out);
1491 MkErrorC_Check(ctx, errVal);
1493 }
static MkBufferListC * MkBufferListC_ObjNew(MK_RT const mkrt, MK_BFL hdl)
struct MkBufferListS * MK_BFL
#define MK_RT_CALL
#define MqRouteGetTree(...)

◆ RouteResolve() [1/2]

MqContextC_A ccmqmsgque::MqContextC::RouteResolve ( const std::string & ident,
MK_NUM retnum = -1 )
inline

C++: MqContextC_A ctx.RouteResolve(MK_STRN ident, MK_NUM retnum = -1) C-API
return a list of all context belonging to ident

Definition at line 1553 of file MqContextC_inline_cc.hh.

1553 {
1554 MK_UNUSED auto ctx = getCTX(MK_NULL_NO);
1555 MkRtSetup_X(ctx);
1556 const MK_STRN ident_hdl = ident.c_str();
1557 MQ_CTX_A __retVal__L = MqRouteResolve(ctx, ident_hdl, retnum);
1558 static __thread MK_NUM szret = 0;
1559 static __thread MqContextC* *ret = NULL;
1560 if (__retVal__L.size+1 > szret) {
1561 szret=__retVal__L.size*2+1;
1562 ret=(MqContextC**)MkSysRealloc(MK_ERROR_PANIC,ret,szret*sizeof(MqContextC*));
1563 }
1564 MK_NUM num;
1565 for (num=0; num<__retVal__L.size; num++) {
1566 ret[num] = MqContextC::MqContextC_ObjNew(MK_RT_CALL __retVal__L.data[num]);
1567 }
1568 ret[num] = NULL;
1569 return (MqContextC_A) {__retVal__L.size,ret};
1570 }
#define MK_ERROR_PANIC
int32_t MK_NUM
MK_EXTERN MK_PTR MkSysRealloc(MK_OBJ const fmtobj, MK_PTR const buf, size_t const size)
static MqContextC * MqContextC_ObjNew(MK_RT const mkrt, MQ_CTX hdl)
return MqContextC from LibMsgqueObject
#define MqRouteResolve(...)

◆ RouteResolve() [2/2]

MqContextC_A ccmqmsgque::MqContextC::RouteResolve ( MK_STRN ident,
MK_NUM retnum = -1 )
inline

C++: MqContextC_A ctx.RouteResolve(MK_STRN ident, MK_NUM retnum = -1) C-API
return a list of all context belonging to ident

Definition at line 1534 of file MqContextC_inline_cc.hh.

1534 {
1535 MK_UNUSED auto ctx = getCTX(MK_NULL_NO);
1536 MkRtSetup_X(ctx);
1537 MQ_CTX_A __retVal__L = MqRouteResolve(ctx, ident, retnum);
1538 static __thread MK_NUM szret = 0;
1539 static __thread MqContextC* *ret = NULL;
1540 if (__retVal__L.size+1 > szret) {
1541 szret=__retVal__L.size*2+1;
1542 ret=(MqContextC**)MkSysRealloc(MK_ERROR_PANIC,ret,szret*sizeof(MqContextC*));
1543 }
1544 MK_NUM num;
1545 for (num=0; num<__retVal__L.size; num++) {
1546 ret[num] = MqContextC::MqContextC_ObjNew(MK_RT_CALL __retVal__L.data[num]);
1547 }
1548 ret[num] = NULL;
1549 return (MqContextC_A) {__retVal__L.size,ret};
1550 }

◆ RouteTraverse() [1/4]

void ccmqmsgque::MqContextC::RouteTraverse ( const std::string & service,
const MkBufferListC & args )
inline

C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL) C-API
traverse a tree down and call service if available.

Definition at line 1601 of file MqContextC_inline_cc.hh.

1601 {
1602 MK_UNUSED auto ctx = getCTX(MK_NULL_NO);
1603 MkRtSetup_X(ctx);
1604 const MK_STRN service_hdl = service.c_str();
1605 const MK_BAC args_hdl = MkBufferListC::getBFL(args,MK_NULL_NO);
1606 enum MkErrorE errVal = MqRouteTraverse(ctx, service_hdl, args_hdl);
1607 MkErrorC_Check(ctx, errVal);
1608 }
MK_BFL getBFL(bool nullB=MK_NULL_NO) const
struct MkBufferListS * MK_BAC
#define MqRouteTraverse(...)

◆ RouteTraverse() [2/4]

void ccmqmsgque::MqContextC::RouteTraverse ( const std::string & service,
MkBufferListC * args = NULL )
inline

C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL) C-API
traverse a tree down and call service if available.

Definition at line 1591 of file MqContextC_inline_cc.hh.

1591 {
1592 MK_UNUSED auto ctx = getCTX(MK_NULL_NO);
1593 MkRtSetup_X(ctx);
1594 const MK_STRN service_hdl = service.c_str();
1595 MK_BAC args_hdl = MkBufferListC::getBFL(args,MK_NULL_YES);
1596 enum MkErrorE errVal = MqRouteTraverse(ctx, service_hdl, args_hdl);
1597 MkErrorC_Check(ctx, errVal);
1598 }
#define MK_NULL_YES

◆ RouteTraverse() [3/4]

void ccmqmsgque::MqContextC::RouteTraverse ( MK_STRN service,
const MkBufferListC & args )
inline

C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL) C-API
traverse a tree down and call service if available.

Definition at line 1582 of file MqContextC_inline_cc.hh.

1582 {
1583 MK_UNUSED auto ctx = getCTX(MK_NULL_NO);
1584 MkRtSetup_X(ctx);
1585 const MK_BAC args_hdl = MkBufferListC::getBFL(args,MK_NULL_NO);
1586 enum MkErrorE errVal = MqRouteTraverse(ctx, service, args_hdl);
1587 MkErrorC_Check(ctx, errVal);
1588 }

◆ RouteTraverse() [4/4]

void ccmqmsgque::MqContextC::RouteTraverse ( MK_STRN service,
MkBufferListC * args = NULL )
inline

C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL) C-API
traverse a tree down and call service if available.

Definition at line 1573 of file MqContextC_inline_cc.hh.

1573 {
1574 MK_UNUSED auto ctx = getCTX(MK_NULL_NO);
1575 MkRtSetup_X(ctx);
1576 MK_BAC args_hdl = MkBufferListC::getBFL(args,MK_NULL_YES);
1577 enum MkErrorE errVal = MqRouteTraverse(ctx, service, args_hdl);
1578 MkErrorC_Check(ctx, errVal);
1579 }