MqContextC - setup and manage a routing-link … More...

Collaboration diagram for MqContextC_Route_CC_API:

Functions
MkBufferListC *	ccmqmsgque::MqContextC::RouteGetTree ()
	`C++: MkBufferListC* ctx.RouteGetTree()` → C-API create an overview about all available routing-target and services …

void	ccmqmsgque::MqContextC::RouteCreate (MK_STRN route, MK_STRN service, MK_BOOL overwrite=false)
	`C++: ctx.RouteCreate(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false)` → C-API create/delete a routing-link between context an a service using route

void	ccmqmsgque::MqContextC::RouteCreate (const std::string &route, const std::string &service, MK_BOOL overwrite=false)
	`C++: ctx.RouteCreate(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false)` → C-API create/delete a routing-link between context an a service using route

void	ccmqmsgque::MqContextC::RouteDelete (MK_STRN route, MK_STRN service, MK_BOOL overwrite=false)
	`C++: ctx.RouteDelete(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false)` → C-API delete a routing-link created with MqRouteCreate

void	ccmqmsgque::MqContextC::RouteDelete (const std::string &route, const std::string &service, MK_BOOL overwrite=false)
	`C++: ctx.RouteDelete(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false)` → C-API delete a routing-link created with MqRouteCreate

MqContextC_A	ccmqmsgque::MqContextC::RouteResolve (MK_STRN ident, MK_NUM retnum=-1)
	`C++: MqContextC_A ctx.RouteResolve(MK_STRN ident, MK_NUM retnum = -1)` → C-API return a list of all context belonging to ident …

MqContextC_A	ccmqmsgque::MqContextC::RouteResolve (const std::string &ident, MK_NUM retnum=-1)
	`C++: MqContextC_A ctx.RouteResolve(MK_STRN ident, MK_NUM retnum = -1)` → C-API return a list of all context belonging to ident …

void	ccmqmsgque::MqContextC::RouteTraverse (MK_STRN service, MkBufferListC *args=NULL)
	`C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL)` → C-API traverse a tree down and call service if available.

void	ccmqmsgque::MqContextC::RouteTraverse (MK_STRN service, const MkBufferListC &args)
	`C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL)` → C-API traverse a tree down and call service if available.

void	ccmqmsgque::MqContextC::RouteTraverse (const std::string &service, MkBufferListC *args=NULL)
	`C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL)` → C-API traverse a tree down and call service if available.

void	ccmqmsgque::MqContextC::RouteTraverse (const std::string &service, const MkBufferListC &args)
	`C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL)` → C-API traverse a tree down and call service if available.

MK_STRN	ccmqmsgque::MqContextC::RouteGetPath ()
	`C++: MK_STRN ctx.RouteGetPath()` → C-API return the absolut route-connection-string up to the current ctx …

Detailed Description

MqContextC - setup and manage a routing-link …

A routing-link is the connection of two context, the route-source and the route-target, with a unspecifiend number of hub-context in between using a specific service-token.

A single context is identified by the context-identifier as returned by MqClassIdentGet.

A routing-link can be created using Service-Level-Routing or Package-Level-Routing.

The Service-Level-Routing is using a round-robin-proxy-service on the hub-context to forward an incoming package to the routing-target.
The Package-Level-Routing is using a routing-database to route the data on the package-level without a service involved.

The difference between Service-Level-Routing and Package-Level-Routing is the public versa private behaviour.

The Service-Level-Routing route EVERY package send to the hub-context using the service-token to the routing-target, this behaviour is called public.
The Package-Level-Routing route ONLY the package(s) from the creator (owner) of the route, this behaviour is called private.

Summary:

The routing-link connect two context, a source-context and a target-context using multiple hub-context in between.
The routing-link is created with the command ctx.RouteCreate(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) using a route-connection-string and a service-token.
The route-connection-string has a syntax like a UNIX-path, understand an application as a tree with the initial client as the root and multiple server and slave server as directories and subdirectories with the final service as the file-data wanted.
The Service-Level-Routing is an extension of the already available MqServiceProxyRoundRobin and MqServiceProxyCtx feature.

TCP/IP layer model

The TCP/IP model define the following layers (from: https://www.geeksforgeeks.org/tcp-ip-model/) :

Network Access Layer

This layer corresponds to the combination of Data Link Layer and Physical Layer of the OSI model. It looks out for hardware addressing and the protocols present in this layer allows for the physical transmission of data.

Internet Layer

This layer parallels the functions of OSI’s Network layer. It defines the protocols which are responsible for logical transmission of data over the entire network
The main protocols residing at this layer are :

IP – stands for Internet Protocol and it is responsible for delivering packages from the source host to the destination host by looking at the IP addresses in the package headers. IP has 2 versions: IPv4 and IPv6. IPv4 is the one that most of the websites are using currently. But IPv6 is growing as the number of IPv4 addresses are limited in number when compared to the number of users.
ICMP – stands for Internet Control Message Protocol. It is encapsulated within IP datagrams and is responsible for providing hosts with information about network problems.
ARP – stands for Address Resolution Protocol. Its job is to find the hardware address of a host from a known IP address. ARP has several types: Reverse ARP, Proxy ARP, Gratuitous ARP and Inverse ARP.

Host-to-Host Layer

This layer is analogous to the transport layer of the OSI model. It is responsible for end-to-end communication and error-free delivery of data. It shields the upper-layer applications from the complexities of data.
The two main protocols present in this layer are :

Transmission Control Protocol (TCP) – It is known to provide reliable and error-free communication between end systems. It performs sequencing and segmentation of data. It also has acknowledgment feature and controls the flow of the data through flow control mechanism. It is a very effective protocol but has a lot of overhead due to such features. Increased overhead leads to increased cost.
User Datagram Protocol (UDP) – On the other hand does not provide any such features. It is the go-to protocol if your application does not require reliable transport as it is very cost-effective. Unlike TCP, which is connection-oriented protocol, UDP is connectionless.

Application Layer

This layer performs the functions of top three layers of the OSI model: Application, Presentation and Session Layer. It is responsible for node-to-node communication and controls user-interface specifications. Some of the protocols present in this layer are: HTTP, HTTPS, FTP, TFTP, Telnet, SSH, SMTP, SNMP, NTP, DNS, DHCP, NFS, X Window, LPD. Have a look at Protocols in Application Layer for some information about these protocols.
Protocols other than those present in the linked article are :

HTTP and HTTPS – HTTP stands for Hypertext transfer protocol. It is used by the World Wide Web to manage communications between web browsers and servers. HTTPS stands for HTTP-Secure. It is a combination of HTTP with SSL(Secure Socket Layer). It is efficient in cases where the browser need to fill out forms, sign in, authenticate and carry out bank transactions.
SSH – SSH stands for Secure Shell. It is a terminal emulations software similar to Telnet. The reason SSH is more preferred is because of its ability to maintain the encrypted connection. It sets up a secure session over a TCP/IP connection.
NTP – NTP stands for Network Time Protocol. It is used to synchronize the clocks on our computer to one standard time source. It is very useful in situations like bank transactions. Assume the following situation without the presence of NTP. Suppose you carry out a transaction, where your computer reads the time at 2:30 PM while the server records it at 2:28 PM. The server can crash very badly if it’s out of sync.

LibMqMsgque layer model

The libmqmsgque layer model is an extension to the TCP/IP layer model.

TCP/IP Application Layer

This layer is used for the libmqmsgque protocol - The protocol is the "language" of the libmqmsgque
library. The layer is defined as protocol-message-format and as protocol-flow-format.

protocol-message-format - Example from protocoll_mq.h → this is the "syntax" of the protocol.

struct HdrS {

MK_STRB ID[4] ;

MK_STRB native ;

MK_STRB charA ;

union HdrIU ctxId ;

MK_STRB charS ;

union HdrIU bdySize ;

MK_STRB charO ;

MK_STRB tok[HDR_TOK_LEN] ;

MK_STRB charT ;

union HdrIU transSId ;

MK_STRB charR ;

union HdrIU routeId ;

MK_STRB charC ;

MK_STRB hs ;

MK_STRB charE ;

MK_STRB tt ;

MK_STRB charF ;

};

struct BdyS {

MK_STRB ID[4] ;

//MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>

union HdrIU numItems;

MK_STRB charE ;

};

struct LtrS {

MK_STRB ID[4] ;

//MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>

MK_BINB ltrP[HDR_ID_SIZE] ;

MK_STRB charE ;

};

#define LTR_SIZE (3 + 1 + HDR_ID_SIZE + 1)

struct RouS {

MK_STRB ID[4] ;

//MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>

MK_BINB rouP[HDR_ID_SIZE] ;

MK_STRB charE ;

};

#define ROU_SIZE (3 + 1 + HDR_ID_SIZE + 1)

struct PackageS {

struct HdrS hdr ;

struct BdyS bdy ;

};
protocol-flow-format - this is the "grammatic" of the protocol.

LibMqMsgque Service Layer

This layer provide the MqContextC-ServiceApi ontop of the Application Layer.

The master-slave-link is used to create a mesh of nodes defined by different parent-context. The master control the slave.

The master-slave-link is used to perform the following tasks:

report error messages from the slave-context to the master-context
to create a slave-child-context if a master-child-context is created
to delete a slave-context if a master-context is deleted

In difference to the client-server-link the master-slave-link connect two independent parent-context in the same process or thread (e.g. node). This leads to the restriction that only the master-context can be a server-context because only one server-context per node is possible.

   node-0   |           node-1/2        |   node-3/4/5
===================================================================

| <- client/server link -> | <- client/server link -> |

             | <-- master/slave link --> |

                           |- client1-0 -|- server3 ...
             |-  server1  -|
             |             |- client1-1 -|- server4 ...
  client0-0 -|
             |-  server2  -|- client1-2 -|- server5 ...

Definition of the "master-context"

the master-context is a parent-context without a child-context available.
the master-context is a client-context or a server-context.
the link between the master-context and the slave-context is done using MqSlaveWorker or MqSlaveCreate

Definition of the "slave-context"

the slave-context is a parent-context without a child-context available.
the slave-context is a client-context.
the slave-context lifetime is controlled by the master-context.
the slave-context report all error-messages to the master-context.
the slave-context is identified by a unique-slave-id starting with 0.
a special form of a slave-context is a worker-context

Definition of the "worker-context"

the worker-context is a slave-context using the image of the master-context self.
the master-context can be a server-context or a client-context.
the worker-context is created using MqSlaveWorker
the worker-context is identified by a unique-slave-id starting with 0.

Definition of the "slave-id"

the slave-id is defined at enum libmqmsgque::MqSlaveE
a slave is identified in his master-context by a slave-id
the slave-id work like a defined well-known-port-number in /etc/services
the slave-id is an integer value with valid values > 0
it is a good practice to plan the usage of your slave-id(s)

slave

slave-id	value	definition
libmqmsgque::MQ_SLAVE_MAX	1024	internal: the maximum slave-id … .
libmqmsgque::MQ_SLAVE_USER	10	internal: start of user-defined-slave-id .
libmqmsgque::MQ_SLAVE_LOOPBACK	0	internal: the loopback-slave-id, (call my own services) .
libmqmsgque::MQ_SLAVE_FILTER	1	internal: the filter-slave-id, (on a master get the filter-slave) .
libmqmsgque::MQ_SLAVE_MASTER	1	internal: the master-slave-id, (on a slave get the master) .
libmqmsgque::MQ_SLAVE_OTHER	1	internal: on the master-ctx get the slave-ctx and on the slave-ctx get the master-ctx .

range

range	definition
0 <= slave-id < libmqmsgque::MQ_SLAVE_MAX	range of valid slave-id's
0 <= slave-id < libmqmsgque::MQ_SLAVE_USER	internale usage
libmqmsgque::MQ_SLAVE_USER <= slave-id < libmqmsgque::MQ_SLAVE_MAX	external usage

Definition of the "LOOPBACK" (0) slave

   client   |           server            |
===========================================

| <--- client/server --->  | <-- loop --> |

            | <------ master/slave -----> |

  client -- | -- server -- | -- client -- #
                   ==             ==      #
                 server -- | -- client -- #

the loopback has always the slave-id = 0 .
the loopback has the same class as the parent, if reflection is not available the MqFactoryInitial is used.
the loopback is used to call a service on the same process or thread.
the loopback is a special filter without an additional process or thread to be started.
the loopback is only internal accessible
the service called by the loopback need the same attention as the service called by the filter, the context of the service is the loopback-context.
the loopback can call MqServiceCreate to create a new link between a service-token and a service-method, by default all services from the master-context (the owner of the loopback) are also accessible by the loopback.
in the service use MqSlaveGetMaster to get the master-context from the loopback-context.

Example from MyLoopServer.cc → create a new loop-server

#include  "LibMqMsgque_cc.hh"

using namespace ccmqmsgque;

// package-item
class MyLoopServer : public MqContextC, public MqServerSetupIF {
  friend class MqFactoryCT<MyLoopServer>;

  // set the "mydata" attribute to the master-context
  MK_STRN mydata = "Hello World";

  // define the factory constructor
  MyLoopServer(MK_TYP const typ, MqContextC* tmpl=NULL) : MqContextC(typ, tmpl) {};

  private:
    // service to serve all EXTERNAL requests for token "HLWO"
    void HLWO_srv () {
      // get the "loopback" context
      auto loop = SlaveGet(MQ_SLAVE_LOOPBACK);
      // call the LOOP service on the SAME server
      loop->Send("W", "LOOP");
      // answer HLWO with string-return from LOOP
      Send("R", "C", loop->ReadSTR());
    }

    // service to serve all INTERNAL requests for token "LOOP"
    void LOOP_srv () {
      // get the "master" context 
      auto master = static_cast<MyLoopServer*>(SlaveGetMaster());
      // answer LOOP with data from MASTER->mydata attribute
      Send("R", "C", master->mydata);
    }

    // define a service as link between the token "HLWO" and the callback "MyFirstService"
    void ServerSetup() {
      // EXTERNAL: link the "HLWO" service with "HLWO_srv"
      ServiceCreate("HLWO", MqServiceICB(&MyLoopServer::HLWO_srv));
      // INTERNAL: link the "LOOP" service with "LOOP_srv"
      SlaveGet(MQ_SLAVE_LOOPBACK)->ServiceCreate("LOOP", MqServiceICB(&MyLoopServer::LOOP_srv));
    }
};

// package-main
int MK_CDECL main(int argc, MK_STRN argv[]) {
  MqMsgque::CcMqSetup();

  // create "MyLoopServer" factory… and the instance
  auto srv = MqFactoryCT<MyLoopServer>::Add("MyLoopServer")->New();

  try {
    srv->LinkCreate( MkBufferListC {argc, argv} );
    srv->ProcessEvent (MQ_WAIT_FOREVER);
  } catch (const std::exception& e) {
    srv->ErrorCatch(e);
  }
  return srv->Exit();
}

Performance analyse

The performance-test is created with:

pipe
- Nhi1Exec perfclient.c --parent --wrk ? @ perfserver.c
spawn, fork, thread
- Nhi1Exec -r=uds perfserver.c --spawn|fork|thread
- Nhi1Exec -r=uds perfclient.c --parent --wrk ?
the number of workers are set with the –wrk option
the cpu is a xeon with 4/8
the performance is calculated as worker-context-created / time-in-sec with a ~2sec (default) measurement period.

performance-test code:

The test-setup is done as:

  perfclient                                worker                            perfserver
  ==========                                ======                            ==========
  |
  |- loop --wrk x
    |- MqSlaveWorker(...)               ->  worker[1] 
    |- MqSend(worker[1],"E","STR0..")   ->  PerfWorker_I160(...)
                                            |- loop endless
                                              |- MqContextCreate(...)
                                              |- MqLinkCreate(...)      <->   MqContextCreate(...)
                                              |- MqContextDelete(...)   <->   MqContextDelete(...)
  |- sleep x sec
  |- loop --wrk x
    |- MqSend(worker[1],"C"..,"END0")   ->  PerfWorker_END0(...)
    |                                       |- stop loop
    |- "callback" - add number to all   <-  |- return #context

Using the worker-code:
// asynchronous call

static enum MkErrorE PerfWorker_STR0( MQ_SERVICE_CALL_ARGS ) {

MK_BFL largv = NULL;

MK_BFL pargv = NULL;

MK_I32 num = MqReadI32_e(mqctx);

pargv = MkBufferListDup(MqReadBFL_e(mqctx));

enum MkErrorE ret = MK_OK;

if (num == -1) num = 999999; // endless

// start test until "doNext" is "false" or "num" exceeds

for (int i=0; doNext && i<num; i++) {

// CREATE the context

MQ_CTX ctx = MqContextCreate(NULL, mqctx);

largv = MkBufferListDup(pargv);

MqLinkCreate_C (ctx, largv) {

MqContextErrorMove(mqctx,ctx);

MqContextDelete(ctx);

goto error;

}

//MqSend(ctx,"E","MARK:C","start");

MkBufferListDelete(largv);

// DELETE the context

MqContextDelete(ctx);

// count the loops

worker_count++;

// give PerfWorker_END0 time to update "doNext"

MqProcessEvent(mqctx, MQ_WAIT_NO, MK_TIMEOUT_DEFAULT);

}

end:

//printV("ret=%i, Delete=%p\n", ret, largv)

MkBufferListDelete(largv);

MkBufferListDelete(pargv);

if (doNext == true) {

// finish the test WITHOUT "END0" -> test on --num, answer required.

MqSend_E(mqctx, "R", "I", worker_count);

}

return ret;

error:

if (doNext == true) {

ret = MkErrorStack_1X(mqctx);

} else {

ret = MkErrorReset_1X(mqctx);

}

goto end;

}

performance-test results:

results generated using the debug environment

setup	–wrk	# worker-context	performance	info
pipe	1	2500	1000	the pipe start a new worker-context with spawn
spawn	1	2500	<1000	same as pipe but use network-protocoll
fork	1	3800	4000	the fork is faster than spawn
thread	1	16500	9000	the thread is faster than fork
pipe	4	8000	4500	the worker scale linear up to number of processors
spawn	4	7600	<4500	-
fork	4	23200	11500	-
thread	4	55500	27500	-
pipe	8	10000	5500	the additional scaling up to the max hyper-threading does not really help
spawn	8	9100	<5500	-
fork	8	23200	11500	-
thread	8	55500	27500	-

example (C++ syntax)

Route-Connection-String

A client-server connection is defined with the route-connection-string as a composition of "identifier" and is build like a UNIX directory tree with the initial client as root.

Example: GUI/Data Frontend

*
 
               |-header
         |-GUI-|
frontend-|     |-footer
         |
         |-DB
 
*

With --ident-from prefix|factory the identifier is defined as prefix-identifier or factory-identifer.

A route-connection-string is the path between TWO locations in the tree using the UNIX syntax:

short	syntax	direction
dot	.	current-context
double-dot	..	previous-context
slash	/	root-context (the client)
"string"	xxx	next-context with name xxx

path between "DB" and "header"

relative: ctx.RouteCreate("../GUI/header",service)
absolute: ctx.RouteCreate("/frontend/GUI/header",service)

path between "footer" and "DB"

relative: ctx.RouteCreate("../../DB",service)
absolute: ctx.RouteCreate("/frontend/DB",service)

path between "frontend" and "footer"

relative: ctx.RouteCreate("GUI/footer",service)
absolute: ctx.RouteCreate("/frontend/GUI/footer",service)

Service-Layer-Routing

The LibMqMsgque-Service-Layer-Routing create proxy-services (MqServiceProxyRoundRobin ...) between frontend and footer. (example c++)

// 1. on "frontend" create a route to the service "MYSV" on the "footer" passing "GUI"
DOING: frontendCtx.RouteCreate("GUI/footer", "MYSV");
 
// 2. on "frontend" call the route using the "loopback" slave
DOING: frontendCtx.SlaveGet(MQ_SLAVE_LOOPBACK).Send("W", "MYSV:..@..", ..);
 
// 3. on "frontend/GUI" proxy "footer"
INTERNAL: frontendCtx.ServiceProxyRoundRobin("MYSV", GUICtx);
INTERNAL: GUICtx.ServiceProxyRoundRobin("MYSV", footerCtx);
 
// 4. on "footer" call the service "MYSV"
SETUP: footerCtx.ServiceCreate("MYSV", ...);

The MqRouteCreate will take the following action:

create a round-robin-proxy on frontend and on GUI using service MYSV
check if service MYSV is available on footer

Attention:

With the proxy on frontend/GUI no other service with the name MYSV on frontend/GUI is possible and every service-call to frontend or GUI using the service MYSV will be redirected to the service MYSV on footer.
The live-time of the Service-Layer-Routing is up to the next MqRouteDelete call using the same arguments. If the proxy-to-footer becomes invalid on a the GUI-context the routing-link will be recreated.
The MqRouteCreate can be used multiple times with the same Route-Connection-String and service-token and overwrite=true to guarantee that a route is available.

Package-Layer-Routing

...

A routing-link is the connection of two context, the route-source and the route-target, with a unspecifiend number of hub-context in between using a specific service-token.

A single context is identified by the context-identifier as returned by MqClassIdentGet.

A routing-link can be created using Service-Level-Routing or Package-Level-Routing.

The Service-Level-Routing is using a round-robin-proxy-service on the hub-context to forward an incoming package to the routing-target.
The Package-Level-Routing is using a routing-database to route the data on the package-level without a service involved.

The difference between Service-Level-Routing and Package-Level-Routing is the public versa private behaviour.

The Service-Level-Routing route EVERY package send to the hub-context using the service-token to the routing-target, this behaviour is called public.
The Package-Level-Routing route ONLY the package(s) from the creator (owner) of the route, this behaviour is called private.

Summary:

The routing-link connect two context, a source-context and a target-context using multiple hub-context in between.
The routing-link is created with the command ctx.RouteCreate(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) using a route-connection-string and a service-token.
The route-connection-string has a syntax like a UNIX-path, understand an application as a tree with the initial client as the root and multiple server and slave server as directories and subdirectories with the final service as the file-data wanted.
The Service-Level-Routing is an extension of the already available MqServiceProxyRoundRobin and MqServiceProxyCtx feature.

TCP/IP layer model

The TCP/IP model define the following layers (from: https://www.geeksforgeeks.org/tcp-ip-model/) :

Network Access Layer

This layer corresponds to the combination of Data Link Layer and Physical Layer of the OSI model. It looks out for hardware addressing and the protocols present in this layer allows for the physical transmission of data.

Internet Layer

This layer parallels the functions of OSI’s Network layer. It defines the protocols which are responsible for logical transmission of data over the entire network
The main protocols residing at this layer are :

IP – stands for Internet Protocol and it is responsible for delivering packages from the source host to the destination host by looking at the IP addresses in the package headers. IP has 2 versions: IPv4 and IPv6. IPv4 is the one that most of the websites are using currently. But IPv6 is growing as the number of IPv4 addresses are limited in number when compared to the number of users.
ICMP – stands for Internet Control Message Protocol. It is encapsulated within IP datagrams and is responsible for providing hosts with information about network problems.
ARP – stands for Address Resolution Protocol. Its job is to find the hardware address of a host from a known IP address. ARP has several types: Reverse ARP, Proxy ARP, Gratuitous ARP and Inverse ARP.

Host-to-Host Layer

This layer is analogous to the transport layer of the OSI model. It is responsible for end-to-end communication and error-free delivery of data. It shields the upper-layer applications from the complexities of data.
The two main protocols present in this layer are :

Transmission Control Protocol (TCP) – It is known to provide reliable and error-free communication between end systems. It performs sequencing and segmentation of data. It also has acknowledgment feature and controls the flow of the data through flow control mechanism. It is a very effective protocol but has a lot of overhead due to such features. Increased overhead leads to increased cost.
User Datagram Protocol (UDP) – On the other hand does not provide any such features. It is the go-to protocol if your application does not require reliable transport as it is very cost-effective. Unlike TCP, which is connection-oriented protocol, UDP is connectionless.

Application Layer

This layer performs the functions of top three layers of the OSI model: Application, Presentation and Session Layer. It is responsible for node-to-node communication and controls user-interface specifications. Some of the protocols present in this layer are: HTTP, HTTPS, FTP, TFTP, Telnet, SSH, SMTP, SNMP, NTP, DNS, DHCP, NFS, X Window, LPD. Have a look at Protocols in Application Layer for some information about these protocols.
Protocols other than those present in the linked article are :

HTTP and HTTPS – HTTP stands for Hypertext transfer protocol. It is used by the World Wide Web to manage communications between web browsers and servers. HTTPS stands for HTTP-Secure. It is a combination of HTTP with SSL(Secure Socket Layer). It is efficient in cases where the browser need to fill out forms, sign in, authenticate and carry out bank transactions.
SSH – SSH stands for Secure Shell. It is a terminal emulations software similar to Telnet. The reason SSH is more preferred is because of its ability to maintain the encrypted connection. It sets up a secure session over a TCP/IP connection.
NTP – NTP stands for Network Time Protocol. It is used to synchronize the clocks on our computer to one standard time source. It is very useful in situations like bank transactions. Assume the following situation without the presence of NTP. Suppose you carry out a transaction, where your computer reads the time at 2:30 PM while the server records it at 2:28 PM. The server can crash very badly if it’s out of sync.

LibMqMsgque layer model

The libmqmsgque layer model is an extension to the TCP/IP layer model.

TCP/IP Application Layer

This layer is used for the libmqmsgque protocol - The protocol is the "language" of the libmqmsgque
library. The layer is defined as protocol-message-format and as protocol-flow-format.

protocol-message-format - Example from protocoll_mq.h → this is the "syntax" of the protocol.

struct HdrS {

MK_STRB ID[4] ;

MK_STRB native ;

MK_STRB charA ;

union HdrIU ctxId ;

MK_STRB charS ;

union HdrIU bdySize ;

MK_STRB charO ;

MK_STRB tok[HDR_TOK_LEN] ;

MK_STRB charT ;

union HdrIU transSId ;

MK_STRB charR ;

union HdrIU routeId ;

MK_STRB charC ;

MK_STRB hs ;

MK_STRB charE ;

MK_STRB tt ;

MK_STRB charF ;

};

struct BdyS {

MK_STRB ID[4] ;

//MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>

union HdrIU numItems;

MK_STRB charE ;

};

struct LtrS {

MK_STRB ID[4] ;

//MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>

MK_BINB ltrP[HDR_ID_SIZE] ;

MK_STRB charE ;

};

#define LTR_SIZE (3 + 1 + HDR_ID_SIZE + 1)

struct RouS {

MK_STRB ID[4] ;

//MK_STRB charN ; ///< <TT>size: 1 .............. -> total: 4 .. -> character .. -> "+"</TT>

MK_BINB rouP[HDR_ID_SIZE] ;

MK_STRB charE ;

};

#define ROU_SIZE (3 + 1 + HDR_ID_SIZE + 1)

struct PackageS {

struct HdrS hdr ;

struct BdyS bdy ;

};
protocol-flow-format - this is the "grammatic" of the protocol.

LibMqMsgque Service Layer

This layer provide the MqContextC-ServiceApi ontop of the Application Layer.

The master-slave-link is used to create a mesh of nodes defined by different parent-context. The master control the slave.

The master-slave-link is used to perform the following tasks:

report error messages from the slave-context to the master-context
to create a slave-child-context if a master-child-context is created
to delete a slave-context if a master-context is deleted

In difference to the client-server-link the master-slave-link connect two independent parent-context in the same process or thread (e.g. node). This leads to the restriction that only the master-context can be a server-context because only one server-context per node is possible.

   node-0   |           node-1/2        |   node-3/4/5
===================================================================

| <- client/server link -> | <- client/server link -> |

             | <-- master/slave link --> |

                           |- client1-0 -|- server3 ...
             |-  server1  -|
             |             |- client1-1 -|- server4 ...
  client0-0 -|
             |-  server2  -|- client1-2 -|- server5 ...

Definition of the "master-context"

the master-context is a parent-context without a child-context available.
the master-context is a client-context or a server-context.
the link between the master-context and the slave-context is done using MqSlaveWorker or MqSlaveCreate

Definition of the "slave-context"

the slave-context is a parent-context without a child-context available.
the slave-context is a client-context.
the slave-context lifetime is controlled by the master-context.
the slave-context report all error-messages to the master-context.
the slave-context is identified by a unique-slave-id starting with 0.
a special form of a slave-context is a worker-context

Definition of the "worker-context"

the worker-context is a slave-context using the image of the master-context self.
the master-context can be a server-context or a client-context.
the worker-context is created using MqSlaveWorker
the worker-context is identified by a unique-slave-id starting with 0.

Definition of the "slave-id"

the slave-id is defined at enum libmqmsgque::MqSlaveE
a slave is identified in his master-context by a slave-id
the slave-id work like a defined well-known-port-number in /etc/services
the slave-id is an integer value with valid values > 0
it is a good practice to plan the usage of your slave-id(s)

slave

slave-id	value	definition
libmqmsgque::MQ_SLAVE_MAX	1024	internal: the maximum slave-id … .
libmqmsgque::MQ_SLAVE_USER	10	internal: start of user-defined-slave-id .
libmqmsgque::MQ_SLAVE_LOOPBACK	0	internal: the loopback-slave-id, (call my own services) .
libmqmsgque::MQ_SLAVE_FILTER	1	internal: the filter-slave-id, (on a master get the filter-slave) .
libmqmsgque::MQ_SLAVE_MASTER	1	internal: the master-slave-id, (on a slave get the master) .
libmqmsgque::MQ_SLAVE_OTHER	1	internal: on the master-ctx get the slave-ctx and on the slave-ctx get the master-ctx .

range

range	definition
0 <= slave-id < libmqmsgque::MQ_SLAVE_MAX	range of valid slave-id's
0 <= slave-id < libmqmsgque::MQ_SLAVE_USER	internale usage
libmqmsgque::MQ_SLAVE_USER <= slave-id < libmqmsgque::MQ_SLAVE_MAX	external usage

Definition of the "LOOPBACK" (0) slave

   client   |           server            |
===========================================

| <--- client/server --->  | <-- loop --> |

            | <------ master/slave -----> |

  client -- | -- server -- | -- client -- #
                   ==             ==      #
                 server -- | -- client -- #

the loopback has always the slave-id = 0 .
the loopback has the same class as the parent, if reflection is not available the MqFactoryInitial is used.
the loopback is used to call a service on the same process or thread.
the loopback is a special filter without an additional process or thread to be started.
the loopback is only internal accessible
the service called by the loopback need the same attention as the service called by the filter, the context of the service is the loopback-context.
the loopback can call MqServiceCreate to create a new link between a service-token and a service-method, by default all services from the master-context (the owner of the loopback) are also accessible by the loopback.
in the service use MqSlaveGetMaster to get the master-context from the loopback-context.

Example from MyLoopServer.cc → create a new loop-server

#include  "LibMqMsgque_cc.hh"

using namespace ccmqmsgque;

// package-item
class MyLoopServer : public MqContextC, public MqServerSetupIF {
  friend class MqFactoryCT<MyLoopServer>;

  // set the "mydata" attribute to the master-context
  MK_STRN mydata = "Hello World";

  // define the factory constructor
  MyLoopServer(MK_TYP const typ, MqContextC* tmpl=NULL) : MqContextC(typ, tmpl) {};

  private:
    // service to serve all EXTERNAL requests for token "HLWO"
    void HLWO_srv () {
      // get the "loopback" context
      auto loop = SlaveGet(MQ_SLAVE_LOOPBACK);
      // call the LOOP service on the SAME server
      loop->Send("W", "LOOP");
      // answer HLWO with string-return from LOOP
      Send("R", "C", loop->ReadSTR());
    }

    // service to serve all INTERNAL requests for token "LOOP"
    void LOOP_srv () {
      // get the "master" context 
      auto master = static_cast<MyLoopServer*>(SlaveGetMaster());
      // answer LOOP with data from MASTER->mydata attribute
      Send("R", "C", master->mydata);
    }

    // define a service as link between the token "HLWO" and the callback "MyFirstService"
    void ServerSetup() {
      // EXTERNAL: link the "HLWO" service with "HLWO_srv"
      ServiceCreate("HLWO", MqServiceICB(&MyLoopServer::HLWO_srv));
      // INTERNAL: link the "LOOP" service with "LOOP_srv"
      SlaveGet(MQ_SLAVE_LOOPBACK)->ServiceCreate("LOOP", MqServiceICB(&MyLoopServer::LOOP_srv));
    }
};

// package-main
int MK_CDECL main(int argc, MK_STRN argv[]) {
  MqMsgque::CcMqSetup();

  // create "MyLoopServer" factory… and the instance
  auto srv = MqFactoryCT<MyLoopServer>::Add("MyLoopServer")->New();

  try {
    srv->LinkCreate( MkBufferListC {argc, argv} );
    srv->ProcessEvent (MQ_WAIT_FOREVER);
  } catch (const std::exception& e) {
    srv->ErrorCatch(e);
  }
  return srv->Exit();
}

Performance analyse

The performance-test is created with:

pipe
- Nhi1Exec perfclient.c --parent --wrk ? @ perfserver.c
spawn, fork, thread
- Nhi1Exec -r=uds perfserver.c --spawn|fork|thread
- Nhi1Exec -r=uds perfclient.c --parent --wrk ?
the number of workers are set with the –wrk option
the cpu is a xeon with 4/8
the performance is calculated as worker-context-created / time-in-sec with a ~2sec (default) measurement period.

performance-test code:

The test-setup is done as:

  perfclient                                worker                            perfserver
  ==========                                ======                            ==========
  |
  |- loop --wrk x
    |- MqSlaveWorker(...)               ->  worker[1] 
    |- MqSend(worker[1],"E","STR0..")   ->  PerfWorker_I160(...)
                                            |- loop endless
                                              |- MqContextCreate(...)
                                              |- MqLinkCreate(...)      <->   MqContextCreate(...)
                                              |- MqContextDelete(...)   <->   MqContextDelete(...)
  |- sleep x sec
  |- loop --wrk x
    |- MqSend(worker[1],"C"..,"END0")   ->  PerfWorker_END0(...)
    |                                       |- stop loop
    |- "callback" - add number to all   <-  |- return #context

Using the worker-code:
// asynchronous call

static enum MkErrorE PerfWorker_STR0( MQ_SERVICE_CALL_ARGS ) {

MK_BFL largv = NULL;

MK_BFL pargv = NULL;

MK_I32 num = MqReadI32_e(mqctx);

pargv = MkBufferListDup(MqReadBFL_e(mqctx));

enum MkErrorE ret = MK_OK;

if (num == -1) num = 999999; // endless

// start test until "doNext" is "false" or "num" exceeds

for (int i=0; doNext && i<num; i++) {

// CREATE the context

MQ_CTX ctx = MqContextCreate(NULL, mqctx);

largv = MkBufferListDup(pargv);

MqLinkCreate_C (ctx, largv) {

MqContextErrorMove(mqctx,ctx);

MqContextDelete(ctx);

goto error;

}

//MqSend(ctx,"E","MARK:C","start");

MkBufferListDelete(largv);

// DELETE the context

MqContextDelete(ctx);

// count the loops

worker_count++;

// give PerfWorker_END0 time to update "doNext"

MqProcessEvent(mqctx, MQ_WAIT_NO, MK_TIMEOUT_DEFAULT);

}

end:

//printV("ret=%i, Delete=%p\n", ret, largv)

MkBufferListDelete(largv);

MkBufferListDelete(pargv);

if (doNext == true) {

// finish the test WITHOUT "END0" -> test on --num, answer required.

MqSend_E(mqctx, "R", "I", worker_count);

}

return ret;

error:

if (doNext == true) {

ret = MkErrorStack_1X(mqctx);

} else {

ret = MkErrorReset_1X(mqctx);

}

goto end;

}

performance-test results:

results generated using the debug environment

setup	–wrk	# worker-context	performance	info
pipe	1	2500	1000	the pipe start a new worker-context with spawn
spawn	1	2500	<1000	same as pipe but use network-protocoll
fork	1	3800	4000	the fork is faster than spawn
thread	1	16500	9000	the thread is faster than fork
pipe	4	8000	4500	the worker scale linear up to number of processors
spawn	4	7600	<4500	-
fork	4	23200	11500	-
thread	4	55500	27500	-
pipe	8	10000	5500	the additional scaling up to the max hyper-threading does not really help
spawn	8	9100	<5500	-
fork	8	23200	11500	-
thread	8	55500	27500	-

example (C++ syntax)

Route-Connection-String

A client-server connection is defined with the route-connection-string as a composition of "identifier" and is build like a UNIX directory tree with the initial client as root.

Example: GUI/Data Frontend

*
 
               |-header
         |-GUI-|
frontend-|     |-footer
         |
         |-DB
 
*

With --ident-from prefix|factory the identifier is defined as prefix-identifier or factory-identifer.

A route-connection-string is the path between TWO locations in the tree using the UNIX syntax:

short	syntax	direction
dot	.	current-context
double-dot	..	previous-context
slash	/	root-context (the client)
"string"	xxx	next-context with name xxx

path between "DB" and "header"

relative: ctx.RouteCreate("../GUI/header",service)
absolute: ctx.RouteCreate("/frontend/GUI/header",service)

path between "footer" and "DB"

relative: ctx.RouteCreate("../../DB",service)
absolute: ctx.RouteCreate("/frontend/DB",service)

path between "frontend" and "footer"

relative: ctx.RouteCreate("GUI/footer",service)
absolute: ctx.RouteCreate("/frontend/GUI/footer",service)

Service-Layer-Routing

The LibMqMsgque-Service-Layer-Routing create proxy-services (MqServiceProxyRoundRobin ...) between frontend and footer. (example c++)

// 1. on "frontend" create a route to the service "MYSV" on the "footer" passing "GUI"
DOING: frontendCtx.RouteCreate("GUI/footer", "MYSV");
 
// 2. on "frontend" call the route using the "loopback" slave
DOING: frontendCtx.SlaveGet(MQ_SLAVE_LOOPBACK).Send("W", "MYSV:..@..", ..);
 
// 3. on "frontend/GUI" proxy "footer"
INTERNAL: frontendCtx.ServiceProxyRoundRobin("MYSV", GUICtx);
INTERNAL: GUICtx.ServiceProxyRoundRobin("MYSV", footerCtx);
 
// 4. on "footer" call the service "MYSV"
SETUP: footerCtx.ServiceCreate("MYSV", ...);

The MqRouteCreate will take the following action:

create a round-robin-proxy on frontend and on GUI using service MYSV
check if service MYSV is available on footer

Attention:

With the proxy on frontend/GUI no other service with the name MYSV on frontend/GUI is possible and every service-call to frontend or GUI using the service MYSV will be redirected to the service MYSV on footer.
The live-time of the Service-Layer-Routing is up to the next MqRouteDelete call using the same arguments. If the proxy-to-footer becomes invalid on a the GUI-context the routing-link will be recreated.
The MqRouteCreate can be used multiple times with the same Route-Connection-String and service-token and overwrite=true to guarantee that a route is available.

Package-Layer-Routing

...

Function Documentation

◆ RouteCreate() [1/2]

void ccmqmsgque::MqContextC::RouteCreate	(	const std::string &	route,
		const std::string &	service,
		MK_BOOL	overwrite = false )

inline

C++: ctx.RouteCreate(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) → C-API
create/delete a routing-link between context an a service using route

Definition at line 1344 of file MqContextC_inline_cc.hh.

                                                                                                           {
    MK_UNUSED auto ctx = getCTX();
    const MK_STRN route_hdl = route.c_str();
    const MK_STRN service_hdl = service.c_str();
    enum MkErrorE errVal = MqRouteCreate(ctx, route_hdl, service_hdl, overwrite);
    MkErrorC_Check(ctx, errVal);
  }

◆ RouteCreate() [2/2]

void ccmqmsgque::MqContextC::RouteCreate	(	MK_STRN	route,
		MK_STRN	service,
		MK_BOOL	overwrite = false )

inline

C++: ctx.RouteCreate(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) → C-API
create/delete a routing-link between context an a service using route

Definition at line 1337 of file MqContextC_inline_cc.hh.

                                                                                         {
    MK_UNUSED auto ctx = getCTX();
    enum MkErrorE errVal = MqRouteCreate(ctx, route, service, overwrite);
    MkErrorC_Check(ctx, errVal);
  }

◆ RouteDelete() [1/2]

void ccmqmsgque::MqContextC::RouteDelete	(	const std::string &	route,
		const std::string &	service,
		MK_BOOL	overwrite = false )

inline

C++: ctx.RouteDelete(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) → C-API
delete a routing-link created with MqRouteCreate

Definition at line 1360 of file MqContextC_inline_cc.hh.

                                                                                                           {
    MK_UNUSED auto ctx = getCTX();
    const MK_STRN route_hdl = route.c_str();
    const MK_STRN service_hdl = service.c_str();
    enum MkErrorE errVal = MqRouteDelete(ctx, route_hdl, service_hdl, overwrite);
    MkErrorC_Check(ctx, errVal);
  }

◆ RouteDelete() [2/2]

void ccmqmsgque::MqContextC::RouteDelete	(	MK_STRN	route,
		MK_STRN	service,
		MK_BOOL	overwrite = false )

inline

C++: ctx.RouteDelete(MK_STRN route, MK_STRN service, MK_BOOL overwrite = false) → C-API
delete a routing-link created with MqRouteCreate

Definition at line 1353 of file MqContextC_inline_cc.hh.

                                                                                         {
    MK_UNUSED auto ctx = getCTX();
    enum MkErrorE errVal = MqRouteDelete(ctx, route, service, overwrite);
    MkErrorC_Check(ctx, errVal);
  }

◆ RouteGetPath()

MK_STRN ccmqmsgque::MqContextC::RouteGetPath ( )

inline

C++: MK_STRN ctx.RouteGetPath() → C-API
return the absolut route-connection-string up to the current ctx …

Definition at line 1442 of file MqContextC_inline_cc.hh.

                                            {
    MK_UNUSED auto ctx = getCTX();
    MK_STRN path_out;
    enum MkErrorE errVal = MqRouteGetPath(ctx, &path_out);
    MkErrorC_Check(ctx, errVal);
    return path_out;
  }

◆ RouteGetTree()

MkBufferListC * ccmqmsgque::MqContextC::RouteGetTree ( )

inline

C++: MkBufferListC* ctx.RouteGetTree() → C-API
create an overview about all available routing-target and services …

Definition at line 1326 of file MqContextC_inline_cc.hh.

                                                   {
    MK_UNUSED auto ctx = getCTX();
    MK_BFL treeP_out;
    enum MkErrorE errVal = MqRouteGetTree(ctx, &treeP_out);
    MkErrorC_Check(ctx, errVal);
    return MkBufferListC::MkBufferListC_ObjNew(MK_RT_CALL treeP_out);
  }

◆ RouteResolve() [1/2]

MqContextC_A ccmqmsgque::MqContextC::RouteResolve	(	const std::string &	ident,
		MK_NUM	retnum = -1 )

inline

C++: MqContextC_A ctx.RouteResolve(MK_STRN ident, MK_NUM retnum = -1) → C-API
return a list of all context belonging to ident …

Definition at line 1387 of file MqContextC_inline_cc.hh.

                                                                                      {
    MK_UNUSED auto ctx = getCTX();
    const MK_STRN ident_hdl = ident.c_str();
    MQ_CTX_A __retVal__L = MqRouteResolve(ctx, ident_hdl, retnum);
    static __thread MK_NUM szret = 0;
    static __thread MqContextC* *ret = NULL;
    if (__retVal__L.size+1 > szret) {
      szret=__retVal__L.size*2+1;
      ret=(MqContextC**)MkSysRealloc(MK_ERROR_PANIC,ret,szret*sizeof(MqContextC*));
    }
    MK_NUM num;
    for (num=0; num<__retVal__L.size; num++) {
      ret[num] = MqContextC::MqContextC_ObjNew(MK_RT_CALL __retVal__L.data[num]);
    }
    ret[num] = NULL;
    return (MqContextC_A) {__retVal__L.size,ret};
  }

◆ RouteResolve() [2/2]

MqContextC_A ccmqmsgque::MqContextC::RouteResolve	(	MK_STRN	ident,
		MK_NUM	retnum = -1 )

inline

C++: MqContextC_A ctx.RouteResolve(MK_STRN ident, MK_NUM retnum = -1) → C-API
return a list of all context belonging to ident …

Definition at line 1369 of file MqContextC_inline_cc.hh.

                                                                             {
    MK_UNUSED auto ctx = getCTX();
    MQ_CTX_A __retVal__L = MqRouteResolve(ctx, ident, retnum);
    static __thread MK_NUM szret = 0;
    static __thread MqContextC* *ret = NULL;
    if (__retVal__L.size+1 > szret) {
      szret=__retVal__L.size*2+1;
      ret=(MqContextC**)MkSysRealloc(MK_ERROR_PANIC,ret,szret*sizeof(MqContextC*));
    }
    MK_NUM num;
    for (num=0; num<__retVal__L.size; num++) {
      ret[num] = MqContextC::MqContextC_ObjNew(MK_RT_CALL __retVal__L.data[num]);
    }
    ret[num] = NULL;
    return (MqContextC_A) {__retVal__L.size,ret};
  }

◆ RouteTraverse() [1/4]

void ccmqmsgque::MqContextC::RouteTraverse	(	const std::string &	service,
		const MkBufferListC &	args )

inline

C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL) → C-API
traverse a tree down and call service if available.

Definition at line 1431 of file MqContextC_inline_cc.hh.

                                                                                             {
    MK_UNUSED auto ctx = getCTX();
    const MK_STRN service_hdl = service.c_str();
    const MK_BAC args_hdl = MkBufferListC::getBFL(args);
    enum MkErrorE errVal = MqRouteTraverse(ctx, service_hdl, args_hdl);
    MkErrorC_Check(ctx, errVal);
  }

◆ RouteTraverse() [2/4]

void ccmqmsgque::MqContextC::RouteTraverse	(	const std::string &	service,
		MkBufferListC *	args = NULL )

inline

C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL) → C-API
traverse a tree down and call service if available.

Definition at line 1422 of file MqContextC_inline_cc.hh.

                                                                                       {
    MK_UNUSED auto ctx = getCTX();
    const MK_STRN service_hdl = service.c_str();
    MK_BAC args_hdl = MkBufferListC::getBFL__null_allow(args);
    enum MkErrorE errVal = MqRouteTraverse(ctx, service_hdl, args_hdl);
    MkErrorC_Check(ctx, errVal);
  }

◆ RouteTraverse() [3/4]

void ccmqmsgque::MqContextC::RouteTraverse	(	MK_STRN	service,
		const MkBufferListC &	args )

inline

C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL) → C-API
traverse a tree down and call service if available.

Definition at line 1414 of file MqContextC_inline_cc.hh.

                                                                                    {
    MK_UNUSED auto ctx = getCTX();
    const MK_BAC args_hdl = MkBufferListC::getBFL(args);
    enum MkErrorE errVal = MqRouteTraverse(ctx, service, args_hdl);
    MkErrorC_Check(ctx, errVal);
  }

◆ RouteTraverse() [4/4]

void ccmqmsgque::MqContextC::RouteTraverse	(	MK_STRN	service,
		MkBufferListC *	args = NULL )

inline

C++: ctx.RouteTraverse(MK_STRN service, MkBufferListC* args = NULL) → C-API
traverse a tree down and call service if available.

Definition at line 1406 of file MqContextC_inline_cc.hh.

                                                                              {
    MK_UNUSED auto ctx = getCTX();
    MK_BAC args_hdl = MkBufferListC::getBFL__null_allow(args);
    enum MkErrorE errVal = MqRouteTraverse(ctx, service, args_hdl);
    MkErrorC_Check(ctx, errVal);
  }

Functions

Detailed Description

TCP/IP layer model

LibMqMsgque layer model

example (C++ syntax)

TCP/IP layer model

LibMqMsgque layer model

example (C++ syntax)

Function Documentation

◆ RouteCreate() [1/2]

◆ RouteCreate() [2/2]

◆ RouteDelete() [1/2]

◆ RouteDelete() [2/2]

◆ RouteGetPath()

◆ RouteGetTree()

◆ RouteResolve() [1/2]

◆ RouteResolve() [2/2]

◆ RouteTraverse() [1/4]

◆ RouteTraverse() [2/4]

◆ RouteTraverse() [3/4]

◆ RouteTraverse() [4/4]