theLink 10.0 NHI1 - theKernel - theLink - theConfig - theSq3Lite - theCompiler - theBrain - theGuard
c - tcl - py - rb - jv - cc
Loading...
Searching...
No Matches
rbmqmsgque HOWTO Context

Basic information about the context usage …

BASIC

MqContextC - the class known as ctx or context is the application-handle of the application-server and also the main data-handle

The context is the package-item with the required features and created by the implementation-layer-programmer. The context can be a client or a server.

HOWTO client-context

The client-context-creation is triggerd by the software-workflow on demand. The client is calling the MqLinkCreate to create a connection to the server using the connection-arguments to specify the target.

The life-cycle of a client is:

ContextCreate create and initialize the MqContextC ...
LinkCreate make ctx to a parent-context and setup a new client-server-link
SendTT MqContextC - append a native PRIMITIVE TYPE value to the send-data-package
ReadTT read a PRIMITIVE TYPE from the read-data-package
LinkDelete close the client-server-link
ContextDelete Destructor - delete a MqContextC instance …
Exit delete the context and exit the current process or thread …

HOWTO server-context

The server-context-creation is always triggerd by the MqLinkCreate command of the client. The server is usually using a factory-constructor to call the MqContextCreate and finally to call the MqContextDelete.
The server-context is fully under control of the client.

The life-cycle of a server is:

SETUPdefine a class and add the setup/cleanup code
IServerSetup define the server-setup-interface (callback) used on startup …
ServiceCreate create a link between a service-token and a service-callback
IServerCleanup define the server-cleanup-interface (callback) used on cleanup …
ServiceDelete delete a service. …
STARTUPdefine the factory and start the listener
FactoryAdd add a new MqFactoryC identified by factory-identifier and defined by factory-constructor
FactoryNew create a new MqContextC from a MqFactoryC
LinkCreate make ctx to a parent-context and setup a new client-server-link
ProcessEvent enter the event-loop and wait for an incoming service-request. …
WORKprocess the service-calls and exit on end
ReadTT read a PRIMITIVE TYPE from the read-data-package
SendTT MqContextC - append a native PRIMITIVE TYPE value to the send-data-package
Exit delete the context and exit the current process or thread …

MqContextC CTOR and DTOR

command alias
(constructor,static) MqContextC MqContextC.Create(?tmpl:MqContextC=nil?) rbmsgque.MqMsgque.MqContextC(?tmpl:MqContextC=None?)
(destructor) ctx.Delete() ctx = nil

ADVANCED client/server

MqContextC - setup and manage a client-server-link

The client-server-link connect two context, a client-parent-context and a server-parent-context. The link can be local (connect two context on the same host) or can be remote (connect two context on different hosts). On-Top the parent-context multiple child-context are allowed.

  !on remote host!                                  !on local host!

      server1---------x                     x----------server2
         |            |                     |             |
         |     child-context-1       child-context-2      |
         |            |                     |             |                      server
 parent-context-1-----x                     x-----parent-context-2
         |                                                |
 (MqConfigS::server)                 (example: MqConfigS::server --fork --uds … --file …)
         |                                                |
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
         |                                                |
      (--tcp)                                  (--pipe, --uds, --tcp)
         |                                                |
 parent-context-1-----x                     x-----parent-context-2
         |            |                     |             |                      client
         |     child-context-1       child-context-2      |
         |            |                     |             |
         x------------x--------client-------x-------------x

                           !on local host!

Definition of a "client-context"

  • every new client-parent-context create a new client-server-link and perform the connect system-call.
  • every new client-server-link start a new server-object and creating a new server-parent-context
  • every deletion of the client-parent-context stop the previous started server-object and delete the server-parent-context
  • the client-context get all the error/warning/info messages from the server-context

Definition of a "server-context"

  • every new server-parent-context is created by request from the client-parent-context:
  • in remote mode every new server-parent-context wait for a tcp or uds connection request using the accept system-call.
client --tcp --host REMOTE_HOST --port REMOTE_PORT ... or
client --uds --file MYFILE ...
  • in local mode new server-parent-context is started by the client-parent-context as pipe:
    client @ server
    • a server-context have to implement the IServerSetup and the MqFactoryC interface.
    • a server-context have to enter the event-loop and wait for incoming service-request using MqProcessEvent together with the MqWaitOnEventE.FOREVER.

Definition of a "parent-context"

  • the parent is the first context created and is created with MqLinkCreate
  • every new parent-context on the client create a new communication to a server.
  • every new parent-context on the client connect to a new parent-context on the server
  • every new parent-context on the server create a new thread or process.
  • the parent-context control the socket communication interface

Definition of a "child-context"

  • the child-context is the second or more context and is created with MqLinkCreateChild.
  • every new child-context on the client create a new child-context on the server.
  • every new child-context is totally independent from the parent-context setup and is able to serve its own services.
  • the child-context is using the parent-context as a tunnel.
  • the child-context is using the the thread or process of the parent-context.
  • the parent of a child-context can be a parent-context or an other child-context. A tree like structure is created.
  • the child-context is using the communication interface from the parent-context.
  • if a context is deleted (parent or child) the depending context (parent or child) is deleted too.

ADVANCED master/slave

MqContextC - create and manage a slave context …

The master-slave-link is used to create a mesh of nodes defined by different parent-context. The master control the slave.

The master-slave-link is used to perform the following tasks:

  • report error messages from the slave-context to the master-context
  • to create a slave-child-context if a master-child-context is created
  • to delete a slave-context if a master-context is deleted

In difference to the client-server-link the master-slave-link connect two independent parent-context in the same process or thread (e.g. node). This leads to the restriction that only the master-context can be a server-context because only one server-context per node is possible.

   node-0   |           node-1/2        |   node-3/4/5
===================================================================

| <- client/server link -> | <- client/server link -> |

             | <-- master/slave link --> |

                           |- client1-0 -|- server3 ...
             |-  server1  -|
             |             |- client1-1 -|- server4 ...
  client0-0 -|
             |-  server2  -|- client1-2 -|- server5 ...

Definition of the "master-context"

  • the master-context is a parent-context without a child-context available.
  • the master-context is a client-context or a server-context.
  • the link between the master-context and the slave-context is done using MqSlaveWorker or MqSlaveCreate

Definition of the "slave-context"

  • the slave-context is a parent-context without a child-context available.
  • the slave-context is a client-context.
  • the slave-context lifetime is controlled by the master-context.
  • the slave-context report all error-messages to the master-context.
  • the slave-context is identified by a unique-slave-id starting with 0.
  • a special form of a slave-context is a worker-context

Definition of the "worker-context"

  • the worker-context is a slave-context using the image of the master-context self.
  • the master-context can be a server-context or a client-context.
  • the worker-context is created using MqSlaveWorker
  • the worker-context is identified by a unique-slave-id starting with 0.

Definition of the "slave-id"

  • the slave-id is defined at enum MqSlaveE
  • a slave is identified in his master-context by a slave-id
  • the slave-id work like a defined well-known-port-number in /etc/services
  • the slave-id is an integer value with valid values > 0
  • it is a good practice to plan the usage of your slave-id(s)
  • slave
    slave-id value definition
    MQ_SLAVE_MAX 1024 internal: the maximum slave-id … .
    MQ_SLAVE_USER 10 internal: start of user-defined-slave-id .
    MQ_SLAVE_LOOPBACK 0 internal: the loopback-slave-id, (call my own services) .
    MQ_SLAVE_FILTER 1 internal: the filter-slave-id, (on a master get the filter-slave) .
    MQ_SLAVE_MASTER 1 internal: the master-slave-id, (on a slave get the master) .
    MQ_SLAVE_OTHER 1 internal: on the master-ctx get the slave-ctx and on the slave-ctx get the master-ctx .
  • range
    range definition
    0 <= slave-id < MQ_SLAVE_MAX range of valid slave-id's
    0 <= slave-id < MQ_SLAVE_USER internale usage
    MQ_SLAVE_USER <= slave-id < MQ_SLAVE_MAX external usage

Definition of the "LOOPBACK" (0) slave

  •    client   |           server            |
    ===========================================
    
    | <--- client/server --->  | <-- loop --> |
    
                | <------ master/slave -----> |
    
      client -- | -- server -- | -- client -- #
                       ==             ==      #
                     server -- | -- client -- #
  • the loopback has always the slave-id = 0 .
  • the loopback has the same class as the parent, if reflection is not available the MqFactoryInitial is used.
  • the loopback is used to call a service on the same process or thread.
  • the loopback is a special filter without an additional process or thread to be started.
  • the loopback is only internal accessible
  • the service called by the loopback need the same attention as the service called by the filter, the context of the service is the loopback-context.
  • the loopback can call MqServiceCreate to create a new link between a service-token and a service-method, by default all services from the master-context (the owner of the loopback) are also accessible by the loopback.
  • in the service use MqSlaveGetMaster to get the master-context from the loopback-context.
  • Example from MyLoopServer.rb create a new loop-server
    require "rbmqmsgque"
    include RbMsgque::MqMsgque
    
    # package-item
    class MyLoopServer < MqContextC
      attr_reader :mydata
    
      # factory startup
      def initialize()
        super()
        ConfigSetServerSetup(method(:ServerSetup))
        # set the "mydata" attribute to the master-context
        @mydata = "Hello World"
      end
    
      # service to serve all EXTERNAL requests for token "HLWO"
      def HLWO_srv
        # get the "loopback" context
        loop = SlaveGet(MqSlaveE::LOOPBACK)
        # call the LOOP service on the SAME server
        loop.Send("W","LOOP")
        # answer HLWO with string-return from LOOP
        Send("R", "C", loop.ReadSTR())
      end
    
      # service to serve all INTERNAL requests for token "LOOP"
      def LOOP_srv(loop)
        # get the "master" context 
        master = loop.SlaveGetMaster()
        # answer LOOP with data from MASTER->mydata attribute
        loop.Send("R", "C", master.mydata)
      end
    
      # define a service as link between the token "HLWO" and the callback "HLWO_srv"
      def ServerSetup
        loop = SlaveGet(MqSlaveE::LOOPBACK)
        # EXTERNAL: link the "HLWO" service with "HLWO_srv"
        ServiceCreate("HLWO",method(:HLWO_srv))
        # INTERNAL: link the "LOOP" service with "LOOP_srv"
        loop.ServiceCreate("LOOP",method(:LOOP_srv))
      end
    end
    
    # ====================================================
    # package-main
    
    # create the "MyLoopServer" factory… and the instance
    srv = MqFactoryC.Add(MyLoopServer).New()
    
    begin
      srv.LinkCreate(ARGV)
      srv.ProcessEvent(MqWaitOnEventE::FOREVER)
    rescue Exception => ex
      srv.ErrorCatch(ex)
    ensure
      srv.Exit()
    end
    

Performance analyse

The performance-test is created with:
  • pipe
    • Nhi1Exec perfclient.c --parent --wrk ? @ perfserver.c
  • spawn, fork, thread
    • Nhi1Exec -r=uds perfserver.c --spawn|fork|thread
    • Nhi1Exec -r=uds perfclient.c --parent --wrk ?
  • the number of workers are set with the –wrk option
  • the cpu is a xeon with 4/8
  • the performance is calculated as worker-context-created / time-in-sec with a ~2sec (default) measurement period.
performance-test code:
  • The test-setup is done as:
      perfclient                                worker                            perfserver
      ==========                                ======                            ==========
      |
      |- loop --wrk x
        |- MqSlaveWorker(...)               ->  worker[1] 
        |- MqSend(worker[1],"E","STR0..")   ->  PerfWorker_I160(...)
                                                |- loop endless
                                                  |- MqContextCreate(...)
                                                  |- MqLinkCreate(...)      <->   MqContextCreate(...)
                                                  |- MqContextDelete(...)   <->   MqContextDelete(...)
      |- sleep x sec
      |- loop --wrk x
        |- MqSend(worker[1],"C"..,"END0")   ->  PerfWorker_END0(...)
        |                                       |- stop loop
        |- "callback" - add number to all   <-  |- return #context 
performance-test results:
  • results generated using the debug environment
    setup –wrk # worker-context performance info
    pipe 1 2500 1000 the pipe start a new worker-context with spawn
    spawn 1 2500 <1000 same as pipe but use network-protocoll
    fork 1 3800 4000 the fork is faster than spawn
    thread 1 16500 9000 the thread is faster than fork
    pipe 4 8000 4500 the worker scale linear up to number of processors
    spawn 4 7600 <4500 -
    fork 4 23200 11500 -
    thread 4 55500 27500 -
    pipe 8 10000 5500 the additional scaling up to the max hyper-threading does not really help
    spawn 8 9100 <5500 -
    fork 8 23200 11500 -
    thread 8 55500 27500 -