Questions and answers related to libmsgaue usability.
1. How to make different services communicate?
- every libmqmsgque communication link is a link between one client and one server
- the link is created and maintained by libmqmsgque as a socket communication using unix-domain-sockets, inet-domain-sockets or pipes
- client and server are using the same communication library called libmqmsgque
2. What exactly does 'libmqmsgque'?
- find and start the service (e.g. server)
- setup and maintain the socket communication interface between client and server
- choose the right byte-order (little/big endian)
- verify 'service' api version
- if an error happen, propagate the error to the client
- do all the send and recv stuff
- do (if necessary) the casting of different types
- provide an event-loop to smoothly integrate multiple tasks
- after the work was done do a clean shutdown of the communication
3. How does one service do to understand the data another service is sending to it?
- DEFINITION: understand means knowing the exact definition of the procedure called.
- client and server using the same protocol as part of libmqmsgque and this protocol include the service called together with the arguments provided in a libmqmsgque package. The type of a argument can be string, binary, list or native. The native type is defined in MkBufferAtomU. The type string is casted into native if the application require this.
- the protocol layer is totally hidden to the programmer
- the server provides a service identified as a 4 byte character string
- the client is calling the service with the arguments included (example calling service
ECOI
):
using libmqmsgque code MqSendSTART (ctx);
MqSendI32 (ctx, 567);
MkErrorCheck (MqSendEND_AND_WAIT (ctx, "ECOI", 10));
using ccmqmsgque code ctx.SendSTART ();
ctx.SendI32 (567);
ctx.SendEND_AND_WAIT ("ECOI", 10);
using csmqmsgque code ctx.SendSTART ();
ctx.SendI32 (567);
ctx.SendEND_AND_WAIT ("ECOI", 10);
using tclmqmsgque code $ctx SendSTART
$ctx SendI32 567
$ctx SendEND_AND_WAIT ECOI 10
using jvmqmsgque code ctx.SendSTART();
ctx.SendI32(567);
ctx.SendEND_AND_WAIT("ECOI", 10);
using pymqmsgque code ctx.SendSTART()
ctx.SendI32(567)
ctx.SendEND_AND_WAIT("ECOI", 10)
using rbmqmsgque code ctx.SendSTART()
ctx.SendI32(567)
ctx.SendEND_AND_WAIT("ECOI", 10)
using plmqmsgque code $ctx->SendSTART();
$ctx->SendI32(567);
$ctx->SendEND_AND_WAIT("ECOI", 10);
and wait (MqSendEND_AND_WAIT) or does not wait (MqSendEND) for an answer.
- the server reply the client-service-call with the return data.
- the number and the type of the call-arguments and the number and the type of the reply-arguments are not agreed by the protocol but the type checking is strict. strict mean that every argument send type have to be read by the same type with the exception that a cast from and to type C is supported. Every type mismatch will result in an error even if the send value will fit into the target type.
4. It is likely that libmqmsgque will similarly need a common basic format?
- yes, the data-package-format is defined and libmqmsgque is used on the both endpoints of the communication to read this format.
5. What does libmqmsgque not do?
- no login verification
- no encoding or decoding of binary data
- no encryption or decryption of binary data
6. What is the overhead of libmqmsgque?
- the latest information is available at:
performance
7. It's not clear to me how many instances of a server might exist
- MqContextC LINK API
- every client-parent-context start its own server
- every started server is totally independent from the other server might exist
- advantage: easy load balancing because every server process is able to use an other processor
- advantage: no namespace pollution by the server
- advantage: no crash dependency between the client and the server. (If the server crashes the client still continue to work)
- advantage: using different programming languages between client and server
8. It's not clear where the state for a service resides
- DEFINITION: The "state" ought to be the totality of everything that can make two identical sequences of libmqmsgque library commands return different results. A typical case could be if one wants to wrap a Tcl shell up as a libmqmsgque service; this then has an internal state (values of variables, commands defined, etc.), and it would have to be part of the total state of the service.
- INTRO: MqContextC LINK API
- every client creating a new parent-context is creating a new parent-context on the server.
- a new parent context on is in fact a new class-instance able to store per-context data.
- the context-class-instance is defined as: MqContextC
8.a When libmqmsgque was first being mentioned, it was described as a replacement for shared libraries.
- from the client point of view it is an replacement
- OLD: I have to link with libXXX.so
- NEW: I have to link with libmqmsgque.so and using service XXX
- NEW: no need for libXXX.so, no linking, no searching for the right library for my OS, no errors on linking again a wrong version of library, no need for installing all libraries on my host and last but not least your tcl event-loop will always been working
8.b My expectations were conditioned by how I understand shared libraries operate. It is now more clear that libmqmsgque is an alternative to shared libraries that has characteristics much different from them.
- I personal call it a network shared library or just a service
- from a programming point of view writing applications based on services is much more easy as writing applications based on shared libraries
- reuse service based code is a lot easier as reusing shared library based code
(Example: libXXX.so need a libperlXXX.so, libtclXXX.so, libphpXXX.so, libjavaXXX.so. libc#XXX.so, infinite long ... interface library but only one libmqmsgque.so interface library)
9. It's not clear how services are shared
- DEFINITION: shared as think of it means that the same code can be used by multiple threads of execution simultaneously.
- a service usually has: libSomthing.(a|lib|so|dll) + libmqmsgque.(a|lib|so|dll) + main.c + context.c + ... *.c
- sharing code means using dynamic linking (so|dll) rather than static linking (a|lib)
- sharing a running service is available as:
- only one client connect to one server
- every client has one parent-context but multiple child-context
- an advanced event-queue is available to serve multiple requests at the same time
9.a (client-server) One interpretation is that multiple clients connect to the same server.
The server might have a common resource (e.g. a database) that the clients want to share. How does one go ahead with that?
9.b Does the above mean that the communication between e.g. AppService and AppServer is something that libmqmsgque cannot handle, because that communication can be many clients to one server?
- the connection between the AppServiceX and the AppServer is just an traditional software engineering project. libmqmsgque is providing a solution to connect the client to the AppService not to the AppServer because the AppServer is APP specific and usually hidden to external programmers.
- the difference between OLD and NEW is that:
9.c what is the difference between APP - API and libmqmsgque - API?
- APP - API: handle the communication between AppServiceX and AppServer using the APP-Protocol
- every application-API providing a method like init (App-Initialization), login (App-VerifyUser), send something (App-SetEntry,...) and read something (App-GetEntry,...))
- APP-API: is part of the APP distribution of application-server provider and is often a closed source library
- libmqmsgque-API: handle the communication between ClientX and AppServiceX using the libmqmsgque-Protocol
9.d does this mean I always have to provide an independent process like AppServiceX to use an already available application?
10. Are services supposed to be re-entrant or serially reusable
- INTRO: MqContextC LINK API
- DEFINITION: Re-entrant means that the same body of code can be used by multiple threads of control at once, provided that each thread supplies its own data segment for the code to use.
- Every Context (parent-context and child-context) has its own services and data.
- DEFINITION: Serially reusable means that the same body of code can be used by multiple threads of control, provided that execution of the code is allowed to complete before another thread can use the shared body of code.
- The body of a service-function get the according context as first parameter of type MqContextS. This parameter is used to read package data (example: MqReadBUF) and to answer the service-request (example: MqSendBUF)
10.a Can an already started server be reused for new requests?
11. Do services have to be locked so that they can complete processing complete requests
- DEFINITION: In this case locked means prevented from execution.
- Every parent-context is a process or a thread.
- Every child-context of this parent-context is part of this single process too.
- One single process can handle only one request per time but is able to listen on other incoming requests if the original request is waiting on an external resource. This waiting is done in the MqSendEND_AND_WAIT and the MqProcessEvent function.
- libmqmsgque was designed to act as an event based scheduler doing io in a parallel manner.
- Example: multiple parent-context (CTX1, CTX2, CTX3) : using 'Tcl' code
# SLEP ... service doing a 'sleep' system-call on the server -> blocking the server
# client is doing the following 3 jobs
after 100 "$CTX1 SendEND_AND_WAIT SLEP 10"; # test-case: service sleeps 10 seconds
after 100 "$CTX2 SendEND_AND_WAIT SLEP 5"; # test-case: service sleeps 5 seconds
after 100 "$CTX3 SendEND_AND_WAIT SLEP 8"; # test-case: service sleeps 8 seconds
# wait on finish all 3 jobs
the example needs 10 seconds to finish and not 23
- Example: single parent-context (CTX) : using 'Tcl' code
# SLEP ... service doing a 'sleep' system-call on the server -> blocking the server
# client is doing the following 3 jobs
after 100 "$CTX SendEND_AND_WAIT SLEP 10"; # test-case: service sleeps 10 seconds
after 100 "$CTX SendEND_AND_WAIT SLEP 5"; # test-case: service sleeps 5 seconds
after 100 "$CTX SendEND_AND_WAIT SLEP 8"; # test-case: service sleeps 8 seconds
# wait on finish all 3 jobs
the example needs 23 seconds to finish and not 10
11.a How does a 'service' detect different 'context' requests''
- INTRO: MqContextC LINK API
- The client-parent-context is linked with the server-parent-context.
- The data-package-format has an data-entry for the context in duty (HdrS::ctxId).
12. How do they signal success or failure
- INTRO: MqContextC LINK API
- Every transaction submit a return-code
MQ_RETURN_OK
or MQ_RETURN_ERROR
to the caller.
- The return-code is part of the package header (HdrS::code) and is processed in the MqSendEND_AND_WAIT function.
- The function MqSendEND_AND_WAIT will return an error of type MkErrorE if the return-code is
MQ_RETURN_ERROR
.