What is a filter and how is the filter used for.
The filter mode is related to a special usage of the theLink software called a command pipeline. To define a filter create a server with:
Every filter has two context one belongs to the left command and one belongs to the right command:
<-- left cmd --> <------- filter -------> <-- right cmd --> <-- left --><- right --> <- server -><- client -> <- master -><- slave --> <-context1-><-context2-> ... command1 @ filter @ command2 ...
The left context is created on application startup and the right context is created as slave of the left context.
client @ filter @ server
<------------ host-1 -------------> <-- network --> <---------- host-2 -----------> <---- client arguments ----> <--- server arguments --> <-- filter arg. --> <--- options ---> client @ filter @ --tcp --port 7777 ........... server --tcp --port 7777 --fork
A bi-directional filter allow a data-flow in both directions and is used in a classical client/server application:
client ... <--> ... server
as a feature enhancement like a protocol-tunnelling:
client @ mq2tunnel ... <--> ... tunnel2mq @ server
or to convert the protocol into an other protocol:
client @ mq2soap ... <--> ... soap-server
To define a bi-directional filter a couple of commands provide support:
aguard
)aguard
), or to save the body in a persistent storage for later use MqDumpImport.A one-directional filter is a special form of a bi-directional filter and allow a data-flow from the left to the right.
This filter is well known from the unix shell to link different commands together:
command1 | command2 | command3
A theLink command pipeline is created with the special character "@" instead of "|" :
msgcmd1 @ msgcmd2 @ msgcmd3
To define a theLink filter... create a service handle with MqServiceCreate or MqServiceProxy ... using the token +FTR and +EOF
token | description |
---|---|
+FTR | required to act on filter data rows. Every filter input data is a list of filter data rows and every row is a list of filter data columns. Every row is send to the following filter-command as +FTR service request |
+EOF | required to act on End-Of-Filter data and is called after all +FTR data was send. Sometimes the filter data can not be served as +FTR data (example: sorting of the input rows need to read all rows before the data can be send to the next filter command) and the+EOF token is used to continue send +FTR data rows. |
and send every data item with MqSendEND_AND_WAIT .
The following code is based on the JAVA example example/java/Filter3.java
:
create the class, implement the the server interface:
define the ServerSetup function required by the interface
get the filter-context to implement the proxy later
define a Generig-Handler (+ALL
) for the current context
and the filter context
define the main function, the factory and create the initial context:
process the command-line arguments and start the initial link:
finally start the event-loop and wait for an incoming service-request:
on exit delete the context and finish the process:
The following code is based on the JAVA example example/java/Filter1.java
:
create a class:
define a service-handler for the +FTR token:
define a service-handler for the +EOF token: The EOF is the end-of-data and is used to finally send out all the collected items at once. This is necessary when filtering the data as a whole, not by row.
get the filter target (slave 0)
loop over data and send ever item
finally send the EOF to signal \€ end-of-data
at the main create initial factory and context:
configure as server:
create the link:
and setup the both filter-callback:
finally start the event.loop and wait for data:
on error catch the message
and on exit cleanup the application
libmqmsgque was designed to act as a glue between different applications. For an overview about the basic concepts we are using the good old shell and using libmqmsgque to extend the usability of the well known pipe '|' syntax.
A shell command-line is a collection of one or more commands linked together using the '|' symbol:
command1 | command2 | command3
command1
, command2
and command3
are started by the shell and the stdout of command1 is the stdin of command2
and the stdout of command2
is the stdin of command3
. The data send through the pipeline are strings and every command in the pipeline have to parse the string output of the previous command to extract the information's needed.
libmqmsgque is adding an additional link character '@' to the shell and the example from above looks like this:
alfacmd1 @ alfacmd2 @ alfacmd3
Only alfacmd1
is started by the shell and gets '@ alfacmd2 @ alfacmd3
' as command-line arguments. libmqmsgque will start the both commands alfacmd2
and alfacmd3
and setup the message-queues:
alfacmd2
receiving the output from alfacmd1
and alfacmd3
receiving the output from alfacmd2
without re-parsing the data again.
For full integration of alfa commands into the shell syntax 2 additional interfaces are necessary
To connect a shell with an alfa command the special alfa command split is used:
shellcmd | atool split @ ...
The split command expect input data from stdin and is sending output data as package to an alfa command. For every input data string an output package is created by splitting the input string into output objects using the the delimiter -d.
To connect an alfa with a shell command the special alfa command join is used.
If the libmqmsgque object was created by atool :
... @ join | shellcmd
or if the libmqmsgque object was not created by atool :
... @ atool join | shellcmd
The join tool expect data from a msgque client as input and create for every input package an stdout output string by joining the objects of the input package together using the delimiter -d .
By default libmqmsgque is using unix-domain sockets (UDS) for communication but inet (TCP) sockets can be used as well. The data-flow is the same as above except that two hosts are involved using libmqmsgque over tcp sockets for connection. The tcp connection is buildup between alfacmd1
and alfacmd2
.
return: hello:world
The following tcl code total.tcl
does 2 things:
package require tclmsgque::MqMsgque set total 0 array set exchange { euro 1.3541 pound 1.9896 dollar 1 } proc FTR {ctx} { set ftr [$ctx SlaveGetFilter] foreach {position amount currency} [$ctx ReadLIST] break set amount [expr {$amount * $::exchange($currency)}] set currency dollar set ::total [expr {$::total + $amount}] $ftr Send "W" "+FTR:CD" $position $amount $ctx SendRETURN } proc EOF {ctx} { set ftr [$ctx SlaveGetFilter] $ftr Send "W" "+FTR:CD" total $::total $ftr Send "W" "+EOF" $ctx SendRETURN } tclmsgque::MqMsgque Main { tclmsgque::MqMsgque::MqContextC create srv srv ConfigSetIsServer yes srv ConfigSetName total try { srv LinkCreate {*}$argv srv ServiceCreate "+FTR" FTR srv ServiceCreate "+EOF" EOF srv ProcessEvent MQ_WAIT_FOREVER } on error {} { srv ErrorCatch } finally { srv Exit } }
using the following command pipeline:
echo -e "nobody 10 euro\nmanager 1000 dollar\nworker 100 pound" | \ atool split -d " " @ sort -1 D @ tclsh total.tcl @ \ atool join -d " : " -0 "%-8s" -1 "%5.2f$"
to create the following result:
nobody : 13.54$ worker : 198.96$ manager : 1000.00$ total : 1212.50$