MqContextC - setup and manage a storage used to persist data-packages …
More...
|
MQ_EXTERN enum MkErrorE | MqStorageErrCnt_RT (MK_RT mkrt, MQ_CTX const ctx, MQ_LTR transLId, MK_I32 *cnt_out) |
| increment and return the database row-error-count for the row defined with transLId
|
|
MQ_EXTERN enum MkErrorE | MqStorageOpen_RT (MK_RT mkrt, MQ_CTX const ctx, MK_STRN const storageFile) |
| switch to a file-based-transaction-database …
|
|
MQ_EXTERN enum MkErrorE | MqStorageClose_RT (MK_RT mkrt, MQ_CTX const ctx) |
| close the storage. …
|
|
MQ_EXTERN enum MkErrorE | MqStorageExport_RT (MK_RT mkrt, MQ_CTX const ctx, MQ_LTR *ltid_out) |
| export the read-data-package into the STORAGE …
|
|
MQ_EXTERN enum MkErrorE | MqStorageImport_RT (MK_RT mkrt, MQ_CTX const ctx, MQ_LTR *transLIdP_inout) |
| import the storage-package into the read-data-package …
|
|
MQ_EXTERN enum MkErrorE | MqStorageIncrRef_RT (MK_RT mkrt, MQ_CTX const ctx, MQ_LTR transLId) |
| increase the internal refCount lock of the database-entry
|
|
MQ_EXTERN enum MkErrorE | MqStorageDecrRef_RT (MK_RT mkrt, MQ_CTX const ctx, MQ_LTR transLId) |
| decrease the internal refCount lock of the database-entry
|
|
MQ_EXTERN enum MkErrorE | MqStorageDelete_RT (MK_RT mkrt, MQ_CTX const ctx, MQ_LTR *transLIdP_inout) |
| delete the storage-row identified by the transLIdP_inout …
|
|
MQ_EXTERN enum MkErrorE | MqStorageCount_RT (MK_RT mkrt, MQ_CTX const ctx, MQ_LTR *cnt_out) |
| count the number of storage-rows …
|
|
MQ_EXTERN enum MkErrorE | MqStorageResolve_RT (MK_RT mkrt, MQ_CTX const ctx, MQ_CTX *otherCtxP_out, MQ_LTR *otherLIdP_out) |
| extract the context->link.protect.rmtTransLId entry from the database
|
|
MqContextC - setup and manage a storage used to persist data-packages …
The storage is divided into: INTERNAL and EXTERNAL storage. Only the read-data-package can be stored or dumped into the storage.
The read-data-package is saved into the storage using:
The read-data-package is restored from the storage using:
Some important facts of the storage-feature:
- The internal-storage depends on SQLite.
- The package-storage can be used to save all kind of package-data.
- The longterm-transaction-package-data is allways saved into storage.
- By default only an in-memory storage is in use, but this can be changed with MqStorageOpen or Storage
- An entire service-call can be saved with MqServiceStorage
The following internal storages are supported:
- default
- The default-storage is set with the configuration parameter --storage fileName and defaults to "#memdb#". If a package have to be saved into the storage and the storage is not open the default-storage id used. The open will always be performed. If an explicit storage is required the default can be changed or a storage can explicit be opened with MqStorageOpen. Keep in mind that the default-storage is a per-context configuration but only one storage per process or thread is currently supported.
- memdb
- This is the default storage and can be set explicitly with MqStorageOpen using the "#memdb#" parameter.
- tmpdb
- This storage is like an in-memory-storage but export data to the TEMPORARY filesystem if the application run out of memory. This storage can explicit be set with MqStorageOpen with the parameter "#tmpdb#".
- filedb
- This storage always work on files. Only this storage is persistent and can explicit be set with MqStorageOpen with the storageFile parameter.
Performance analyse:
- The performance is tested with:
Nhi1Exec perfclient.c --all --storage VALUE @ perfserver.c
.
- The parameter VALUE is set to #memdb#, #tmpdb# or a filedb file.
- If –storage is not set the filedb database is used together with an internal file-name.
database | performance | host crash | application crash | info |
memdb | 30.000 | data lost | data lost | non persistent |
tempdb | < 30.000 | data lost | data lost | uses memory and/or temporary file |
filedb (mem) | 10.000 | data lost | data safe | in memory filesystem |
filedb (disc) | 50 | data safe | data safe | disc-speed is the key factor |
◆ MqStorageClose
#define MqStorageClose |
( |
| ... | ) |
|
Value:
MQ_EXTERN enum MkErrorE MqStorageClose_RT(MK_RT mkrt, MQ_CTX const ctx)
close the storage. …
Definition at line 808 of file msgque_overload_mq.h.
◆ MqStorageClose_C
#define MqStorageClose_C |
( |
| ... | ) |
|
◆ MqStorageClose_E
#define MqStorageClose_E |
( |
| ... | ) |
|
◆ MqStorageClose_NULL
#define MqStorageClose_NULL |
( |
| ... | ) |
|
◆ MqStorageCount
#define MqStorageCount |
( |
| ... | ) |
|
Value:
MQ_EXTERN enum MkErrorE MqStorageCount_RT(MK_RT mkrt, MQ_CTX const ctx, MQ_LTR *cnt_out)
count the number of storage-rows …
Definition at line 831 of file msgque_overload_mq.h.
◆ MqStorageCount_C
#define MqStorageCount_C |
( |
| ... | ) |
|
◆ MqStorageCount_E
#define MqStorageCount_E |
( |
| ... | ) |
|
◆ MqStorageCount_e
#define MqStorageCount_e |
( |
| ... | ) |
|
Value:
#define MK_EMBEDDED(type, call,...)
int MQ_LTR
32bit longterm transaction token because PHP-32 does NOT support long long
Definition at line 834 of file msgque_overload_mq.h.
◆ MqStorageCount_NULL
#define MqStorageCount_NULL |
( |
| ... | ) |
|
◆ MqStorageDecrRef
#define MqStorageDecrRef |
( |
| ... | ) |
|
Value:
MQ_EXTERN enum MkErrorE MqStorageDecrRef_RT(MK_RT mkrt, MQ_CTX const ctx, MQ_LTR transLId)
decrease the internal refCount lock of the database-entry
Definition at line 812 of file msgque_overload_mq.h.
◆ MqStorageDecrRef_C
#define MqStorageDecrRef_C |
( |
| ... | ) |
|
◆ MqStorageDecrRef_E
#define MqStorageDecrRef_E |
( |
| ... | ) |
|
◆ MqStorageDecrRef_NULL
#define MqStorageDecrRef_NULL |
( |
| ... | ) |
|
◆ MqStorageDelete
#define MqStorageDelete |
( |
| ... | ) |
|
Value:
MQ_EXTERN enum MkErrorE MqStorageDelete_RT(MK_RT mkrt, MQ_CTX const ctx, MQ_LTR *transLIdP_inout)
delete the storage-row identified by the transLIdP_inout …
Definition at line 794 of file msgque_overload_mq.h.
◆ MqStorageDelete_1
#define MqStorageDelete_1 |
( |
| ctx | ) |
|
◆ MqStorageDelete_C
#define MqStorageDelete_C |
( |
| ... | ) |
|
◆ MqStorageDelete_E
#define MqStorageDelete_E |
( |
| ... | ) |
|
◆ MqStorageDelete_e
#define MqStorageDelete_e |
( |
| ... | ) |
|
◆ MqStorageDelete_NULL
#define MqStorageDelete_NULL |
( |
| ... | ) |
|
◆ MqStorageErrCnt
#define MqStorageErrCnt |
( |
| ... | ) |
|
Value:
MQ_EXTERN enum MkErrorE MqStorageErrCnt_RT(MK_RT mkrt, MQ_CTX const ctx, MQ_LTR transLId, MK_I32 *cnt_out)
increment and return the database row-error-count for the row defined with transLId
Definition at line 836 of file msgque_overload_mq.h.
◆ MqStorageErrCnt_C
#define MqStorageErrCnt_C |
( |
| ... | ) |
|
◆ MqStorageErrCnt_E
#define MqStorageErrCnt_E |
( |
| ... | ) |
|
◆ MqStorageErrCnt_e
#define MqStorageErrCnt_e |
( |
| ... | ) |
|
◆ MqStorageErrCnt_NULL
#define MqStorageErrCnt_NULL |
( |
| ... | ) |
|
◆ MqStorageExport
#define MqStorageExport |
( |
| ... | ) |
|
Value:
MQ_EXTERN enum MkErrorE MqStorageExport_RT(MK_RT mkrt, MQ_CTX const ctx, MQ_LTR *ltid_out)
export the read-data-package into the STORAGE …
Definition at line 841 of file msgque_overload_mq.h.
◆ MqStorageExport_C
#define MqStorageExport_C |
( |
| ... | ) |
|
◆ MqStorageExport_E
#define MqStorageExport_E |
( |
| ... | ) |
|
◆ MqStorageExport_e
#define MqStorageExport_e |
( |
| ... | ) |
|
◆ MqStorageExport_NULL
#define MqStorageExport_NULL |
( |
| ... | ) |
|
◆ MqStorageImport
#define MqStorageImport |
( |
| ... | ) |
|
Value:
MQ_EXTERN enum MkErrorE MqStorageImport_RT(MK_RT mkrt, MQ_CTX const ctx, MQ_LTR *transLIdP_inout)
import the storage-package into the read-data-package …
Definition at line 800 of file msgque_overload_mq.h.
◆ MqStorageImport_1
#define MqStorageImport_1 |
( |
| ctx | ) |
|
◆ MqStorageImport_C
#define MqStorageImport_C |
( |
| ... | ) |
|
◆ MqStorageImport_E
#define MqStorageImport_E |
( |
| ... | ) |
|
◆ MqStorageImport_e
#define MqStorageImport_e |
( |
| ... | ) |
|
◆ MqStorageImport_NULL
#define MqStorageImport_NULL |
( |
| ... | ) |
|
◆ MqStorageIncrRef
#define MqStorageIncrRef |
( |
| ... | ) |
|
Value:
MQ_EXTERN enum MkErrorE MqStorageIncrRef_RT(MK_RT mkrt, MQ_CTX const ctx, MQ_LTR transLId)
increase the internal refCount lock of the database-entry
Definition at line 816 of file msgque_overload_mq.h.
◆ MqStorageIncrRef_C
#define MqStorageIncrRef_C |
( |
| ... | ) |
|
◆ MqStorageIncrRef_E
#define MqStorageIncrRef_E |
( |
| ... | ) |
|
◆ MqStorageIncrRef_NULL
#define MqStorageIncrRef_NULL |
( |
| ... | ) |
|
◆ MqStorageLog
#define MqStorageLog |
( |
| ... | ) |
|
Value:
MQ_EXTERN enum MkErrorE MqStorageLog_RT(MK_RT mkrt, MQ_CTX const ctx, MK_STRN const callfunc)
log the storage the status of the open transactions
Definition at line 820 of file msgque_overload_mq.h.
◆ MqStorageLog_1 [1/2]
#define MqStorageLog_1 |
( |
| ctx | ) |
|
◆ MqStorageLog_1 [2/2]
#define MqStorageLog_1 |
( |
| ctx | ) |
|
◆ MqStorageLog_C
#define MqStorageLog_C |
( |
| ... | ) |
|
◆ MqStorageLog_E
#define MqStorageLog_E |
( |
| ... | ) |
|
◆ MqStorageLog_NULL
#define MqStorageLog_NULL |
( |
| ... | ) |
|
◆ MqStorageOpen
#define MqStorageOpen |
( |
| ... | ) |
|
Value:
MQ_EXTERN enum MkErrorE MqStorageOpen_RT(MK_RT mkrt, MQ_CTX const ctx, MK_STRN const storageFile)
switch to a file-based-transaction-database …
Definition at line 825 of file msgque_overload_mq.h.
◆ MqStorageOpen_C
#define MqStorageOpen_C |
( |
| ... | ) |
|
◆ MqStorageOpen_E
#define MqStorageOpen_E |
( |
| ... | ) |
|
◆ MqStorageOpen_NULL
#define MqStorageOpen_NULL |
( |
| ... | ) |
|
◆ MqStorageResolve
#define MqStorageResolve |
( |
| ... | ) |
|
Value:
MQ_EXTERN enum MkErrorE MqStorageResolve_RT(MK_RT mkrt, MQ_CTX const ctx, MQ_CTX *otherCtxP_out, MQ_LTR *otherLIdP_out)
extract the context->link.protect.rmtTransLId entry from the database
Definition at line 846 of file msgque_overload_mq.h.
◆ MqStorageResolve_C
#define MqStorageResolve_C |
( |
| ... | ) |
|
◆ MqStorageResolve_E
#define MqStorageResolve_E |
( |
| ... | ) |
|
◆ MqStorageResolve_NULL
#define MqStorageResolve_NULL |
( |
| ... | ) |
|
◆ MqStorageClose_RT()
close the storage. …
the next storage request will open the storage again with the location from Storage
- Parameters
-
- Exceptions
-
◆ MqStorageCount_RT()
count the number of storage-rows …
- Parameters
-
[in] | mkrt | the MkRuntimeS instance to work on - the runtime argument, used by MK_RT_CALL (C-only) |
[in] | ctx | the MqContextS instance to work on |
[out] | cnt_out | number of rows, OLL if nothing is available |
- Exceptions
-
◆ MqStorageDecrRef_RT()
◆ MqStorageDelete_RT()
delete the storage-row identified by the transLIdP_inout …
The transLIdP_inout is the value returned by a previous MqStorageImport. If transLIdP_inout is None
than OLL
is used. MqStorageImport and MqStorageDelete work together link an transaction and every successfull transcation have to delete the transLIdP_inout out of the storage. After sucessfull delete the transLIdP_inout is set to 0LL
, on error the initial value is unchanged.
- Parameters
-
[in] | mkrt | the MkRuntimeS instance to work on - the runtime argument, used by MK_RT_CALL (C-only) |
[in] | ctx | the MqContextS instance to work on |
[in,out] | transLIdP_inout | pointer to storage-id to delete, will be 0LL on success |
- Exceptions
-
◆ MqStorageErrCnt_RT()
increment and return the database row-error-count for the row defined with transLId
The storage is typical used in the following workflow:
<----- AT CALLING CLIENT ----->|<----- AT RECEIVING SERVER ----->
|
*setup transaction |
make service-request --> *start service-handler
| save request into storage
wait for confirmation <-- confirm data receiving
| *stop service-handler
==================== enter event loop =====================
| *start event-handler
| load reqest from storage
| process data
*start transaction <-- send result-data
process result-data | *stop event-handler
*stop transaction |
Example Filter6 - store incoming packages into storage
- Parameters
-
[in] | mkrt | the MkRuntimeS instance to work on - the runtime argument, used by MK_RT_CALL (C-only) |
[in] | ctx | the MqContextS instance to work on |
[in] | transLId | storage-id … return from MqStorageImport |
[out] | cnt_out | the new value or -1 on error |
- Exceptions
-
◆ MqStorageExport_RT()
export the read-data-package into the STORAGE …
The export include all data required to setup a read-data-package later again. After import with MqStorageImport the read-data-package and the environment will be set as it was on MqStorageExport
- Parameters
-
[in] | mkrt | the MkRuntimeS instance to work on - the runtime argument, used by MK_RT_CALL (C-only) |
[in] | ctx | the MqContextS instance to work on |
[out] | ltid_out | if ltid_out != NULL return the storage-id |
- Exceptions
-
- See also
- MqDumpExport, MqStorageImport
◆ MqStorageImport_RT()
import the storage-package into the read-data-package …
Select the read-data-package from the database using the transLIdP. Only a package with refCount = 0
will be selected.
After selection the refCount is incremented and the next call to MqStorageImport using the same transLId will fail.
MqStorageImport is used together with MqStorageDelete or MqStorageDecrRef to delete the selected package from database or to undo the select.
- Parameters
-
[in] | mkrt | the MkRuntimeS instance to work on - the runtime argument, used by MK_RT_CALL (C-only) |
[in] | ctx | the MqContextS instance to work on |
[in,out] | transLIdP_inout | storage-id or None or 0 … return from MqStorageExport
*transLIdP > 0 | return the read-data-package identified with the storage-id. |
*transLIdP == 0 | return the top-most (FIFO) read-data-package and set the transLIdP to the storage-id. |
*transLIdP < 0 | return an error |
transLIdP == NULL | like transLIdP == 0 |
*transLIdP invalid | like transLIdP < 0 |
|
- Exceptions
-
- Attention
- Read MqProcessEvent for the dependency with the event-loop.
-
There is no guarantee that a package will be read FIFO.
In principle, the first free package (refCount=0
) with the smallest transLId
is read.
However, if the package hangs in subsequent processing and the refCount
is not reset or the package is not deleted, then the package remains locked and a subsequent call to MqStorageImport will read the next free package.
- This approach protects a server or router from blocking after an unpredictable failure.
- See also
- MqDumpExport, MqStorageExport, MqStorageDelete, MqStorageDecrRef, MqStorageIncrRef
◆ MqStorageIncrRef_RT()
◆ MqStorageLog_RT()
log the storage the status of the open transactions
◆ MqStorageOpen_RT()
switch to a file-based-transaction-database …
the value can also be set using the --storage
commandline option.
- Parameters
-
[in] | mkrt | the MkRuntimeS instance to work on - the runtime argument, used by MK_RT_CALL (C-only) |
[in] | ctx | the MqContextS instance to work on |
[in] | storageFile | the file used to create the transaction-database. Allowed values are:
"#memdb#" for a in-memory-database (default)
"#tmpdb#" for a temporary-database-file
filename for a persistent-database-file
|
- Exceptions
-
◆ MqStorageResolve_RT()
extract the context->link.protect.rmtTransLId
entry from the database
- Parameters
-
[in] | mkrt | the MkRuntimeS instance to work on - the runtime argument, used by MK_RT_CALL (C-only) |
[in] | ctx | the MqContextS instance to work on |
[out] | otherCtxP_out | the ctx of the resolve |
[out] | otherLIdP_out | the transLId of the resolve |