Apache camel using seda - asynchronous

I want to have a behavior like this:
Camel reads a file from a directory, splits it into chunks (using streaming), sends each chunk to a seda queue for concurrent processing, and after the processing is done, a report generator is invoked.
This is my camel route:
from("file://c:/mydir?move=.done")
.to("bean:firstBean")
.split(ExpressionBuilder.beanExpression("splitterBean", "split"))
.streaming()
.to("seda:processIt")
.end()
.to("bean:reportGenerator");
from("seda:processIt")
.to("bean:firstProcessingBean")
.to("bean:secondProcessingBean");
When I run this, the reportGenerator bean is run concurrently with the seda processing.
How to make it run once after the whole seda processing is done?

The splitter has built-in parallel so you can do this easier as follows:
from("file://c:/mydir?move=.done")
.to("bean:firstBean")
.split(ExpressionBuilder.beanExpression("splitterBean", "split"))
.streaming().parallelProcessing()
.to("bean:firstProcessingBean")
.to("bean:secondProcessingBean");
.end()
.to("bean:reportGenerator");
You can see more details about the parallel option at the Camel splitter page: http://camel.apache.org/splitter

I think you can use the delayer pattern of Camel on the second route to achieve the purpose.
delay(long) in which the argument indicates time in milliseconds. You can read more abuout this pattern here
For eg; from("seda:processIt").delay(2000)
.to("bean:firstProcessingBean"); //delays this route by 2 seconds
I'd suggest the usage of startupOrder to configure the startup of route though.
The official documentation provides good details on the topic. Kindly read it here
Point to note - " The routes with the lowest startupOrder is started first. All startupOrder defined must be unique among all routes in your CamelContext."
So, I'd suggest something like this -
from("endpoint1").startupOrder(1)
.to("endpoint2");
from("endpoint2").startupOrder(2)
.to("endpoint3");
Hope that helps..
PS : I'm new to Apache Camel and to stackoverflow as well. Kindly pardon any mistake that might've occured.

Related

LocustIO: How to do batch request

I started to use LocustIO for load testing a 3rd party API which provides a way to do batch requests (http://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_BatchRequests).
How can this be done using LocustIO?
I tried with the following:
def batch(self):
response = self.client.request(method="POST", url="/$batch", auth=("ABC", "DEF"), headers={"ContentType": "multipart/mixed; boundary=batch_36522ad7-fc75-4b56-8c71-56071383e77b"}, data="Content-Type: application/http\nContent-Transfer-Encoding: binary\n\nGET putyoururlhere HTTP/1.1\nAccept: application/json\n\n\n")
Auth is something I need to have authentication to the API, but that's not the point of the question and "putyoururlhere" should be replaced with the actual url. Either way, it gives errors when executing the test, so I must be doing something wrong.
People with experience how to do this?
Kind regards!
The data parameter should be your POST body (only), you cant put additional headers in it the way you did. You probably just want to add them as additional entries in the dict you pass as headers
Se the documentation for python requests library for more details. https://requests.readthedocs.io/en/master/

What does "rerun in upstream in pipeline"?

I am defining a pipeline in data factory, I had some errors that I correct.
The first activity is calling an usql script to do some aggregation, I changed the script plenty of time but the error is still:
[{"errorId":"E_CSC_USER_SYNTAXERROR","severity":"Error","component":"CSC","source":"USER","message":"syntax
error. Final statement did not end with a semicolon","details":"at
token 'usql', line 4\r\nnear the ###:\r\n**************\r\nCLARE
#lineitemsfile string =
\"/datalakerepo/input/2016/01/01lineitems.txt\";\nDECLARE #ordersfile
string = \"/datalakerepo/input/2016/01/01orders.txt\";\nsales.usql ###
\n","description":"Invalid syntax found in the
script.","resolution":"Correct the script syntax, using expected
token(s) as a
guide.","helpLink":"","filePath":"","lineNumber":4,"startOffset":228,"endOffset":232}].
seem like not all usql script is read from the data factory, so I though that may be the "rerun in upstream in pipeline" have something to do with this, like clear cache from previous script.
Anyone knows what "rerun in upstream in pipeline" does?
Many thanks!
"Rerun with upstream in pipeline" basically means "recalculate with all dependencies". For example, if one has pipeline1 -> dataset1 -> pipeline2 and tries to rerun pipeline2 with dependecies, then pipeline1 and pipeline2 will be both executed. I believe it works same with several chained activities within single pipeline.

Asterisk catch a Incoming call and transfer it to a specific exten

I have been building a Window Form desktop application using C# that interfaces with Asterisk using Asterisk.NET.
My first problem is catch a Incoming call and transfer it to specific exten.
The first my idea is using OriginateAction, when a call come, I use Dial event and catch it and use OriginateAction to call to a specific exten.
RedirectAction originateAction = new RedirectAction();
originateAction.Channel = e.Channel;
originateAction.Context = "default";
originateAction.Exten = "203";
originateAction.Priority = 1;
ManagerResponse originateResponse = manager.SendAction(originateAction);
Console.WriteLine(originateResponse);
But it not work like my wish.
The second my idea is using RedirectAction:
RedirectAction originateAction = new RedirectAction();
originateAction.Channel = e.Channel;
originateAction.Context = "default";
originateAction.Exten = "203";
originateAction.Priority = 1;
ManagerResponse originateResponse = manager.SendAction(originateAction);
Console.WriteLine(originateResponse);
And it not work.
I have find on many websites but the documents is very little.
How can I solve this issue?
Thanks!
I would suggest using some kind of dynamic dialplan instead of "catching" calls reactively. Why not use an AGI script?
Essentially, your application tells a database or other central system what to do when calls matching certain criteria come in. Then Asterisk runs the script you setup when calls reach a certain context (such as all incoming calls), and then the script routes the call dynamically based on the inputs given by your application.
Since you seem to like .NET, here's a .NET AGI project to help you get started: AsterNET. It looks like the library you mentioned, Asterisk.NET, is also capable of Fast CGI (what AGI uses), but the last release was in 2009, whereas AsterNet is active as recently as 3 months ago.
I personally use phpAGI to do all kinds of neat ACD and call routing stuff in our call center.
For more info on AGI, see the official docs.
Edit:
I should probably also explain some basic call flow terminology (from the docs):
Originate: Generates an outgoing call to a Extension/Context/Priority or Application/Data. Example: User clicks a button, Originate a call to their desk phone, when they answer that call, it executes dialplan, or a dialplan application.
Redirect: Redirect (transfer) a call. Example: Agent and Customer are talking, but Manager wants to take over the call. Use Redirect to "take" the call from Agent and ring the Manager.
Dial: (in dialplan only, not AMI) Dial the technology/channel specified. Note that you can only Originate from your .NET application, not Dial.
Can you show your event handler code? It looks like that library would say something like manager.NewChannel += new ManagerEventHandler(new_channel);

Twain always returns TWRC_NOTDSEVENT

I use twain 2.3 (TWAINDSM.DLL) in my application with HP Scanjet 200 TWAIN Protocol 1.9.
My TWAIIN calls are:
OpenDSM: DG_CONTROL, DAT_PARENT, MSG_OPENDSM
OpenDS: DG_CONTROL, DAT_IDENTITY, MSG_OPENDS
EnableDS: DG_CONTROL, DAT_USERINTERFACE, MSG_ENABLEDS
ProcessDeviceEvent: DG_CONTROL, DAT_EVENT, MSG_PROCESSEVENT
and as a result of the last call I allways get TWRC_NOTDSEVENT instead of TWRC_DSEVENT.
Could please someone help with this?
Once you use DG_CONTROL / DAT_EVENT / MSG_PROCESSEVENT, all messages from the applications message loop must be sent to the data source for processing. Receiving TWRC_NOTDSEVENT means the forwarded message isn't for the source so the application should process it as normal.
Keep forwarding all messages to the source until you receive MSG_XFERREADY which means there is data to transfer. Once the transfer is finished and you have sent MSG_DISABLEDS that's when you can stop forwarding messages to the source.
Twain is a standard, and when many company implement that standard, not all of them do the same way. Along the way to support Twain, we will learn and adjust the code to support all the different implementations.
I experienced this situation before, and this is my workaround:
Instead of placing (rc == TWRC_DSEVENT) at the beginning of code (will skip the following MSG_XFERREADY processing afterward) you can move the comparison to the end after MSG_XFERREADY processing, so that MSG_XFERREADY is always checked and processed.
(rc == TWRC_DSEVENT) is only to determine if we should forward the window message or not.
I don't know your specific situation. I ran into a similar issue because I called OpenDSM with a HWND/wId which is from another process. You should call OpenDSM with the HWND of
the active window/dialog which is owned by current process.

Count messages in an msmq sub-queue

Is there any way (C# or native) of counting the messages in a message queue (sub queue).
Using a queue name of "DIRECT=OS:slc11555001\private$\file-queue;retry"
I want to know how many messages are in the sub queue. At the moment I can see using the management console that there are in fact messages in that queue. If the OS can do it, so should I.
MQMgmtGetInfo returns 0xc00e0020 (which is not a documented error code).
I am therefore confused.
I am using code from here:
http://functionalflow.co.uk/blog/2008/08/27/counting-the-number-of-messages-in-a-message-queue-in/
The error is as follows (from http://support.microsoft.com/kb/304287):
MQ_ERROR_UNSUPPORTED_FORMATNAME_OPERATION (0xC00E0020).
MQMgmtGetInfo won't understand the subqueue format name.
The retry subqueue only really exists as a logical division of private$\file-queue queue.
You can call a count on the file-queue but not subqueues within it.
Cheers
John
From the MSDN page on subqueues:
Subqueues are created implicitly, so only the following APIs can be
used with subqueues: MQOpenQueue, MQCloseQueue, MQCreateCursor,
MQReceiveMessage, MQReceiveMessageByLookupId, MQHandleToFormatName,
MQMoveMessage, and MQPathNameToFormatName. Calling any of the other
Message Queuing APIs returns an error

Resources