I need to build a Windows Service in VB.net under Visual Studio 2003. This Windows service should read the flat file (Huge file of about a million records) from the local folder and upload it to the corresponding database table. This should be done in Rollback mode (Database transaction). While transferring data to table, the service should also be listening to additional client requests. So, if in between client requests for a cancel operation, then the service should rollback the transactions and give feedback to the client. This windows service also keeps writing continuously to two log files about the status and error records.
My client is ASPX page (A website).
Can somebody help me explain how to organize and achieve this functionality in a windows service(Processing and listening for additional client requests simultaneously. Ex. Cancellation request).
Also could you suggest me the ideal way of achieving this (like if it is best to implement it as web service or windows service or just a remote object or some other way).
Thank you all for your help in advance!
You can architect your service to spawn "worker threads" that do the heavy lifting, while it simply listens for additional requests. Because future calls are likely to have to deal with the current worker, this may work better than, say, architecting it as a web service using IIS.
The way I would set it up is: service main thread is listening on a port or pipe for a communication. When it gets a call to process data, it spawns a worker thread, giving it some "status token" (could be as simple as a reference to a boolean variable) which it will check at regular intervals to make sure it should still be running. Thread kicks off, service goes back to listening (network classes maintain a buffer of received data so calls will only fail if they "time out").
If the service receives a call to abort, it will set the token to a "cancel" value. The worker thread will read this value on its next poll and get the message, rollback the transaction and die.
This can be set up to have multiple workers processing multiple files at once, belonging to callers keyed by their IP or some unique "session" identifier you pass back and forth.
You can design your work like what FTP do. FTP use two ports, one for commands and another for data transfer.
You can consider two classes, one for command parsing and another for data transfer, each one on separate threads.
Use a communication channel (like a privileged queue) between threads. You can use Syste.Collections.Concurrent if you move to .NET 4.0 and more threading features like CancellationTokens...
WCF has advantages over web service, but comparing it to windows service needs more details of your project. In general WCF is easier to implement in compare to windows service.
Related
We have a flow that processes incoming bank statements from an external system to SAP. The process itself is rather simple
Receive bank statement via SFTP
Send to SAP via FTP
Call SAP RFC with filename as parameter
This all happens in an orchestration and on BizTalk side it's working fine.
Now, they have noticed that SAP has some issues when too many bank statements arrive at the same time. So we need to redesign the orchestration so it handles them 1-by-1.
So, my first thought was to redesign it as a Singleton orchestration to solve this issue. Does anybody have some other suggestions to fix this issue?
The messages don't need to be processed in a specific order. Just slower. :-)
I'm just a bit afraid of the possible sideffects of a Singleton.
You could consider putting the port on a dedicated host and configure resource based throttling.
If that doesn't suit, consider a Resource Dispenser pattern which is described here:
https://social.technet.microsoft.com/wiki/contents/articles/23924.biztalk-server-resource-dispenser-send-port-edition.aspx
Basically, messages are queued for a limited number of receiving service instances, and the destination send ports are set to ordered delivery, so each will have only on active instance.
I am implementing an ASP.NET application that needs to service conventional http requests but the responses require data that I need to acquire from providers that are executables that provide their data over sockets. My plan to implement was:
1) In Application_Start, start a new thread that starts a socket server
2) In Session_Start, launch the session-specific process that will ultimately connect to the socket server, and from there do a Monitor.Wait on a session-specific lock object which I've stored in Application.Contents by Session key
3) When the socket server sees a new connection, make the data available to the appropriate session Contents and do a Monitor.Pulse on the session-specific lock object
Is this technically feasible in IIS? Can this concept function as a stable system?
Before answering, please bear in mind I am not asking "is this the recommended approach", I am aware it is not and if I had the option to write this system from scratch I would do this differently. I'm also not able to change the fact that the programs communicate using sockets.
Given the constraints this approach makes sense.
Shutdown and recycling of IIS worker processes are always throny issues when it comes to keeping state in a web app. Note, that your worker process can recycle pretty much at any time for many reasons. Some of those reasons are unavoidable: Server reboot, app deployment, bug leading to a process crash. So you need to think through what happens in those cases: All sessions will be lost while the child processes still run. Suggested solution: Add the children into a Windows Job Object and configure the Job to be killed when the parent exits.
With overlapped IIS worker recycling you can have two functioning workers running at the same time. You must deal with that possibility.
Consider the possibility that the child process immediately crashes. It will never make a connection. Make sure your app doesn't hang waiting for the connection forever.
I'd like to develop a simple solution using .NET for the following problem:
We have several computers in a local network:
10 client computers that may need to execute a program that is only installed on two workstations
The two workstations that are only used to execute the defined program
A server that can be used to install a service available from all previously described computers
When a client computer needs to execute the program, he would send a request to the server, and the server would distribute the job to a workstation when available for execution, and inform the client computer when the execution has been performed.
I'm not very used to web and services development so I'm not sure if it's the best way to go, but below is a possible solution I thought about:
A web service on the server stores in queues or in a database the list of tasks with their status
The client computer calls the web service to execute a program and gets a task id. Then calls it every second with the task id to know if the execution has been performed.
The workstations that are available call the web service every second to know if there is something to execute. If yes, the server assigns the task, and the workstation calls the web service when the execution is completed.
I summarized this in the below figure:
Do you think to a simpler solution?
Have a look at signalr! You could use it as messaging framework and you would not need to poll the service from 2 different diretions. With signalR you would be able to push execution orders to the service and the service will notify the client once the execution has been processed. The workstation would be connected with signalR, too. They would not need to ask for execution orders as the webservice would be able to push execution orders to either all or a specific workstation.
I have an existing application (WPF) that monitors OPC Servers and alarms. There is a requirement for this to be accessible via a browser so that users can view the status of alarms etc remotely. I'm feeling out of my depth (I'm not a Web developer) and I just need some advice on the best technology to accomplish this.
I've written several WCF Services, but all these have done is, via a function call, crunch some data sending back a result.
This 'service' will have to be persistent and able to be interrogated by x number of clients. For example, a client will need to be able to connect, stay connected and be informed of events as an when they happen. This has been a major problem in the past when I've developed WCF services (channel faults etc) and I've learnt to only keep a connection open for as long as it's needed. Is a WCF Service the best option in this case (as opposed to a normal Window's Service)
I need to be able to 'push' information from the service to clients. So, someone navigates to a webpage, the page shows in realtime, what is happening in the service. Do I need to use timers since this could be big problem if session state cannot be maintained.
I've read about Observer Design Pattern, but can this be implemented in ASP.net and how would ASP connect (and remain connected) to a remote windows service? Again, do I have to resort to timers?
I apologise it this appears vague, but the situation boils down to the following:
A process that's continually running (somewhere), receiving connections from remote clients (desktop/web), and then keeping the clients informed as events (alarms going off etc) occur.
I want to develop a web application using ASP.NET running on IIS.
If a user submits a MAXIMA input command, the code behind will ask a custom windows service to create a new distinct temporary process executing an external assembly.
More precisely, there is only one windows service serving for all users, but each user will be associated with a distinct, temporary process running an external assembly.
The windows service contains a single socket listening on a certain port and a list of asynchronous sockets for communication. Each socket of the list will communicate with a distinct, temporary process running an external assembly which works as a client socket.
Note that: I use a process rather than an application domain because the external assembly is a batch file (not managed assembly).
My questions are:
How to call windows service from code behind?
How to associate each user with a distinct, temporary process?
How to improve scalability if there are more and more users working simultaneously?
If the Maxima input command entered by a user cause long-running process, what is the wise way to notify the user about the progress?
The following link provide you with more detail about my project: https://sourceforge.net/projects/aspmaxima/forums/forum/1190702/topic/3786806
Thank you in advance.
You should not be using codebehind in an MVC app.
Scalability while interoprating with unmanaged code is hard. The only sane way to do this is to decompose the problem.
When you launch an unmanaged app, it already has its own process.
Multiple task flows in a service called from a web app, with monitoring? You're describing Windows Server AppFabric. Host your service with AppFabric, and you won't have to write all of this yourself.
Regarding scalability, when you're dealing with unmanaged processes, you're going to have to limit the number which can start concurrently. Trial and error will be necessary to determine the optimum on specific hardware.
You can only monitor an unmanaged task's progress if that app specifically provides for it.
Launching arbitrary unmanaged code from a service is dangerous, because the launched app, by default, inherits the service's (typically raised) permissions. Consider using specific, limited credentials for the launched app instead of the default.