I am trying to use Topshelf to create a Rebus endpoint that will run as a service. How should this be set up and are there any examples?
You can take a look at the Rebus samples repository, where the integration service sample in particular shows what you're after.
As you can see in Program.cs it uses Topshelf to basically just hold on to a Windsor container, which it disposes when the application shuts down.
The Castle Windsor installer syntax causes the installers to be automatically picked up, where the RebusInstaller shows how you'd typically let Rebus inject itself into your container, and the HandlerInstaller shows how you can add handlers to the container.
It should be fairly easy to adapt the sample to use another container - just remember to dispose it when the application shuts down, thus giving Rebus a chance to finish messages currently being handled and stop its worker threads.
Related
I have a .Net core application that consists of some background tasks (hosted services) and WEB APIs (which controls and get statuses of those background tasks). Other applications (e.g. clients) communicate with this service through these WEB API endpoints. We want this service to be highly available i.e. if a service crashes then another instance should start doing the work automatically. Also, the client applications should be able to switch to the next service automatically (clients should call the APIs of the new instance, instead of the old one).
The other important requirement is that the task (computation) this service performed in the background can’t be shared between two instances. We have to make sure only one instance does this task at a given time.
What I have done up to now is, I ran two instances of the same service and use a SQL server-based distributed locking mechanism (SqlDistributedLock) to acquire a lock. If a service could acquire a lock then goes and do the operation while the other node waiting to acquire the lock. If one service crashed the next node could be able to acquire the lock. On the client-side, I used Polly based retry mechanism to switch the calling URL to the next node to find the working node.
But this design has an issue, if the node which acquired the lock loses the connectivity to the SQL server then the second service managed to acquire the lock and started doing the work while the first service is also in the middle of doing the same.
I think I need some sought of leader election (seems done it wrongly), Can anyone help me with a better solution for this kind of a problem?
This problem is not specific to .Net or any other framework. So please make your question more general so as to make it more accessible. Generally the solution to this problem lies in the domain of Enterprise Integration Patterns, so consult the references as the status quo may change.
At first sight and based on my own experience developing distributed systems, I suggest two solutions:
use a load balancer or gateway to distribute requests between your service instances.
use a shared message queue broker to put requests in and let each service instance dequeue a request for processing.
Either is fine and I can use both for my own designs.
I am working on an app which has the web component (visited via browser) and background task processing component, to which web component delegates some long running stuff.
I've just hit an issue when I refreshed my web browser only to find it loading indefinitely (first spotted in AJAX, but later in normal request).
It did not really look apparent but as soon as I shut down the background Symfony command which also utilizes EntityManager the browser get unblocked and proceeds with request.
My app uses RabbitMQ to store job requests which are publish by web component. The Symfony command uses the same "backbone" to create RabbitMQ consumer and take consume those jobs.
I tried, without any result:
Restarting Apache
Restating RabbitMQ
Purging RabbitMQ queue
Using different EntityManagers for web and command
I use OldSoundRabbitMqBundle (link) to facilitate communication between those two.
The web component gets stuck regardless of action being called (not related to RabbitMQ producer).
Has anyone stumbled upon similar issue?
This happens on dev box, I haven't got around giving it a spin on a production server, nor would I until I find out more about this.
It would seem that I misused the locking mechanism in Postgres. Indeed the task processing component is a long-running task, but given that it is Symfony command, Doctrine connection is being established as early as possible.
Now comes the tricky part: I used the LOCK TABLE statement to lock some tables away from concurrent access (EXCLUSIVE type). Without closing the connection (not entity manager), those locks are left intact, until I restart the command (every 10th task).
This was the root cause.
I am still investigating some edge-cases, but since I moved away to advisory locking, I had no more lock-ups.
when I use the cloudify(2.7) to deploy an application(e.g. an application app includes two services A and B ),I try to use the Admin.addEventListener() to add some eventListener,but it does't work !
I try to add the ProcessingUnitStatusChangedEventListener ,when I debug the code,the value of (ProcessingUnitStatusChangedEvent)event.getNewStatus() changes from SCHEDULED to INTACT,then SCHEDULED,then INTACT again,
I also try to add the ProcessingUnitInstanceLifecycleEventListener,when I debug the code,the status is intact,but the service is not available!
Is there any other listener or method to know the application(not the services) is available,or I use the listener in the wrong way?
First, the Admin API is internal - use it at your own risk. And you should not be using it the way you are - Cloudify adds a lot of logic on top of the internal Admin API.
Second, it is not exactly clear where you are executing your code from.
You can always use the rest client to get an accurate state of the application. Look at https://github.com/CloudifySource/cloudify/blob/master/rest-client/src/main/java/org/cloudifysource/restclient/RestClient.java#L388
In addition, if you are running this code in a service lifecycle event handler, the easiest way to implement this is to have your 'top' level service, the one that should be available last, write an application entry to the shared attributes store in its 'postStart' event. Everyone else can just periodically poll on this entry. The polling itself is very fast, all in-memory operations.
If you do not have a top-level service, or your logic is more complicated then that, you would need to use the Service Context API to scan each service and its instances to see if they are up. An explanation on getting service instance state is available here:
cloudify service dependsOn other service
I have a WebAPI app that needs to run some tasks every 30 minutes or so. How can I create a service object that encapsulates this processing and starts on application start. I need also to be able to inject this service into some of the controllers. I also need to some of my other Autofac services when the task runs (every 30 min.).
Maybe, you can use NCron (https://code.google.com/p/ncron/). NCron is a light-weight library for building and deploying scheduled background jobs.
Also, you can use Autofac integration to perform dependency injection for your NCron jobs (https://code.google.com/p/ncron/wiki/DependencyInjection).
I have an ASP.NET Webform which currently calls a Java WebService. The ASP.NET Webform is created/maintained inhouse, whereas the Java WS is a package solution where we only have a WS interface to the application.
The problem is, that the Java WS is sometimes slow to respond due to system load etc. and there is nothing I can do about this. So currently at the moment there is a long delay on the ASP.NET Webform sometimes if the Java-WS is slow to respond, sometimes causing ASP.NET to reach its timeout value and throw the connection.
I need to ensure data connectivity between these two applications, which I can do by increasing the timeout value, but I cannot have the ASP.NET form wait longer than a couple of seconds.
This is where the idea of a queuing system comes into place.
My idea is, to have the ASP.NET form build the soap request and then queue it in a local queue, where then a Daemon runs and fires off the requests at the Java-WS.
Before I start building something from scratch I need a couple of pointers.
Is my solution viable ?
Are there any libraries etc already out there that I can achieve this functionality with ?
Is there a better way of achieving what i am looking for ?
You can create a WindowsService hosting a WCF service.
Your web app can them call the WCF methods of your Windows Service.
Your windows service can call the java web service methods asynchronously, using the
begin/End pattern
Your windows service can even store the answers of the java web service, and expose them through another WCF methods. For example you could have this methods in your WCF service:
1) a method that allows to call inderectly a java web service and returnd an identifier for this call
2) another method that returns the java web service call result by presenting the identifier of the call
You can even use AJAX to call the WCF methods of your Windows Service.
You have two separate problems:
Your web form needs to learn to send a request to a service and later poll to get the results of that service. You can do this by writing a simple intermediate service (in WCF, please) which would have two operations: one to call the Java service asynchronously, and the other to find out whether the async call has completed, and return the results if it has.
You may need to persistently queue up requests to the Java service. The easiest way to do this, if performance isn't a top concern (and it seems not to be), is to break the intermediate service in #1 into two: one half calls the other half using a WCF MSMQ binding. This will transparently use MSMQ as a transport, causing queued requests to stay in the queue until they are pulled out by the second half. The second half would be written as a Windows service so that it comes up on system boot and starts emptying the queue.
you could use MSMQ for queuing up the requests from you client.
Bear in mind that MSMQ doesn't handle anything for you - it's just a transport.
All it does is take MSMQ messages and deliver them to MSMQ queues.
The creation of the original messages and the processing of the delivered messages is all handled in your own code on the sending and receiving machines: the destination machine would have to have MSMQ installed plus a custom service running to pick them up and process them
Anyway there is a librays for interop with MSQM using JAVA : http://msmqjava.codeplex.com/
Another way could be you can create a queue on one of your windows box and then create a service that pick up the messages form the Queue and foreward them to the Java service