Hi I am using log4j2 in my application and I am using log4j2 asynchronous logging..I have created four asynchronous loggers writing to four different files. While running the application I could see multiple async logging threads in my server and they are always active.. is this the normal behavior? enter image description here
Related
I am working on an asp.net mvc-5 web application, deployed under windows 2012 & iis-8. my asp.net mvc have many CRUD operations which are implemented as action methods inside my asp.net mvc.
But my asp.net mvc web application will be doing scheduled long running network scan process, the network scan will mainly do the following steps:-
Get the list of our servers and vms from our database.
Get the scanning username and password for each server and vm from a third party tool, using Rest API.
Call some powershell scripts to retrieve the servers & vms info such as network info, memory, name, etc.
Update our ERP system with the scan info using Rest API.
Now I did a pilot project using the following approach:-
I define a Model method inside my asp.net mvc to do the above 4 steps.
Then I install hangfire tool which will be calling the scan method on predefined scheduler.
Also I create a View inside my asp.net mvc which allow users to set the hangfire schedule settings (this require to do an IIS reset on the host server for hangfire to get the new settings).
Now I run a test scan for a round 150 servers which took around 40 minutes to complete , and it worked well. The only thing I noted is that if I set the schedule to run on non-business hours (where no activity is made on IIS) then hangfire will not be able to call the job, and once the first request is made the missed jobs will run. I overcome this limitation by defining a windows task which calls IIS each 15 minutes, to keep application pool live, and it worked well...
Now the other approach I am reading about is doing my above is as follow:-
Instead of defining Model method inside asp.net mvc to do the scan, I can create a separate console application to do the scan.
Then inside my asp.net mvc to create a view which allow users to create and schdule a task inside the windows tasks scheduler. I can do so by integrating with the windows task scheduler API.
Where this windows task will be calling the console application.
Now I am not sure which approach is better and why ? now generally speaking long running/background jobs should not run under iis.. But at the same time defining these long running processes as console app and calling these apps inside windows task scheduler will create extra dependencies on my web application. And will add extra effort when moving the application from move server to another (for example from test to live)..
Beside this I read that tools such as hangfire, quartz and other are designed to allow running long running tasks inside IIS and they eliminate the need to create console applications and scheduling these console applications using task scheduler ..
So can anyone advice on this?
In my opinion, if it is possible to solve the scheduling problem on the web application side, there is no need to create a scheduler task or a new console application for triggering purposes. The problem you will probably face when using scheduling task in a web application is generally common as you might see is that: The scheduler works like a charm during debugging of the web application, but not being able to trigger after publishing it to IIS. At this point the problem is generally related to IIS rather than the schedulers Quartz.NET, Hangfire, etc. Although there are lots of articles or solution methods posted on the web, unfortunately only some of them is working properly. In addition to this, most of them require lots of configuration settings on the web and machine configuration.
However, there are also some kind of solutions for such a kind of scheduling problem and I believe in that it is worthy to give a try Keep Alive Service For IIS 6.0/7.5. Just install it on the server to which you publish your application and enjoy. Then your published application will be alive after application pool recycling, IIS/Application restarting, etc. That is also used in our MVC application in order to send notification mails weekly and has been worked for months without any problem. Here are the sample code that I use in our MVC application. For more information please visit Scheduled Tasks In ASP.NET With Quartz.Net and Quartz.NET CronTrigger.
*Global.asax:*
protected void Application_Start()
{
JobScheduler.Start();
}
*EmailJob.cs:*
using Quartz;
public class EmailJob : IJob
{
public void Execute(IJobExecutionContext context)
{
SendEmail();
}
}
*JobScheduler.cs:*
using Quartz;
using Quartz.Impl;
public class JobScheduler
{
public static void Start()
{
IScheduler scheduler = StdSchedulerFactory.GetDefaultScheduler();
scheduler.Start();
IJobDetail job = JobBuilder.Create<EmailJob>().Build();
ITrigger trigger = TriggerBuilder.Create()
.WithIdentity("trigger1", "group1")
.StartNow()
.WithSchedule(CronScheduleBuilder
.WeeklyOnDayAndHourAndMinute(DayOfWeek.Monday, 10, 00)
//.WithMisfireHandlingInstructionDoNothing() //Do not fire if the firing is missed
.WithMisfireHandlingInstructionFireAndProceed() //MISFIRE_INSTRUCTION_FIRE_NOW
.InTimeZone(TimeZoneInfo.FindSystemTimeZoneById("GTB Standard Time")) //(GMT+02:00)
)
.Build();
scheduler.ScheduleJob(job, trigger);
}
}
Also I create a View inside my asp.net mvc which allow users to set the hangfire schedule settings (this require to do an IIS reset on the host server for hangfire to get the new settings).
You're resetting your webserver to update a task's schedule? That doesn't sound healthy. What you might do is keep track of what the scheduled time should be, and on execution, check if the current time is within a certain range of the scheduled time (or has already been executed), otherwise abort the job.
The only thing I noted is that if I set the schedule to run on non-business hours (where no activity is made on IIS) then hangfire will not be able to call the job, and once the first request is made the missed jobs will run. I overcome this limitation by defining a windows task which calls IIS each 15 minutes, to keep application pool live, and it worked well...
Hangfire's documentation has a page about running delayed tasks that mentions what you need to change to accomodate this.
Using Windows' Task Scheduler doesn't seem like a good idea; it's not meant for the execution of ad-hoc, short-lived tasks. You probably need elevation to create tasks, and you'd probably need to define another scheduled task to clean up the mountain of tasks that would exist after a few dozen background jobs have been executed.
You're also correct that using Windows' Task Scheduler would make it more difficult to move your application around.
I created a service with a RESTful API in ASP.NET, hosted in IIS. Inside this service, I would like to create an actor system with Akka.NET.
Upon creating the actor system:
var actorSystem = ActorSystem.Create("myActorSystem");
The following exception is thrown:
A first chance exception of type 'System.InvalidOperationException' occurred in System.Web.dll
Additional information: An asynchronous operation cannot be started at this time. Asynchronous operations may only be started within an asynchronous handler or module or during certain events in the Page lifecycle. If this exception occurred while executing a Page, ensure that the Page is marked <%# Page Async="true" %>. This exception may also indicate an attempt to call an "async void" method, which is generally unsupported within ASP.NET request processing. Instead, the asynchronous method should return a Task, and the caller should await it.
The actor system is inherently a concurrent system with asynchronous messages being exchanged between actors. As explained here, this actor system would not survive IIS taking down the AppDomain, which is probably why the aforementioned exception is thrown.
This article explains how to run background tasks in ASP.NET. However, I don't see how I could use this for my actor system, as I have no control over the lifecycle of background tasks that might be created by Akka.NET.
Is there a way to make this work, or should I abandon the idea of having an actor system in an ASP.NET application?
EDIT: I also saw a question on Stackoverflow about implementing a REST service using Akka. Any advice about a solution similar to the Spray toolkit, but working for Akka.NET would be welcome.
I've used Akka.NET and Akka.Remote inside ASP.NET MVC applications that are doing up to 1000 requests per second on EC2 - so I'll share some of the tips and tricks I used to get it up and running successfully. Had a prototype version that even used Akka.Cluster but ended up not shipping that version.
Best place to call ActorSystem.Create is inside Global.asax Application_Start().
Hang onto a static reference to the ActorSystem object inside Global.asax itself, using a static field or property. Helps ensure that the ActorSystem itself doesn't get garbage-collected in long-running applications.
Create a separate static helper class to initialize any top-level actors your applications needs - i.e. actors at the top of the /user/ hierarchy. This class should also provide actor paths that your ASP.MVC controllers and action methods can use for Tell and Ask operations.
Creating the ActorSystem is a bit of an expensive operation, because lots of system-level stuff gets fired up at once. It's definitely best to do this once at application startup and then just cache the result inside the Application class.
Creating individual actor instances is cheap - you should be able to do this no-problem inside ASP.NET MVC action methods. If you see this error come up again, please let us know what part in the request-handling process this error occurred and with which version of ASP.NET.
Edit: added some updated guidance for how to do this on ASP.NET Core
https://petabridge.com/blog/akkadotnet-aspnetcore/
Keep your ActorSystem as a shared property in some static class container - this way you may access it from the rest of your application. Actor system initialization/disposal can be done by:
Global.asax - use ActorSystem.Create(...) inside Global.asax Application_Start and dispose it with system.Shutdown() on Application_End.
OWIN - create actor system in OWIN's Startup.Configuration method and shut it down by binding to host.OnAppDisposing event (how-to link).
Remember that IIS will startup your web app only after first request and tear it down automatically after some time when it's idle. Therefore make sure, that your deployment script will ping application after publishing and set idle timeout (link) for long enough if you want your Akka actor system to run continuously.
Second option
Separate your Actor System logic and deploy it, for example, as a Windows Service (or Linux deamon). Turn on Akka.Remoting for it and create a proxy client, which will forward all application long-running sensitive tasks to external service. Similar solution is often used for things such as schedulers or event buses, when your application logic must be working continuously.
I have an ASP.NET Webform which currently calls a Java WebService. The ASP.NET Webform is created/maintained inhouse, whereas the Java WS is a package solution where we only have a WS interface to the application.
The problem is, that the Java WS is sometimes slow to respond due to system load etc. and there is nothing I can do about this. So currently at the moment there is a long delay on the ASP.NET Webform sometimes if the Java-WS is slow to respond, sometimes causing ASP.NET to reach its timeout value and throw the connection.
I need to ensure data connectivity between these two applications, which I can do by increasing the timeout value, but I cannot have the ASP.NET form wait longer than a couple of seconds.
This is where the idea of a queuing system comes into place.
My idea is, to have the ASP.NET form build the soap request and then queue it in a local queue, where then a Daemon runs and fires off the requests at the Java-WS.
Before I start building something from scratch I need a couple of pointers.
Is my solution viable ?
Are there any libraries etc already out there that I can achieve this functionality with ?
Is there a better way of achieving what i am looking for ?
You can create a WindowsService hosting a WCF service.
Your web app can them call the WCF methods of your Windows Service.
Your windows service can call the java web service methods asynchronously, using the
begin/End pattern
Your windows service can even store the answers of the java web service, and expose them through another WCF methods. For example you could have this methods in your WCF service:
1) a method that allows to call inderectly a java web service and returnd an identifier for this call
2) another method that returns the java web service call result by presenting the identifier of the call
You can even use AJAX to call the WCF methods of your Windows Service.
You have two separate problems:
Your web form needs to learn to send a request to a service and later poll to get the results of that service. You can do this by writing a simple intermediate service (in WCF, please) which would have two operations: one to call the Java service asynchronously, and the other to find out whether the async call has completed, and return the results if it has.
You may need to persistently queue up requests to the Java service. The easiest way to do this, if performance isn't a top concern (and it seems not to be), is to break the intermediate service in #1 into two: one half calls the other half using a WCF MSMQ binding. This will transparently use MSMQ as a transport, causing queued requests to stay in the queue until they are pulled out by the second half. The second half would be written as a Windows service so that it comes up on system boot and starts emptying the queue.
you could use MSMQ for queuing up the requests from you client.
Bear in mind that MSMQ doesn't handle anything for you - it's just a transport.
All it does is take MSMQ messages and deliver them to MSMQ queues.
The creation of the original messages and the processing of the delivered messages is all handled in your own code on the sending and receiving machines: the destination machine would have to have MSMQ installed plus a custom service running to pick them up and process them
Anyway there is a librays for interop with MSQM using JAVA : http://msmqjava.codeplex.com/
Another way could be you can create a queue on one of your windows box and then create a service that pick up the messages form the Queue and foreward them to the Java service
I am working on a functionality that will generate around 2000 excel sheets. THe process will accept input from front end web application. I dont want my web application to wait till the process of generating the excel files is over. I should write a console application and trigger it through web application or a web service? Whic is the better way of implementation?
Thanks,
Rohit
I would suggest storing excel-gen as a task in a database, then have a background service constantly running, checking that task queue, processing it (creating spreadsheets), then finally updating job records as done. Your asp.net app can in turn, aside from creating job records, simply check on status periodically or on page-refresh.
Duplicate
This is a close duplicate of Dealing with a longer running process in WCF. Please considering posting your answer to that one instead of this.
Original Question
I'm implementing the business layer of an application that must run some background processes at scheduled times. The business layer is made up of several WCF services all running under the same web application.
The idea is defining a set of 'tasks' that must be run at different times (eg. every 5 minutes, everyday at 23:00, etc). That wouldn't be hard to implement as a windows service, but the problem is, the tasks need access to data caches that are living in the services, so this 'scheduler' must run under the IIS context in order to access that data.
What I'm doing currently is using a custom ServiceHostFactory in one of the WCF services which spawns a child thread and returns. The child thread sleeps and wakes up every X minutes to see if there are scheduled tasks and executes them.
But I'm worried about IIS randomly killing my thread when it recycles the application pool or after some inactive time (eg. no activity on any of the WCF services, which listen for requests from the presentation layer). The thread must run uninterrupted regardless of activity on the services. Is this really possible?
I have found an article by someone doing the same thing, but his solution seems to be pinging the server from the child thread itself regularly. Hopefully there is a better solution.
I have at some point implemented a Windows Service that would load a web page on a regular basis. The purpose of that was was that the site was hosting a Workflow Foundation runtime, and we wanted to ensure that the web application was brought back up after IIS recycling the application pool. Perhaps the same approach can be used in this case; have a service (or Scheduled Task in Windows; even simpler) run every x minutes and load a page that will check for tasks.
Is it a possibility to run either a Windows Service or place applications in the Windows Scheduler to execute methods in the WCF at certain times? Maybe use a BackgroundWorker inside the WCF. Another option would be for WCF to spawn other applications to do the business logic, passing the appropriate data, or pointers to the data in memory(unsafe).