I need to create an asynchronous scheduler inside nginx server to update a variable. Let me give you an example what I mean by this and why I need it.
Imagine config file that looks something like this:
http {
lua_shared_dict foo 5m;
server {
location /set {
content_by_lua '
local foo = ngx.shared.foo
ngx.say(foo:get("12345"))
';
}
}
}
I specified variable foo that resides in shared memory and all worker processes have access to it. What I want to do is to set those values from lua script that will be called every minite. Just for reference it will be going to the Redis and then retrieve necessary data, and update this variable. I know I can do this in content_by_lua in every call, but it's highly inefficient for a huge volume of traffic.
I would like a separate process that would be triggered every minute or so to just go and one task. Is there anything like this in nginx or are there any modules that could help me with that?
You can use the new ngx.timer API provided by ngx_lua. See the documentation for details:
https://www.nginx.com/resources/wiki/modules/lua/#ngx-timer-at
You can create a new timer in your timer handler to make the timer keeps triggering like a cronjob ;)
BTW, the timer is per-worker process, you can use the lua-resty-lock library in your timer handler to ensure that only one timer is active at a time among all the nginx workers: https://github.com/agentzh/lua-resty-lock
You can use ngx.timer.every API. Using ngx.timer.every API is recommended over using recursive ngx.timer.at API.
Related
I want to use dynamic and central configuration strategy for my microservices (MS) (multiple instances of same type having same configuration) and I'm using Azure App Configuration (AAC). I want to minimize calls to AAC so when one MS is starting I would like to read configuration from AAC and keep that until changes made i.e I dont want for every call to Configuration["Env:service:some-param"] generate calls to AAC. The notification part I have solved via eventgrid and servicebus-events so all MSs reaches notification that changes have been made but I really cant find any good solution to force to reload Configuration from AAC on demand.
In Program.cs I hook up AAC via:
config.AddAzureAppConfiguration(options =>
options
.Connect(connection)
.ConfigureRefresh(refresh =>
{
refresh.Register(environment + ":" + service + ":<Some-param>",true)
.SetCacheExpiration(TimeSpan.FromDays(1));
_environmentRefresher = options.GetRefresher();
})
why I set SetCacheExpiration(TimeSpan.FromDays(1)) is because I dont want to make unnecessary calls to AAC and I thought that if I fetch the refresher and triggers it when event occurs configuration would be reloaded but thats seems not to be the case due to SetCacheExpiration seems to override everything so my question is ... Is given scenario not solvable in .net core or can I achieve this in some way?
Make sure to call IConfigurationRefresher.SetDirty if you want to force the cache expiration to occur using an event based refresh model. Otherwise you'll encounter the problem you mention where the cache hasn't been invalidated yet and the refresh call will be a no-op.
I am working on a issue with the design of a service which is basically redirection.
The request link I get will contain some params (abc.com/param1=v1¶m2=v2).
I need to do two tasks on this link
I need to format the link and redirect user to another domain with
some params passed(xyz.com/p1=v2) depending on the value of ,say,
param1, This step should be as fast as possible
I need to save the link details to my DB after some processing.
I am planning to do this with nginx+lua(openresty)+(Redis or Mongodb?) combination.
As the two are unrelated task I am planning to split it, to do both asynchronously.
As the first task in a redirection, ngx.redirect("/link") seems apt for the case.
But the documentation says redirect call will terminates the processing of the current request
How can I make these two tasks independent and redirection will happen as fast a possible and should not wait for the completion of the second task.
Can I make storing done by another thread and how give this job to another thread?
Yeah of course you easlt can, first of all you have to perfecctly understand the Order of lua module directive, then for make you Mongodb process in a ceperate thread you have to call in with ngx.location.capture($url), where $url is the url in your location block :
location redirect/handling {
... //
content_by_lua_file url/to/your/code/forRedirectHandling
ngx.location.capture(mongo/save):
}
location mongo/save {
content_by_lua_file url/to/mongodbHandlingdCode
}
The ngx.location.capture() will point to your second location block and make your code in another thread (nginx worker).
Pls see the openresty documentation for know wich directive to use (access_by_lua, log_by_lua...)
hope this help :)
I am working for my client using Asp.net webAPI2 and angularJS. Now my client have following requirement,but i am unable to understand what type of project i have to create like WebAPI project,window service or any other? Anyone please tell me what the client actually want and how can i do it?
QueueManager will need to be some kind of a service which would be able to run jobs on a timed basis. We envision it being a service that runs on a continuous loop, but has a Thread.Sleep at the end of each iteration with a duration of x-seconds (“x” being set in a config file.) You should create this QueueManager service as a new project within the Core.Jobs project; I would like to have the project name be “Core.Jobs.QueueManager”, along with the base namespace.
Here are the functions that the QueueManager will do for each iteration:
1) Do a worker healthcheck (JobsAPI: Queue/WorkerHealthCheck – already created)
a. This method will just return a 200 status code, and a count of workers. Not need to act on the return value.
Look at Hangfire, it is quite easy to set up and simple to use.
http://docs.hangfire.io/en/latest/background-methods/performing-recurrent-tasks.html
I have IIS7.5. We currently have a weighted rating for entities on our website. Calculating the weighted rating is extremely slow, to the point loading the homepage now takes more than 10 seconds to load.
To solve this, I'd like to store the weighting in the database with each entity, and have IIS run a script every 5-10 minutes that recalculates the weightings.
How do I go about doing this? It would be easiest for me if it ran a webpage URL.
One approach is to use a Windows service for this rather than calling a web URL.
This can then run completely out-of-band in the background to perform calculations. Details on this are here:
http://msdn.microsoft.com/en-us/library/d56de412%28v=VS.100%29.aspx
A few advantages include:
Your IIS process will not be affected, so your users will see no slowdown
The service can be stopped or started independently of the Web site
However, you'll need to have reasonably full access to the server to install and run the service.
You can use a Cache entry for this, set to expire 10 minutes in the future.
When you add the item, use a callback function for the CacheItemRemovedCallback parameter - in this callback function do your database work and re-add the expiring cache entry.
Other options include:
Using one of the timer classes included in the BCL - there is a MSDN magazine article describing and comparing the different ones.
Writing a windows service to do this.
Using a scheduled task.
Windows service and schedules tasks still require you to have some way to communicate the results to IIS.
To not use client, only server to continuously call this function, you can create a thread on the server to call the function that calculates it.
A better way to start the thread or timer is in the Global.asax, like this sample:
public class Global : System.Web.HttpApplication
{
private Timer _timer;
void Application_Start(object sender, EventArgs e)
{
int period = 1000 * 60 * 10; // 10 minutes
_timer = new Timer(TimerCallback, null, 1000, period);
}
private void TimerCallback(object state)
{
// Do your stuff here
}
}
}
I have done something like this earlier and I had used windows scheduled tasks to call my script at specific intervals of time.
A simple batch file with WGET or similar, and help from Scheduled Tasks will do it.
http://www.gnu.org/software/wget/
you can try this to test this idea:
wget http://localhost/filename.ashx
I have a php script which does the accepted answer described here.
It doesn't work unless I add the following before fclose($fp)
while (!feof($fp)) {
$httpResponse .= fgets($fp, 128);
}
Even a blank for loop would do the job instead of the above!!
But whats the point? I wanted Async calls :(
To add to my pain, the same code is running fine without the above code snippet in an Apache driven environment.
Anybody knows if Nginx or php-fpm having a problem with such requests?
What you're looking for can only be done on Linux flavor systems with a PHP build that includes the Process Control functions (PCNTL library).
You'll find it's documentation here:
http://php.net/manual/en/book.pcntl.php
Specifically what you want to do is "fork" a process. This creates an identical copy of the current PHP script's process including all memory references and then allows both scripts to continue executing simultaneously.
The "parent" script is aware that it is still the primary script. And the "child" script (or scripts, you can do this as many times as you want) is aware that is is a child. This allows you to choose a different action for the parent and the child once the child is spun off and turned into a daemon.
To do this, you'd use something along these lines:
$pid = pcntl_fork(); //store the process ID of the child when the script forks
if ($pid == -1) {
die('could not fork'); // -1 return value means the process could not fork properly
} else if ($pid) {
// a process ID will only be set in the parent script. this is the main script that can output to the user's browser
} else {
// this is the child script executing. Any output from this script will NOT reach the user's browser
}
That will enable a script to spin off a child process that can continue executing along side (or long after) the parent script outputs it's content and exits.
You should keep in mind that these functions must be compiled into your PHP build and that the vast majority of hosting companies will not allow access to them on their servers. In order to use these functions, you generally will need to have a Virtual Private Server (VPS) or a Dedicated server. Not even cloud hosting setups will usually offer these functions as if used incorrectly (or maliciously) they can easily bring a server to it's knees.