I have a news app, which fetches rss feed through a cron every hour. As the list of files (URLs) it has to fetch is in 100's it gets stuck. Is it possible to write a cron script, which is conditional, like fetching only 10 at a time from the list of 100's and run
You can do it simply by adding a timestamp field in database. Which update on every fetch. So when your url fetch by cron its update the last fetch time. Then you can make a query which fetch the urls according to last fetched time in DESC order and Limit 10.
So only 10 urls fetched which is not fetch from a long time. I think this will solve your problem.
Related
I have a script that works as a daemon.
This script every so often is supposed to retrieve the order data and then processes the data.
In a situation where the script is running and retrieves the data of a given order and the order has a status of, for example, "on-hold" and then I change its status to "processing" the script still sees the status "on-hold" when I retrieve the data via wc_get_order because it uses an internal wp cache that is not refreshed.
So how do I retrieve the most current order data from the database.
I searched in the source code if there is perhaps a parameter to force the retrieval of data from the database but did not find it.
After hours of searching I made it.
wp_using_ext_object_cache( false );
wp_cache_flush();
wp_cache_init();
Using this 3 lines of code clear cache.
I have built a micro-service where there is an API called deleteToken. This API(when invoked) is supposed to change the status in a tuple in db corresponding to token (identified with token id) to "MARK-DELETE". Once that tuple has status "MARK_DELETE" then after 30 days there should be a rest call made to downstream service API called deleteTokenFromPartner. There is no such mandate like call to deleteTokenFromPartner has to be made right after 30 days, it can be done few hours later 30 days also. So what I thought was I will write a scheduler (using Quartz, Java Executor service) with scheduled period in such a way that it will run once everyday. what it will do is it will query db and find out all rows which has status="MARK_DELETE" and status update is older than 30 days. After then it will iteratively call deleteTokenFromPartner for each and every row. There is one db which is highly available and we may not have any issue with consistency as we delete after 30 days. But the problem I am seeing is, as this is a micro-service which has N instances so every instance will query db, get the same set of rows and make call to same rows. Can I make any tweak so that this duplicated calls can be avoided. FYI we don't make any config changes using hostnames and if only one instance will be capable of running the scheduler that too will be fine.
I have clients connecting to the database with javascript.
I also have code running on my server and I'm trying to do a transaction following example as shown here:
https://firebase.google.com/docs/database/server/save-data#section-transactions
Here's a simplified structure of my data
users:
userguid
resource : "room1"
printer : "printer1"
resources
rooms
room1
printers
printer1
counter : 15
The web client would write a request to their own node under "users".
The server is watching for those request and updates the counter for that resource.
If i have the transaction watching for child added I get null for counter so I can't increment the number. If I also watch for child modified the I will get the correct counter value.
I understand from the documentation that the value in transaction can be null but I'm not sure how I can fix my use case to do what I need.
Basically I don't want the client touching the counter, I want the server to read and update that value.
I've gone thru this post
Firebase runTransaction not working
but I'm not clear on how to structure my code to deal with this.
Before I tackle this solution, I wanted to run it by the community to get feedback.
Questions:
Is my approach feasible? i.e. can it even be done this way?
Is it the right/most efficient solution?
If it isn’t the right solution, what would be a better approach?
Problems:
Need to send mass emails through the application.
The shared hosted server only permits a maximum of 500 emails to be sent per hour before getting labeled a spammer
Server timeout while sending batch emails
Proposed Solution:
Upon task submittal (i.e. the user provides all necessary email information using a form and frontend template, selects the target audience, etc..), the action will then:
Determines how many records (from a stored db of contacts) the email will be sent to
If the number of records in #1 above is more than 400:
Assign a batch number to all these records in the DB.
Run a CRON job that:
Every hour, selects 400 records in batch “X” and sends the saved email template until there are no more records with batch “X”. Each time a batch of 400 is sent, it’s batch number is erased (so it won’t be selected again the following hour).
If there is an unfinished CRON JOB scheduled ahead of it (i.e. currently running), it will be placed in a queue.
Other clarification:
To send these emails I simply iterate over the SWIFT mailer using the following code:
foreach($list as $record)
{
mailers::sendMemberSpam($record, $emailParamsArray);
// where the above simply contains: sfContext::getInstance()->getMailer()->send($message);
}
*where $list is the list of records with a batch_number of “X”.
I’m not sure this is the most efficient of solutions, because it seems to be bogging down the server, and will eventually time out if the list or email is long.
So, I’m just looking for opinions at this point... thanks in advance.
im working on a project, where i want to fetch last minute flights and then save them into my database. The problem is that i don't want scrape everytime the user visits the website and then save into my database because that will only cause alot of duplicates. Can i somehow make the website fetch the data for me on a scheduled time and then delete previous records in the database?
If you want the OS to execute a task periodically, cron job is what you want.
Either get the cron job to call your program via the command line, or use wget to fetch the page that would trigger the data fetching.
More on cron jobs:
http://www.thesitewizard.com/general/set-cron-job.shtml