We’re running into an issue sending duplicate notifications to our users using the Notifications module on our Mercury Pressflow implementation. The duplicate messages are identical save one thing- the [node-url] token is being replaced with ‘default’ in one of the messages. All the other tokens in the message are being replaced correctly.
The duplicate emails do not happen consistently, maybe 10-15% of the notifications sent out, however a duplicate message always has the proper url & the ‘default’ url.
The only major modification we’ve made to Mercury was spinning off MySQL to it’s own server and adding replication. We currently have the reads set up to round robin between the 2 MySQL instances.
I have done the following troubleshooting based on finding similar issues
made sure the cron job is calling the correct url
replaced all configurations named ‘default’ with the site name (Memcached, Varnish, and Apache configs)
disabled caching in an init_hook in the notifications module
Has anyone out there experienced anything similar with Notifications and Mercury? Any and all advice is greatly appreciated.
The "Mercury" stack is external to Drupal and doesn't affect how email is queued or sent. Something within your messaging/notifications configuration or use is causing multiple messages to be created.
If you have any custom code here, I would look at that and try to trace the token variance.
Related
Is there a way to set up Alfresco to send email to admins on each server error as in Python Django Error reporting via e-mail?
Thanks in advance for any hint.
The first thing I would say is that you need to really look in to the viability of using e-mails to help admins spot problems with servers. Sending an e-mail can be a heavy operation and as such, particularly with many repeated errors, can make an existing problem worse. And what about the cases where the ability to send e-mails goes down or a problem so severe happens that it simply cannot send e-mails?
For properly setting up reporting on your servers you might want to look at Logstash + Kibana:
http://alfresco.blog.redpill-linpro.com/2014/07/01/alfresco-logging-with-logstash-and-kibana-part-1-the-basics/
That being said, it should be possible to configure log4j to use an SMTP appender if that is the route you want to go down:
What is the proper way to configure SMTPAppender in log4j?
In Alfresco 4.2 you should need to add something such as the following to the log4j.properties in tomcat/webapps/alfresco/WEB_INF/classes/:
#CONFIGURE SMTP
log4j.appender.email=org.apache.log4j.net.SMTPAppender
log4j.appender.email.SMTPHost=mail.mydomain.com
log4j.appender.email.SMTPUsername=myuser#mydomain.com
log4j.appender.email.SMTPPassword=mypw
log4j.appender.email.From=myuser#mydomain.com
log4j.appender.email.BufferSize=1
log4j.appender.email.EvaluatorClass=TriggerLogEvent
log4j.appender.email.layout=org.apache.log4j.PatternLayout
log4j.appender.email.layout.ConversionPattern=%m
We are not able to call any of our APIs on our organization, we are receiving classification_failure on each API call even the testing ones that are related to weather.yahoo.com etc..
each time we open the API proxy the message the following message is displayed (No server entry found with ID 1c53328c-57b6-4f69-bdf0-ce4b39e64ef3) .
we had an increase of the traffic on our apis , is it possilble that apiggee closed the api because they monitored high volume of traffic.
I created another organization it is all working fine
I passed this along to our support team. It may be a hardware problem. (Just saw this problem with a colleague and he got it resolved. Check yours to see if it was related)
You can also email help#apigee.com with details about your org and traffic spike.
I am sending out a nightly email through rules scheduler, when I manually execute it sends out one email as it should, however when it runs on the schedule it sends me 10 duplicate emails. I've looked all over and can't seem to find any solution to the problem.
Thanks in advance for any suggestions
Use Job scheduler module. In this module you first insert the data in job_schedule and create a queue for each schedule . when crons run it start executing each queue and send mails then it delete its entry from the job_scheduler table. hence it will not send same mail again and again to the same person.. There is proper documentation in job_scheduler module in drupal7. Just go through it.
This sounds like a bug in the Rules module; it has its quirks. I see you have reported this issue in the Rules issue queue: http://drupal.org/node/1314916, which is what I was first going to suggest. So now I know your issue is for Rules 7.x-2.x dev integration with Views 7... both of which have more than a few bugs. I strongly suspect this issue has as much to do with Views as with Rules. (The 10x repetition seems unlikely to be a coincidence since 10 is a default value for results-per-page in Views, etc)
When you report an issue, it's helpful to include all pertinent information (Drupal version, steps to replicate, what's written to the log, etc). I'd personally suggest seeing if you can replicate your issue in a clean installation of Drupal with just the modules necessary to run your test. If you can replicate it that way, it's easier to provide enough information for the developers to identify the issue and resolve it. (e.g. use Devel generate to create some nodes and dummy users, then create a very simple view, e.g. just titles of the five most recent nodes, and use that view as the source for your email content. Does it send 5 copies? You may need to configure a localhost mail server to test this.
i have to write application for sending newsletter.
what is the best way to send newsletter thoundands of users?
My requirement is
Each mail is seprately as To :
Every mail has unique Unsubscribe link
Is is good to use SMTP mail class of .net?
I look aound may questions in so but can't decide which approcah i should go?
There are many suggestions
Multi threaded Windows service
Use Mail Server
Add thread.sleep(2000) between each send.
can anyone suggest good way to imepement this?
I would not recommend asp.net webpage to send, even if you do start it in a separate background thread. I would think you run the risk of the server recycling your process in the middle of the send, which would mess it up. You really need to write some kind of separate service or application to send your emails.
The simplest option would be to just create a quick and dirty console or windows form application.
Also logging is critical just like the other poster said. If it fails you want to know exactly what got sent out and where it stopped so that when you restart it you don't mail all the people who it did work for again. You want to be able to input the starting point for the send, so if you need to restart at number email #5000 you can.
The classes in System.Net.Mail namespace will work just fine for sending your mail.
One of the biggest problems will be finding a email host that will let you send so many emails. Most email hosts have throttling and sometime it changes depending upon server conditions so if the server is being heavily used then the email limits will be more restrictive, and you may only get to set 500 emails per hour.
We have a newsletter that goes out to around 20000 people as separate emails and we had to play around with the delay between emails until we found one that would work for our email host. We ended up with 1.2 sec between emails, so that might be a good starting point.
I think there are email hosts specialize in bulk mailings though so if you get one of those it might not be a problem.
Also if you host your own email this may not be a problem. And if you do host your own mail you will have the option of dropping the mail in the pickup directory and you could just dump it all in there as fast as you want, and let the email service pick it up at it's own pace.
EDIT: Here is the settings to add to the config file for setting the pickup directory
<system.net>
<mailSettings>
<smtp from="support#test.com" deliveryMethod="SpecifiedPickupDirectory" >
<specifiedPickupDirectory pickupDirectoryLocation="Z:\Path\To\Pickup"/>
</smtp>
</mailSettings>
</system.net>
Definitely do not do this in ASP.NET. This is one of the biggest mistakes that new web developers make.
This needs to be a windows app or service that can handle this much volume.
I've written pages that send emails, but not nearly the volume yours will. Nonetheless, I would recommend the following based on code I have implemented in the past:
Use the web application to write out the email and all the recipient addresses to database table(s).
Have a process that is outside of ASP.NET actually send the emails. This could be a vbs file that is set up as a scheduled task, or (preferably) a windows service. The process would take the text of the email, append the unsubscribe link, and once sent successfully flag the database record as sent. That way, if the send fails, it can try again later (the send process loops over all the records flagged as unsent).
If you need a log of what was sent and when, you just need to keep the sent records in the database tables. Otherwise, just delete the records once sent successfully.
IMHO sending emails within the ASP.NET worker process is a bad idea because you don't know how long it will take and if the send fails there's little opportunity to retry before the page times out.
Create a webpage to "Design" the newsletter in. When they hit Send, queue the newsletter up somewhere (database) and use another program (windows service, etc) to send the queued letter. This will be many times more effecient and potentially fault tolerant if designed properly.
I have written a Newsletter module (as part of a bigger system) in ASPNET MVC 2, Entity Framework and using the System.Net.Mail namespace. It is kicked off in view and actually just runs in a controller with a supporting method to do the send. As each email is sent I track whether there is a hard bouce (an exception is thrown) and I update that database record stating a fail with the exception, otherwise I update the record stating success. We also do personalisation so we have 'tags' that get replaced by an extra field in the database (stored as XML for flexibility). This helps handle an unsubscribe function.
My code is quite simple (please don't flame me for using exception handling as business logic ;) and it works like a charm.
This is all done on a VPS at http://maximumasp.com which also hosts 4 sites with pretty decent traffic. We use their SMTP servers. We notified them that we needed this service and have had no problems relationship-wise.
We had 2GB of RAM on the machine running Windows 2008 and it was doing 6 emails/sec. We bumped it up to 3GB as the web sites needed it and now the mailout is doing about 20emails/sec. Our mailouts range from 2,000 to 100,000 email addresses.
In short, ASP.NET can be used to handle a mailout, and if you add in some logic to handle record updating the worry of losing your way mid-send is mitigated. Yes there are probably slicker ways to do this. We are looking in to MQMS and threading, and separating that out to windows service to make it more stable and scalable as we put more clients and larger lists on, but for now it works just fine with reasonable reporting and error handling.
I need to design a bug alert system, where the web support team is notified via email when a user of our website encounters an error of any sort (database exception, or a 404)
What would be the best way to design this section of the project? Any ideas would be appreciated.
You may want to look into using the global.asax file for application-wide error intercepting. A quick search yields this step-by-step walk-through:
http://aspnetresources.com/articles/CustomErrorPages.aspx
Depending on the volume of traffic you're expecting, sending an e-mail every time an error is intercepted may not be the best approach. At best, you'd flood inboxes (and make the support staff very unhappy), and at worst you'd get your mail servers blacklisted for spamming. The approach that I've used in the past on high-traffic sites is to queue up errors in a table that is read and purged at a set interval by a separate process. The process would aggregate the errors, grouping them by type, number of occurrences, etc, then send out an e-mail report to the support mailing lists.
ASP.NET health monitoring may be of interest: http://msdn.microsoft.com/en-us/library/ms998306.aspx. It's really simpler to use than this article first appears and doesn't require any additional components - it's all built-in.
I would implement an HTTPmodule that captures the onError event.
This is would allow the module to be reused over multiple applications. The destination email addresses, SMTP server etc, could be in the HTTPmodule, overriden in the web.config file for maximum flexibility.