My knowledge and experience of this is limited, so not sure if I've considered everything and would appreciate some advice.
Current scenario: one Exchange server, one database server, multiple load balanced webservers, all sending confirmation and alert emails on a regular basis. This exchange server is used extensively across the business which is causing a high load which the exchange server can't handle. At one point the server went down for an extended length of time, leading to emails not reaching their destination and with no logging of email attempts to determine what emails failed, so that they can be manually sent at a later date. Additionally, the website waits for the SMTP response so there's a delay in the web page loading.
Plan: A second SMTP server is to be introduced to reduce the load on the original server, along with implementing some improvements to the website to reduce it's resource hogging and improve logging of all email attempts (just header info), with SMTP response status.
Possible Solutions:
Change the website's email method to log, then attempt SMTP via one or other server, record the smtp response. Pros: Quicker to develop, Cons: No asynchronous processing of email queue.
Log email headers, then use MSMQ or RabbitMQ to send email request to a queue, and have a listener service that handles the process of actually sending the email to the Exchange server. Additionally if it continues to fail to reach the SMTP servers, leave it in the queue and retry again at a later time. Pros: Can re-use the infrastructure and experience to use MQ in future feature requirements like user requests to download data, a la Facebook. Cons: Time to learn how to do it, and actually develop when faced with a deadline.
Log email to db, then send to a local pickup folder which relays to the Exchange server, but I think this has no failover support, or method for updating the status in the email log, so while it seems quick to set up, doesn't really meet all of the requirements as far as I can see.
What I'm tempted to do is do is develop in two phases, initially just the first solution then develop the MSMQ/RabbitMQ functionality to make it asynchronous at a later date, but thought I'd ask for opinions to see if I'm overlooking something.
You may want to have a look at NServiceBus.
Specifically, there's some very new email functionality. A satellite has been created that gives you the benefits of the retries that come with NServiceBus to make your email sending more robust.
You get a separate queue without requiring a new endpoint or configuring any message mappings. This way every endpoint has this capability built in and the only config needed is a SmtpClient config section.
The API for this feature is Bus.SendEmail(new MailMessage). They’re using the System.Net.Mail.MailMessage as the abstraction for sending emails.
The code is currently in their develop branch. A sample demonstrating all this can be found here.
For more details see here
Related
I am a JR dev and I have an inherited a very fractured application that sends e-mail alerts for a lot of different events. I need to stop these alerts from going out temporarily.
Unfortunately the code is very dispersed and I can't comb through it all to find every piece of code that sends an e-mail. There's many functions that send e-mails. There's no central sendEmail function that's called through the application.
Is there a way stop SMTP for the whole application?
There are several ways you could go about this.
But it is difficult to diagnose without source code or some kind of guidance.
Solutions:
Simply look into the SMTP settings and leave them blank or comment them out
Search for a centralized send function somewhere in the application (provided it exists)
Look into whether or not your application has an SMTP service ex. SendGrid.
Although if this is to be a temporary solution, I would advocate using try/catch blocks to find which emails
yield success later down the line. Logging success metrics will allow you to document and understand the application better.
https://learn.microsoft.com/en-us/dotnet/standard/exceptions/how-to-use-the-try-catch-block-to-catch-exceptions
Additional info:
Disable sending emails in asp.net
I'm upgrading a drupal website. There are thousands of users and it is using messaging and notifications. By accident I have sent out loads of notifications to users. Is there any way I can prevent the emails going out to users and still carry out my upgrades? I don't want to disable the modules as I need them enabled to do the upgrades.
Any help appreciated
If you mean to temporarily disable/reroute mails on a test/staging instance of your site, take a look at the various options provided in this post.
If you are talking about a live site, be careful - those methods mostly apply to all mails send by Drupal, so you would also disable other mails like e.g. a 'reset password' request.
Usually in Drupal, mail is sent out instantaneously to the mailserver. In some situations, people setup queues in Drupal to send mail. These mails are then sent out from the queue (maybe 50 to 100 at a time for example) when the cron.php runs.
Are you using any of these modules (or similar ones)?
http://drupal.org/project/queue_mail
http://drupal.org/project/mailq
If you are, then you might be able to "empty" the queue for any pending emails.
In general, you can make sure that mails are not sent by blocking the smtp port (usually port 25) on your computer while you are doing an upgrade. You can easily block the port by making a temporary firewall rule.
There might be also be a queue on your mailserver. So you might be able to go to the webserver and delete any pending emails that have not been sent out yet.
I have a data entry and editing form and in every data entry or update event, I have to send an email to a dynamic list of recipients. I have been sending the email as soon as the user clicks the save or edit buttons but am thinking of first saving the data to the database, and then sending the email later. I want to do this partly to improve the response time of the application as the email sending tends to take a long time than desired.
Has any one done some thing some how related to this, is there a better way of implementing something similar or does one know a good tutorial on such.
The email body is html formatted.
You could write a Windows Service that handles sending your emails, then use a Message Queue as the method of passing data from your application to the service. I.e. your applicaiton saves the data, then adds a message to the Queue. The service continually polls the queue for messages, sending each one as an email.
I agree with ck about using a service and a message queue, but there are some alternatives.
One is to use a service that polls the database at a regular interval. This lets you avoid the message queue at the cost of a higher cpu load (the service will do many unnecessary database calls).
You could also do this directly in the database using either a database trigger or a scheduled job in the database. The latest versions of SQL Server supports running stored procedures written in C# or Vb.Net so you could probably reuse much of your existing code here.
Finally you could go for a simple solution where you do the email sending on a separate thread in your asp.net application. This way you avoid the need of a service application and you can reuse your code more or less as it is today.
One way to do this is write to the database, and then put a message on a queue that tells an email service (written as a Windows service) that there are emails to send. The email service then talks to the database to find what it actually needs to do. This decouples the email service from the web application and also avoids polling.
This is slightly different to ck's solution in that the queue message is used as a trigger rather than containing the email information. This decouples the web app and the email service to some extent, and means the email service can be reused by multiple clients without each client having to observe (and keep in step with) the same email message format.
I have read a few other posts on here about mass emailing, but I need some other information. We want to integrate our mass e-mail system into our current application, but I fear this may not be a good idea. Does anyone have any input as to what kind of resources are needed to send, potentially, 1000's of emails an hour? We only have one web server right now and do not have access to our mail server. We do have full access to our web server, however. Would it be a bad idea to host the mass e-mail application on our current server? Should we have a separate sever to run this type of application and call the server via web services (WCF)? Each email would need to be sent separately because the links on the e-mail are for tracking and unsubscribing purposes. I appreciate your thoughts.
Daniel
Yes its a bad idea,
One of the key things for detecting spam, is the IP address it came from (Known as its reputation) if you start sending out alot of emails which end up getting marked as spam (Which is likly, even if the users said they wated to recieve email) your IP address could end up in trouble. I wouldnt fancy doing this on my webserver.
As for specs, your bandwidth is going to be hit hard when you start doing somthing like this. Although I think you would be suprised at how fast you can send out emails (Mail Server Dependant)
Either way when it comes to sending mass mail campaigns out, I'd speak to a third party if its somthing you are serious about, they have the knowledge and hardware to perform campaigns that are not going to damage your IP rep.
I think this issue all depends on your mail server. I've been in a project where we sent out 50-100K emails in a few hours and the main limitation was the capacity of the mail server. In my opinion it would be fine to have the mail client run on the web server as it won't be using too much resources if well written, the main question is what the mail server can take.
You can help your reputation on some email providers by contacting them and or filling out bulk mail registrations.
This is for yahoo:
http://help.yahoo.com/l/us/yahoo/mail/postmaster/bulkv2.html
I would not recommend using exchange for anything like this. You will want a separate bulk mail server like Ironport.
http://www.ironport.com/
Otherwise your internal emails will get stuck in the queue with all of your bulk emails which you definitely don't want.
The actual code involved does not have to be that complex. In my case we used an asynchronous task to loop over an email list table in the DB pulling out a batch of 20 or so at a time sending then sleeping for a number of seconds. Email providers do not like receiving hundreds/thousands of emails from a source at one time. Even better if you can stagger the list by providers.
Is there any mail queue concept in ASP.NET?
I want to send thousands of different mail to thousands of users (i.e. each user will have a different mail). I want to send the mail at a particular time, so each user receives it at a constant time.
There really is not mail queue in the Core framework. You can send individual messages synchronously or asynchronously, but you can't really send a bunch at once.
You can queue your messages by storing them to a database or file server and then kicking off a job to loop through your saved messages and send them off.
Also, not all of your users will receive the messages at the same time, even if you could send them at the same time. There are too many external variables and dependencies (network traffic, mail server loads, spam filters) to accurately predicate when or even if your users receive their messages.
There's no native MailQueue concept within .NET framework. The queue will have to be implemented yourself. In your case, you would like the mails for each recipients to be sent at about the same time for all batches. Am I right?
Well, this is a bit tricky. You can use any SMTP server, localhost or external ones. But that also mean although you can dispatch to the SMTP server at a specific time, there's no guarantee it will reach the recipients immediately.
There are a whole bunch of stuff on mail delivery which are not exactly programming related (grey listing, spam filtering etc etc).
The alternative is to have full control on the sending and have your app directly sending the mails to the recipients' mail servers. Well that is workable and I suggest you use a commercial or a good open source component for that. Anyhow, there's still a whole bunch of issues you need to deal with, (e.g. some receiving mail server like Yahoo might block the sending a few times and let it through after a few retries).
I've posted a related question, take a look at the replies here.