Is there any mail queue concept in ASP.NET?
I want to send thousands of different mail to thousands of users (i.e. each user will have a different mail). I want to send the mail at a particular time, so each user receives it at a constant time.
There really is not mail queue in the Core framework. You can send individual messages synchronously or asynchronously, but you can't really send a bunch at once.
You can queue your messages by storing them to a database or file server and then kicking off a job to loop through your saved messages and send them off.
Also, not all of your users will receive the messages at the same time, even if you could send them at the same time. There are too many external variables and dependencies (network traffic, mail server loads, spam filters) to accurately predicate when or even if your users receive their messages.
There's no native MailQueue concept within .NET framework. The queue will have to be implemented yourself. In your case, you would like the mails for each recipients to be sent at about the same time for all batches. Am I right?
Well, this is a bit tricky. You can use any SMTP server, localhost or external ones. But that also mean although you can dispatch to the SMTP server at a specific time, there's no guarantee it will reach the recipients immediately.
There are a whole bunch of stuff on mail delivery which are not exactly programming related (grey listing, spam filtering etc etc).
The alternative is to have full control on the sending and have your app directly sending the mails to the recipients' mail servers. Well that is workable and I suggest you use a commercial or a good open source component for that. Anyhow, there's still a whole bunch of issues you need to deal with, (e.g. some receiving mail server like Yahoo might block the sending a few times and let it through after a few retries).
I've posted a related question, take a look at the replies here.
Related
I'm currently working on an IRC-Chat and we want to add the option to chat with other people privately (User-To-User) which works fine, but the messages aren't stored, meaning that a user loses all private messages after disconnecting. They also can't message a person once they have disconnected.
All of this isn't the case with channels, where messages are stored for X time, allowing a asynchronous communication.
Is there a way of allowing asynchronous messaging for private User-To-User messages without storing the messages in an extra system? Or is this simply a limitation of IRC?
IRC isn't designed to store messages - it's a feature, not a bug. One way to get around this is to configure the server to 'simulate' past messages (essentially pretending to be the other user to send past messages). To do this, you would need to store messages - however, it would be on the server itself, not on a separate database or a third party.
My knowledge and experience of this is limited, so not sure if I've considered everything and would appreciate some advice.
Current scenario: one Exchange server, one database server, multiple load balanced webservers, all sending confirmation and alert emails on a regular basis. This exchange server is used extensively across the business which is causing a high load which the exchange server can't handle. At one point the server went down for an extended length of time, leading to emails not reaching their destination and with no logging of email attempts to determine what emails failed, so that they can be manually sent at a later date. Additionally, the website waits for the SMTP response so there's a delay in the web page loading.
Plan: A second SMTP server is to be introduced to reduce the load on the original server, along with implementing some improvements to the website to reduce it's resource hogging and improve logging of all email attempts (just header info), with SMTP response status.
Possible Solutions:
Change the website's email method to log, then attempt SMTP via one or other server, record the smtp response. Pros: Quicker to develop, Cons: No asynchronous processing of email queue.
Log email headers, then use MSMQ or RabbitMQ to send email request to a queue, and have a listener service that handles the process of actually sending the email to the Exchange server. Additionally if it continues to fail to reach the SMTP servers, leave it in the queue and retry again at a later time. Pros: Can re-use the infrastructure and experience to use MQ in future feature requirements like user requests to download data, a la Facebook. Cons: Time to learn how to do it, and actually develop when faced with a deadline.
Log email to db, then send to a local pickup folder which relays to the Exchange server, but I think this has no failover support, or method for updating the status in the email log, so while it seems quick to set up, doesn't really meet all of the requirements as far as I can see.
What I'm tempted to do is do is develop in two phases, initially just the first solution then develop the MSMQ/RabbitMQ functionality to make it asynchronous at a later date, but thought I'd ask for opinions to see if I'm overlooking something.
You may want to have a look at NServiceBus.
Specifically, there's some very new email functionality. A satellite has been created that gives you the benefits of the retries that come with NServiceBus to make your email sending more robust.
You get a separate queue without requiring a new endpoint or configuring any message mappings. This way every endpoint has this capability built in and the only config needed is a SmtpClient config section.
The API for this feature is Bus.SendEmail(new MailMessage). They’re using the System.Net.Mail.MailMessage as the abstraction for sending emails.
The code is currently in their develop branch. A sample demonstrating all this can be found here.
For more details see here
The web site I am developing will be sending tens of thousands of emails daily (and that number will be growing) - registration, notifications, alerts, etc. I will have a dedicated server box that will be actually generating and sending emails by request from the asp.net application (asp.net app calls a WCF method on the email box and provides various parameters for an email).
Now, I am trying to figure out what's the best way of queueing those email jobs on the email server. The call from asp.net app has to be async so that asp.net app doesn't wait for email server to create and send actual email.
Originally I was just creating a worker thread for each email job request but number of emails is going to be really high and I'm not sure if creating hundreds of simultaneous threads is a good idea performance wise. My next thought is to use MSMQ but I'm not sure about its performance and scalability.
Any ideas/production examples?
Thanks!
At a previous job, we had to queue messages for delivery, much like you are explaining. We decided to create a database record that represented each message. At message creation time, we created the mail message in .NET and then saved it into the database. A separate process (Windows service built in .NET) would periodically check to see if there were messages to be sent (delivery date was in the past and status was unsent). It would then re-create the mail message from the information it received from the stored procedure and sent the message along its merry way.
The procedure that returned the messages ready for sending also performed throttling logic based on the day and time of the call (we allowed more of our bandwidth to be used at night and the weekends than during the day).
We also had need for tracking bouncebacks, message opens, and click-throughs which meant having a database record that represented the email was necessary so we could relate events (bounce, open, click) with individual emails and recipients.
I have a data entry and editing form and in every data entry or update event, I have to send an email to a dynamic list of recipients. I have been sending the email as soon as the user clicks the save or edit buttons but am thinking of first saving the data to the database, and then sending the email later. I want to do this partly to improve the response time of the application as the email sending tends to take a long time than desired.
Has any one done some thing some how related to this, is there a better way of implementing something similar or does one know a good tutorial on such.
The email body is html formatted.
You could write a Windows Service that handles sending your emails, then use a Message Queue as the method of passing data from your application to the service. I.e. your applicaiton saves the data, then adds a message to the Queue. The service continually polls the queue for messages, sending each one as an email.
I agree with ck about using a service and a message queue, but there are some alternatives.
One is to use a service that polls the database at a regular interval. This lets you avoid the message queue at the cost of a higher cpu load (the service will do many unnecessary database calls).
You could also do this directly in the database using either a database trigger or a scheduled job in the database. The latest versions of SQL Server supports running stored procedures written in C# or Vb.Net so you could probably reuse much of your existing code here.
Finally you could go for a simple solution where you do the email sending on a separate thread in your asp.net application. This way you avoid the need of a service application and you can reuse your code more or less as it is today.
One way to do this is write to the database, and then put a message on a queue that tells an email service (written as a Windows service) that there are emails to send. The email service then talks to the database to find what it actually needs to do. This decouples the email service from the web application and also avoids polling.
This is slightly different to ck's solution in that the queue message is used as a trigger rather than containing the email information. This decouples the web app and the email service to some extent, and means the email service can be reused by multiple clients without each client having to observe (and keep in step with) the same email message format.
I have read a few other posts on here about mass emailing, but I need some other information. We want to integrate our mass e-mail system into our current application, but I fear this may not be a good idea. Does anyone have any input as to what kind of resources are needed to send, potentially, 1000's of emails an hour? We only have one web server right now and do not have access to our mail server. We do have full access to our web server, however. Would it be a bad idea to host the mass e-mail application on our current server? Should we have a separate sever to run this type of application and call the server via web services (WCF)? Each email would need to be sent separately because the links on the e-mail are for tracking and unsubscribing purposes. I appreciate your thoughts.
Daniel
Yes its a bad idea,
One of the key things for detecting spam, is the IP address it came from (Known as its reputation) if you start sending out alot of emails which end up getting marked as spam (Which is likly, even if the users said they wated to recieve email) your IP address could end up in trouble. I wouldnt fancy doing this on my webserver.
As for specs, your bandwidth is going to be hit hard when you start doing somthing like this. Although I think you would be suprised at how fast you can send out emails (Mail Server Dependant)
Either way when it comes to sending mass mail campaigns out, I'd speak to a third party if its somthing you are serious about, they have the knowledge and hardware to perform campaigns that are not going to damage your IP rep.
I think this issue all depends on your mail server. I've been in a project where we sent out 50-100K emails in a few hours and the main limitation was the capacity of the mail server. In my opinion it would be fine to have the mail client run on the web server as it won't be using too much resources if well written, the main question is what the mail server can take.
You can help your reputation on some email providers by contacting them and or filling out bulk mail registrations.
This is for yahoo:
http://help.yahoo.com/l/us/yahoo/mail/postmaster/bulkv2.html
I would not recommend using exchange for anything like this. You will want a separate bulk mail server like Ironport.
http://www.ironport.com/
Otherwise your internal emails will get stuck in the queue with all of your bulk emails which you definitely don't want.
The actual code involved does not have to be that complex. In my case we used an asynchronous task to loop over an email list table in the DB pulling out a batch of 20 or so at a time sending then sleeping for a number of seconds. Email providers do not like receiving hundreds/thousands of emails from a source at one time. Even better if you can stagger the list by providers.