I am using OpenPop.net in my application. What this application does is that it downloads mails from a pop3 account saves all the attachments (CSV files) and processes them. This processing takes a lot of time. I am getting this exception which I am not able to figure out:
Exception message: OpenPop.Pop3.Exceptions.PopServerException: The stream used to retrieve responses from was closed
at OpenPop.Pop3.Pop3Client.IsOkResponse(String response)
at OpenPop.Pop3.Pop3Client.SendCommand(String command)
at OpenPop.Pop3.Pop3Client.DeleteMessage(Int32 messageNumber)
At the end of processing the CSV, the mails are deleted from the pop3 account. I believe this is where this exception is happening.
You are really having two issues here.
One is that you are doing a lot of processing while being connected to the POP3 server. When you are idle too long, the server will simply disconnect you to save resources.
What you should do, is fetch one email, process the attachments and then reconnect to fetch the next. You could also fetch all the attachments and then process them offline.
Second, I guess you are connecting to a gmail account. Gmail has some weird characteristics. A thread tries to find these characteristics. One of them is that, when you have fetched an email, it will not be available in the next POP3 session with the server. You can connect using a special username, where you append recent: in front of your normal username. This will show you the emails received in the last 30 days, despite of having been showed to an earlier POP3 session.
Hope it helps.
It sounds like something is trying to read a stream that has already been closed. Are you handling the streams at all, or is this done completely internal to the API? If you are handling them at all, there is a chance you are closing the streams (this often happens if someone uses a StreamReader, most people don't realize that closing the StreamReader also closes the underlying stream).
Related
I would like to ask about Rebus Timeout Manager. I know we have Internal timeout manager and External timeout manager and I have been using Internal timeout manager for quite some time. And I have been sharing one timeout database (Sql Server) for all my endpoints.
I would like to know if this is correct.
Secondly I would like to know if I can also use one external Timeout Manager for all my endpoints.
My questions comes from the the fact that the information contained in the Timeouts table (id,due_time,headers,body) has no connection with the endpoint that sent a message to the timeout manager.
I just would like to get assurance.
Regards
You can definitely use the internal timeout manager like you're currently doing.
The MSSSQL-based timeout storage is safe to use concurrently from multiple instances, as it used some finely trimmed lock hints when reading due messages, thus preventing issues that could otherwise have happened due to concurrent access.
But it's also a valid (and often very sensible) approach to create a dedicated timeout manager and then configure all other Rebus instances to use that.
And you are absolutely right that the sender of the timeout is irrelevant. The recipient is determined when sending the timeout, so that
await bus.DeferLocal(TimeSpan.FromMinutes(2), "HELLO FROM THE PAST 🙂");
will send the string to the bus' own input queue, and
await bus.Defer(TimeSpan.FromMinutes(2), "HELLO FROM THE PAST 🙂");
will send the string to the queue mapped as the owner of string:
.Routing(r => r.TypeBased().Map<string>("string-owner"))
In both cases, the message will actually be sent to the timeout manager, which will read the rbs2-deferred-until and rbs2-defer-recipient headers and keep the message until it is due.
I have tried a lot of variations to return the response quickly to user and send email in background but failed. Here is one sample that I have tried,
ThreadPool.QueueUserWorkItem(callback =>
// It is important to put this line inside try catch otherwise it might crash the process
client.SendAsync(message, message)
);
When I put a break point on client.SendAsync, no respose will be sent to user until this line execute. So what is the proper way to send email in background without waiting for response.
Update: I think I got the issue, actually it's the VS debugger that stop the asp.net thread(suspend ) until you Step Through. The above code works for my scenario.
You could place the email in a Message Queue and process the actual sending of it in a separate service that reads from that Message Queue.
Depending upon the number and size of the emails, you might also consider writing them to a pickup folder as well. You can use a local IIS SMTP server to relay the mail through a smart host. In practice, this has worked very well for us.
Send Email Asynchronously with ASP.NET http://www.asp.net/web-forms/videos/how-do-i/how-do-i-send-email-asynchronously-with-aspnet
My asp.net (mvc2) web app webinar registration system sends emails with connection info. Currently we just loop thru the list of attendees and hit a System.Net.Mail.SendMail for each. We send mail via our GoogleApps account (non-premium).
My use case is to send < 50 emails at a time and the system's been working fine. On the occasions where the number pushes a 100 addresses I find error returns with the message:
Service not available, closing transmission channel. The server response was: 4.7.0 Try again later, closing connection. (MAIL) g31sm4823143ibh.4
Is this message specific to Google's servers (searched on the phrase - seems like 'no'). Is there a recognized 'best practice' for sending emails by code?
Could it be that gmail just notices that you're sending a lot of messages in a rapid rate indicating it to them you're a spammer (in their eyes)?
If you have hosting I suggest you talk with your provider and see what they can offer you or provide an email address with which you can send more messages, in a controlled way of course.
GMail definately has a sending limit.
I dont think its widely publicised, but it certainly sounds like you're hitting it.
Slightly old question for a followup, but to help others landing here, things have changed since then (and they constantly change):
A registered G Suite user can't relay more than 10,000 messages in a 24-hour period, and can't relay messages to more than 10,000 unique recipients per 24-hour period. Users exceeding either of these limits see the error "550 5.4.5 Daily SMTP relay limit exceeded for user."
https://support.google.com/a/answer/2956491#sendinglimitsforrelay
i have to write application for sending newsletter.
what is the best way to send newsletter thoundands of users?
My requirement is
Each mail is seprately as To :
Every mail has unique Unsubscribe link
Is is good to use SMTP mail class of .net?
I look aound may questions in so but can't decide which approcah i should go?
There are many suggestions
Multi threaded Windows service
Use Mail Server
Add thread.sleep(2000) between each send.
can anyone suggest good way to imepement this?
I would not recommend asp.net webpage to send, even if you do start it in a separate background thread. I would think you run the risk of the server recycling your process in the middle of the send, which would mess it up. You really need to write some kind of separate service or application to send your emails.
The simplest option would be to just create a quick and dirty console or windows form application.
Also logging is critical just like the other poster said. If it fails you want to know exactly what got sent out and where it stopped so that when you restart it you don't mail all the people who it did work for again. You want to be able to input the starting point for the send, so if you need to restart at number email #5000 you can.
The classes in System.Net.Mail namespace will work just fine for sending your mail.
One of the biggest problems will be finding a email host that will let you send so many emails. Most email hosts have throttling and sometime it changes depending upon server conditions so if the server is being heavily used then the email limits will be more restrictive, and you may only get to set 500 emails per hour.
We have a newsletter that goes out to around 20000 people as separate emails and we had to play around with the delay between emails until we found one that would work for our email host. We ended up with 1.2 sec between emails, so that might be a good starting point.
I think there are email hosts specialize in bulk mailings though so if you get one of those it might not be a problem.
Also if you host your own email this may not be a problem. And if you do host your own mail you will have the option of dropping the mail in the pickup directory and you could just dump it all in there as fast as you want, and let the email service pick it up at it's own pace.
EDIT: Here is the settings to add to the config file for setting the pickup directory
<system.net>
<mailSettings>
<smtp from="support#test.com" deliveryMethod="SpecifiedPickupDirectory" >
<specifiedPickupDirectory pickupDirectoryLocation="Z:\Path\To\Pickup"/>
</smtp>
</mailSettings>
</system.net>
Definitely do not do this in ASP.NET. This is one of the biggest mistakes that new web developers make.
This needs to be a windows app or service that can handle this much volume.
I've written pages that send emails, but not nearly the volume yours will. Nonetheless, I would recommend the following based on code I have implemented in the past:
Use the web application to write out the email and all the recipient addresses to database table(s).
Have a process that is outside of ASP.NET actually send the emails. This could be a vbs file that is set up as a scheduled task, or (preferably) a windows service. The process would take the text of the email, append the unsubscribe link, and once sent successfully flag the database record as sent. That way, if the send fails, it can try again later (the send process loops over all the records flagged as unsent).
If you need a log of what was sent and when, you just need to keep the sent records in the database tables. Otherwise, just delete the records once sent successfully.
IMHO sending emails within the ASP.NET worker process is a bad idea because you don't know how long it will take and if the send fails there's little opportunity to retry before the page times out.
Create a webpage to "Design" the newsletter in. When they hit Send, queue the newsletter up somewhere (database) and use another program (windows service, etc) to send the queued letter. This will be many times more effecient and potentially fault tolerant if designed properly.
I have written a Newsletter module (as part of a bigger system) in ASPNET MVC 2, Entity Framework and using the System.Net.Mail namespace. It is kicked off in view and actually just runs in a controller with a supporting method to do the send. As each email is sent I track whether there is a hard bouce (an exception is thrown) and I update that database record stating a fail with the exception, otherwise I update the record stating success. We also do personalisation so we have 'tags' that get replaced by an extra field in the database (stored as XML for flexibility). This helps handle an unsubscribe function.
My code is quite simple (please don't flame me for using exception handling as business logic ;) and it works like a charm.
This is all done on a VPS at http://maximumasp.com which also hosts 4 sites with pretty decent traffic. We use their SMTP servers. We notified them that we needed this service and have had no problems relationship-wise.
We had 2GB of RAM on the machine running Windows 2008 and it was doing 6 emails/sec. We bumped it up to 3GB as the web sites needed it and now the mailout is doing about 20emails/sec. Our mailouts range from 2,000 to 100,000 email addresses.
In short, ASP.NET can be used to handle a mailout, and if you add in some logic to handle record updating the worry of losing your way mid-send is mitigated. Yes there are probably slicker ways to do this. We are looking in to MQMS and threading, and separating that out to windows service to make it more stable and scalable as we put more clients and larger lists on, but for now it works just fine with reasonable reporting and error handling.
I am writing an application that will need to send a massive amount of emails to our students who will be selected from our database (each email will be personalized to the extent that will include their name, course of study etc...so needs to be sent one at a time).
I could do this looping over an SmtpClient, but I'm afraid that with the numbers I'm trying to send, I'll ultimately run into timeout issues or my thread being killed because of lack of machine resources.
At this point I'm just looking for suggestions of a better way to handle this, or if looping over SmtpClient is an ok solution, how I should go about handling it to prevent what I posted above.
Would a web service be a better alternative?
Please advise,
TIA
My suggestion would be to batch this out. Do not try to run ASP.NET code to create 20K emails and send them out as you are bound to runinto timeout and performance issues. Instead populate a table with recipient, subject, body and begin a batch process from a windows service. This way your code is executed in a way where lag can be managed, instead of having a web page wait for the request to return.
Ok, first - this is hardly massive, I have been handling 50.000 emails+
Let the emails be written to a FOLDER - directory. Then use the local SMTP service you can install, pointing it at the folder as pickup directory. This will at least make sure you have a decent buffer in between (i.e. you can finish the asp.net side, while the emails are not sent at that point).
What about using Database Mail tool provided by SQL Server itself. You can still use asp.net but Email will be sent by SQL Server and all you have to do call the stored procedure and leave things to SQL Server.
I wouldn't recommend batch option since there won't be any optimization or queue thing unless you have to put a lot of effort to do so.
Database Mail tool provides queue and priority options as well. If a problem occurs, the e-mail will be queued again and sent later but the other ones will be still being sent.
It is called Database Mail in SQL Server 2008 and SQL Mail in previous versions.
For more information, please check the links below :
http://blog.sqlauthority.com/2008/08/23/sql-server-2008-configure-database-mail-send-email-from-sql-database/
http://msdn.microsoft.com/en-us/library/ms175887.aspx
You could use javascript on a timer to request the script which sends mail in small chunks that wont time out. Then you'd call the script from the browser. Add a progress bar, authentication, etc.
You don't need a web service, but rather a way to batch the emails into a queue (perhaps an "Email_Queue" table in the DB). Then a Windows service running on a server could read through the queue in a first-in/first-out basis at reasonable chunks at a time being sent to the SMTPClient, removing items from the queue as they are processed. You would probably need to run some measurements to determine what the chunk size and delay would be for your mail server.