Amazon SES Stops Working - asp.net

I set up Amazon SES and it initially worked for a few hours, then all of a sudden stopped. All of the e-mails I'm sending as, and our domain, have been verified. We are no sending bulk e-mails - only a few hundred per day. Whenever I make changes to the web.config it seems to allow it work again for another 2-3 hours. For example, It stopped working so I switched port 587 to 25, and it began working for 2-3 hours, then stopped. Then I switched back to 587 and the same thing happened. Once it stops working it doesn't ever seem to start again on its own. It is running on two load balanced servers, asp.net framework v2.0, IIS 7.5. Here is the code I'm using:
web.config:
<system.net>
<mailSettings>
<smtp deliveryMethod="Network" from="no-reply#ourdomain.com">
<network defaultCredentials="false" host="email-smtp.us-east-1.amazonaws.com" userName="***" password="***" port="587" />
</smtp>
</mailSettings>
</system.net>
c# code:
var smtpClient = new SmtpClient() { EnableSsl = true };
var mailMessage =
new MailMessage(fromAddress, toAddress, subject, body)
{
IsBodyHtml = true
};
smtpClient.Send(mailMessage);
Here are the two errors I've been receiving:
The following exception was thrown by the web event provider '(null)' in the application '/' (in an application lifetime a maximum of one exception will be logged per provider instance):
System.Web.HttpException: Unable to send out an e-mail to the SMTP server. Please ensure that the server specified in the <smtpMail> section is valid. ---> System.Net.Mail.SmtpException: Failure sending mail. ---> System.IO.IOException: Received an unexpected EOF or 0 bytes from the transport stream.
at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)
at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
at System.Threading.ExecutionContext.runTryCode(Object userData)
at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result)
at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.Mail.SmtpConnection.Flush()
at System.Net.Mail.ReadLinesCommand.Send(SmtpConnection conn)
at System.Net.Mail.SmtpConnection.GetConnection(String host, Int32 port)
at System.Net.Mail.SmtpClient.Send(MailMessage message)
--- End of inner exception stack trace ---
at System.Net.Mail.SmtpClient.Send(MailMessage message)
at System.Web.Management.MailWebEventProvider.SendMail(MailMessage msg)
--- End of inner exception stack trace ---
at System.Web.Management.MailWebEventProvider.SendMail(MailMessage msg)
at System.Web.Management.SimpleMailWebEventProvider.SendMessageInternal(WebBaseEventCollection events, Int32 notificationSequence, Int32 begin, DateTime lastFlush, Int32 discardedSinceLastFlush, Int32 eventsInBuffer, Int32 messageSequence, Int32 messagesInNotification, Int32 eventsInNotification, Int32 eventsLostDueToMessageLimit)
at System.Web.Management.SimpleMailWebEventProvider.SendMessage(WebBaseEvent eventRaised)
at *****.Global.SimpleMailWithSslWebEventProvider.ProcessEvent(WebBaseEvent raisedEvent)
at System.Web.Management.WebBaseEvent.RaiseInternal(WebBaseEvent eventRaised, ArrayList firingRuleInfos, Int32 index0, Int32 index1)
The following exception was thrown by the web event provider '(null)' in the application '/' (in an application lifetime a maximum of one exception will be logged per provider instance):
System.Web.HttpException: Unable to send out an e-mail to the SMTP server. Please ensure that the server specified in the <smtpMail> section is valid. ---> System.Net.Mail.SmtpException: Service not available, closing transmission channel. The server response was: Timeout waiting for data from client.
at System.Net.Mail.MailCommand.CheckResponse(SmtpStatusCode statusCode, String response)
at System.Net.Mail.SmtpTransport.SendMail(MailAddress sender, MailAddressCollection recipients, String deliveryNotify, SmtpFailedRecipientException& exception)
at System.Net.Mail.SmtpClient.Send(MailMessage message)
at System.Web.Management.MailWebEventProvider.SendMail(MailMessage msg)
--- End of inner exception stack trace ---
at System.Web.Management.MailWebEventProvider.SendMail(MailMessage msg)
at System.Web.Management.SimpleMailWebEventProvider.SendMessageInternal(WebBaseEventCollection events, Int32 notificationSequence, Int32 begin, DateTime lastFlush, Int32 discardedSinceLastFlush, Int32 eventsInBuffer, Int32 messageSequence, Int32 messagesInNotification, Int32 eventsInNotification, Int32 eventsLostDueToMessageLimit)
at System.Web.Management.SimpleMailWebEventProvider.SendMessage(WebBaseEvent eventRaised)
at *****.Global.SimpleMailWithSslWebEventProvider.ProcessEvent(WebBaseEvent raisedEvent)
at System.Web.Management.WebBaseEvent.RaiseInternal(WebBaseEvent eventRaised, ArrayList firingRuleInfos, Int32 index0, Int32 index1)
I tried to get help on the Amazon forums but haven't had any luck. It seems like something is interfering with the connection but I don't know what. Any ideas? Thanks.

This sounds like a tricky issue indeed - I'm not 100% positive on this, but you seem to miss disposing the SMTP client, which is mentioned in Sending email is often very slow after changing to VPC instance to cause the very error you are seeing:
When I do not dispose of the SmtpClient, I get the error...
'Service not available, closing transmission channel. The server
response was: Timeout waiting for data from client.'
This would actually be a good explanation for intermittent problems, i.e. depending on the executed code path this issue could trigger the SMTP connection to Amazon SES to hang, which is obviously remedied by switching to another port, implying the connection being reset - that's what the SmtpClient.Dispose Method ensures as well:
Sends a QUIT message to the SMTP server, gracefully ends the TCP
connection, and releases all resources used by the current instance of
the SmtpClient class.
Accordingly, an appropriate pattern would be to facilitate a using Statement, as demonstrated in Getting Started with Amazon SES and .NET:
String username = "SMTP-USERNAME"; // Replace with your SMTP username.
String password = "SMTP-PASSWORD"; // Replace with your SMTP password.
String host = "email-smtp.us-east-1.amazonaws.com";
int port = 25;
using (var client = new System.Net.Mail.SmtpClient(host, port))
{
client.Credentials = new System.Net.NetworkCredential(username, password);
client.EnableSsl = true;
client.Send
(
"FROM#EXAMPLE.COM", // Replace with the sender address.
"TO#EXAMPLE.COM", // Replace with the recipient address.
"Testing Amazon SES through SMTP",
"This email was delivered through Amazon SES via the SMTP end point."
);
}
Please note that the sample is using port 25, which I highly recommend to avoid due to the usually implied sending restrictions, see Addendum below for details regarding the respective Amazon EC2 throttling.
Good luck!
Addendum
Amazon EC2 throttling on port 25
You are probably aware of this (and I actually don't think it's an issue here), but Amazon EC2 imposes default sending limits on email sent via port 25 and throttles outbound connections if you attempt to exceed those limits which still applies when using Amazon SES, see Amazon SES SMTP Issues:
You are sending to Amazon SES from an Amazon EC2 instance via port 25
and you cannot reach your Amazon SES sending limits or you are
receiving time outs — Amazon SES EC2 imposes default sending limits on
email sent via port 25 and throttles outbound connections if you
attempt to exceed those limits. To remove these limits, submit a
Request to Remove Email Sending Limitations. You can also connect to
Amazon SES via port 465 or port 587, neither of which is throttled.
Consequently I'd remove port 25 out of your testing/switching scenario and use ports 465/587 only instead in order to avoid false leads (as quoted you could also request to get this limitation removed though, but it will take a few hours and port 25 seems to be best avoided in the first place) - it's a bit unfortunate that several official Amazon SES samples are using port 25 without even mentioning this easily triggered issue indeed.

Although this question is specific to .NET v2.0, I recently came across this issue on .NET v4.5, as well. Even after adding using around SMTP client (as suggested in earlier posts), I was still intermittently receiving this error. After many hours of research I came accross AWS's support for STARTTLS port (2587; http://docs.aws.amazon.com/ses/latest/DeveloperGuide/smtp-connect.html) and, in the very fine print, Microsoft states when "EnableSsl" is true on SMTP client the client expects a STARTTLS session (https://msdn.microsoft.com/en-us/library/system.net.mail.smtpclient.enablessl(v=vs.110).aspx). Thus, after updating my port to 2587 and surround the SmtpClient in a "using {}" I was able to successfully send ~20k emails without reproducing the error.
Thus, the working configuration would be:
<system.net>
<mailSettings>
<smtp deliveryMethod="Network">
<network host="email-smtp.us-west-2.amazonaws.com" port="2587" enableSsl="true" password="password" />
</smtp>
</mailSettings>
</system.net>
One disclaimer is that I still don't fully understand the true configuration of either AWS or .NET SMTP client, so this is still a bit circumstantial.

i am also using AWS ses service
smtp deliveryMethod="Network" from="XXXXXXX#ZXXX.com"
network enableSsl="true" host="email-smtp.us-east-1.amazonaws.com" userName="******" password="*********" port="25"
smtp
Hope this works for you.

Related

Measurement Protocol - System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive

I have an environmental issue with Measurement Protocol. I coded a server-side call to Google Analytics, sending an event using this as a guide:
https://gist.github.com/0liver/11229128
This works in 3 environments, but in the 4th, I get this error:
System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) --- End of inner exception stack trace --- at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead) --- End of inner exception stack trace --- at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context) at System.Net.HttpWebRequest.GetRequestStream() at ASP.abc_media_test_tlstest_aspx.GATrackEvent2(String type, String category, String action, String label, String cid, Nullable`1 value) at ASP.abc_media_test_tlstest_aspx.btnSubmit_Click(Object sender, EventArgs e)
I thought maybe it was a WAF issue, but it is not in blocking mode. I've also tried explicitly setting the TLS version. Nothing works. I've run out of ideas. Anyone got any?

VisualStudio cloud-based load test socket exception 100% of requests - webtest alone works OK

I am taking my first steps with visual studio enterprise load tests.
I have created a webtest following this tutorial: https://www.visualstudio.com/en-us/docs/test/performance-testing/run-performance-tests-app-before-release
The webtest works OK, hits my page properly and succeeds.
However when I have created a load test with this webtest, each request fails (regardless of the number of users, even with 2 users)
The error is
A connection attempt failed because the connected party did not
properly respond after a period of time, or established connection
failed because connected host has failed to respond 10.27.8.6:443
and the stack trace
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult) at
System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure,
Socket s4, Socket s6, Socket& socket, IPAddress& address,
ConnectSocketState state, IAsyncResult asyncResult, Exception&
exception)
The thing that potentially might be an issue is the fact that I have a self signed SSL certificate that obviously shows the certificate error warning. However, this is something I cannot change for now, so the tests will have to ignore this.
Also, I used https://www.webperformance.com/download/ tool to do these tests and it all worked OK with a relatively high intensity, so this cannot be a matter of poor performance of the site.
Any idea what could it be?
Not sure what more details can I give than below (as I said, for sure its not a matter of performance, because similar load test with different tool runs OK. It is also not a matter of webtest being wrong because it still passes when run as single test).
Webtest setup
Settings:
Loadtest setup
Loadtest settings

Random "The remote certificate is invalid according to the validation procedure." Exceptions

We have a web application that occasionally makes HTTPS requests to third party web services. Most of the time these work without issue. However occasionally the application will begin throwing exceptions on each request:
System.Net.WebException: The underlying connection was closed: Could not establish
trust relationship for the SSL/TLS secure channel. --->
System.Security.Authentication.AuthenticationException: The remote certificate is
invalid according to the validation procedure.
at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, Exception exception)
at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
...etc...
Now the odd part is that when we recycle the web application, the exception stops being thrown. For a while anyway.
We determined that a library was occasionally setting System.Net.ServicePointManager.ServerCertificateValidationCallback to a method which did not correctly validate certificates. The reason it seemed random is that certificate validation worked until the library was used and that callback was set.
The solution is to fix the library. And as general rule never globally set ServerCertificateValidationCallback in production code.

Relationship between HttpContext & HttpConnection

Does a HttpContext always have only one HttpConnection? When I execute a request, does it maintain only one HttpConnection even though there may be multiple socket connections due to redirects etc.?
HttpContext always refers to the current or the last connection used by the HTTP exchange regardless of how many intermediate connection may have been employed in the course of the exchange. Connection re-use and pooling is a responsibility of the connection manager.

Issue during rendering using C# in Compound Templating

A sudden issue appeared on one of the environments when publishing/rendering a certain piece of code in Compound Templating. It is an DTAP environment, and the issue is only happening on the Production environment (as always ;-), where there are two CMS machine and two publishers. All showing the same behavior.
The code which causes the problem is:
StructureGroup SG = (StructureGroup)engine.GetObject("/webdav/pub/root/etc/etc...");
The error in the Template builder is:
System.Net.WebException: The underlying connection was closed: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.77.66.136:80
bij System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
bij System.Net.Sockets.Socket.InternalConnect(EndPoint remoteEP)
bij System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Int32 timeout, Exception& exception)
--- Einde van intern uitzonderingsstackpad ---
bij System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)
bij System.Net.HttpWebRequest.GetRequestStream()
bij System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
bij Tridion.ContentManager.Templating.CompoundTemplates.DomainModel.Proxy.CompoundTemplateWebService.GetDebuggingState(String debuggerSessionId, String lastLogMessageId)
bij Tridion.ContentManager.Templating.CompoundTemplates.DomainModel.DebugObject.Start(Template template, Object debugItem, LoggingOptions loggingOptions)
What is odd that we are not really trying to get to an external server. At least, that's what it looks like. Maybe I am missing something in how Tridion works. I am fairly certain it is a security issue, but I don't know where to look.
I figured out the problem of the error with the help of Nickoli and Nuno's advice.
This error is shown when the Template Builder is trying to run from a location in a network, which doesn't have access to the code. I thought it was related to the engine.GetObject method, because the error would only appear when that method was run.
There was an error in the code in the end as well, but that had to do with a bad recursive method.

Resources