HTTP requests stops working after some time - http

I have strange problem with my Game written in Unity (Mono). I have login functionality and then, after successfull login i'm sending some kind of keep-alive requests to ensure token is updated (I'm sending it every 30s).
The problem is, that after some time (sometimes it's 1 hour, sometimes 2.5 hours) all my requests have timeout status.
To be sure about my connection status I made some checks in code: I'm making simple GET to http://google.com and to main website of my API (not an API call. Just website GET). When I got timeout on API next time i figured out that:
After 1 timeout I have it always. Restarting application helps
I'm getting timeouts on API call and on GET request for base API website
Google is still responding with status 200 (no timeouts here)
Implementation:
On beginning i've been using RestSharp to handle requests, but the problem occured and there was a decision to throw RestSharp away and now we're using classic WebClient
class BetterWebClient : WebClient
{
private WebRequest _Request = null;
public TimeSpan? Timeout { get; set; }
protected override WebRequest GetWebRequest(Uri address)
{
this._Request = base.GetWebRequest(address);
if ( Timeout.HasValue )
{
_Request.Timeout = (int)Timeout.Value.TotalMilliseconds;
}
else
{
_Request.Timeout = 10*1000; //10s
}
if (this._Request is HttpWebRequest)
{
((HttpWebRequest)this._Request).AllowAutoRedirect = true;
}
return this._Request;
}
}
My HandleRequest function (which also calls Google and API website) looks like this:
public static void HandleRequest<TR>(string url, RestResponse<TR> executeGetRequest) where TR : new()
{
using (BetterWebClient w = new BetterWebClient() {Timeout = TimeSpan.FromSeconds(2)})
{
try
{
string downloadString = w.DownloadString("http://www.google.com");
Debug.Log("Google request: OK");
}
catch ( Exception )
{
Debug.LogError("Google request failed");
}
try
{
string downloadString = w.DownloadString("myAPIwebsite");
Debug.Log("WorldOfRescue.com request: OK");
}
catch ( Exception )
{
Debug.LogError("WorldOfRescue.com request failed");
}
}
var client = new BetterWebClient() {Timeout = TimeSpan.FromSeconds(5)};
client.Headers.Add("Accept", "application/json");
client.Headers.Add("Content-Type", "application/json");
try
{
string downloadString = client.DownloadString(url);
Debug.Log("Request for "+url+ " completed. Response: " + downloadString);
}
catch ( WebException e )
{
Debug.Log(e.Status);
throw;
}
catch ( Exception e )
{
Debug.Log(e.ToString());
throw;
}
finally
{
client.Dispose();
}
}
Do you have any idea why it happens? It looks like something is blocking me to send request to specific website, but in same time other website is working fine.

Related

SocketError when creating multiple HttpClient Connections with the Load balancer(F5)

This is the exact exception in the error logs.
I have a requirement where I should be creating a set of service calls and wait till all of them are completed successfully before moving on to do a different set of service calls.
I have two services Service1 and Service2.
There are 2 instances of Service2 in 2 different servers and we have set up an F5(Load Balancer) to distribute the load evenly.
Let's say I have 10 service calls to be made from Service1 to Service2 at a time and F5 will share those 10 calls among the 2 servers. i.e. 5 Calls to Service2 of each server.
But I observed that if any one of those 10calls is taking lot of time to complete the work it should do(The work to be done in Service2 has some heavy lifting) then i get a socket exception thrown and entire process gets stopped.
However when I dont use the F5 load balancer and just use 1 instance of the Service2. Then however long the process takes for any of those 10 calls it doesn't throw any exception.
I am not sure if this is an issue with F5 configuration or with the way connections are made with the F5 from .Net code.
Please go through the below code to get some idea of what i am trying to do and let me know if any code change would help me resolve it.
for (int i=0 ; i< ReqList.Count;i++)
{
maxTasks++;
ClassA reqList = new ClassA();
reqList = ReqList[i];
List<ClassA> recsByReq = recs.Where(x => x.ReqId == reqList.ReqtId).ToList();
ClassC service2Input = new ClassC();
service2Input.DetaiList = listofRecs;
service2Input.RecList = recsByReq;
taskList.Add(_service2.Service2MethodCall(service2Input, service2Resource));
if(maxTasks == 10 || ReqList.Count == 10 || i==ReqList.Count-1)
{
Task.WaitAll(taskList.ToArray(),-1);
maxTasks = 0;
taskList.Clear();
}
}
The Service2MethodCall is where I am creating a HttpClient to make connections with the 2nd service i.e. Service2,
public class Service2: IService2 {
private ServiceClient GetService2(string resource)
{
return new ServiceClient(_service2BaseUrl, TimeSpan.FromMinutes(60))
{
Resource = resource,
};
}
public async Task Service2MethodCall(ClassC service2Input, string resource)
{
try
{
var client = GetService2(resource);
await client.PostAsync(JsonConvert.SerializeObject(service2Input).ToString());
}
catch (Exception ex)
{
throw new Exception("The Service encountered an error during the Service2 call for Req ID : " +
service2Input.RecsList.Find(x => x.ReqId != "").ReqId.ToString(), ex);
}
}
}
The PostAsync() method creates a new HttpClient with HttpClientHandler object for each of the call.
public async Task PostAsync(object data) {
using(var httpClientHandler = new HttpClientHandler()) {
httpClientHandler.ServerCertificateCustomValidationCallback = (message, cert, chain, sslPolicyErrors) => {
if (sslPolicyErrors == SslPolicyErrors.None) {
return true; //Is valid
}
if (cert.GetCertHashString().ToUpper() == _acceptedThumbprint.ToUpper()) {
return true;
}
return false;
};
using(var client = new HttpClient(httpClientHandler)) {
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(ResponseFormatter.MediaType);
client.Timeout = Timeout.InfiniteTimeSpan;
Uri uri = BuildUri();
if (!String.IsNullOrWhiteSpace(SignatureKey)) {
client.DefaultRequestHeaders.TryAddWithoutValidation("Authorization",
RequestFormatter.GenerateAuthHeaderEncodedUserSignature(uri, AuthUserName, SignatureKey, data));
}
//_logger.WriteUsage(client.BaseAddress.ToString());
HttpResponseMessage response = await RequestFormatter.PostAsync(client, uri, data);
string content = await response.Content.ReadAsStringAsync();
try {
response.EnsureSuccessStatusCode();
}#
pragma warning disable CS0168 // The variable 'rex' is declared but never used
catch (HttpRequestException rex)# pragma warning restore CS0168 // The variable 'rex' is declared but never used
{
//_logger.WriteUsage("Response for POST to: {0} did not yield a successful status code. Message: {1}" + uri.ToString() + content);
throw new ApiHttpException(response.StatusCode, content);
}
}
}
}
Is there some thing I can do within the code to avoid this situation once and for all?

FCM throws 401 after some time for data message notification

I am sending data message notification from Java app server to the FCM rest endpoint. Everything works fine, data messages are received by the app without any issues, however after some time (without any noticeable trend) , FCM stars returning 401. I am using Apache common's HTTPClient library to make the http callss. This is the relevant code snippet
final HttpPost httpPost = new HttpPost("https://fcm.googleapis.com/v1/projects/proj1/messages:send");
httpPost.setHeader("Content-Type", "application/json");
httpPost.setHeader("Authorization", "Bearer "+ accessToken);
responseBody = httpclient.execute(httpPost, responseHandler);
And this snippet is for getting the access token for API authorization
static{
FileInputStream refreshToken = null;
refreshToken = new FileInputStream("C:/prj/proserviceaccoutkey.json");
googleCredentials=GoogleCredentials.fromStream(refreshToken).createScoped("https://www.googleapis.com/auth/firebase.messaging");
options = new FirebaseOptions.Builder() .setCredentials(googleCredentials).build();
}
// Gets called each time a data message needs to be sent
public static synchronized String getAccessToken()
{
if(googleCredentials.getAccessToken()==null)
try {
googleCredentials.refresh();
} catch (IOException e) {
e.printStackTrace();
}
return googleCredentials.getAccessToken().getTokenValue();
}
looks like googleCredentials.getAccessToken() will always return non-null, even when the cahce token in no longer valid, and this is why token was not getting refreshed in the code. Applied the following fix, and it's working now.
public static synchronized String getAccessToken()
{
if(googleCredentials!=null)
try {
googleCredentials.refresh();
} catch (IOException e) {
e.printStackTrace();
}
return googleCredentials.getAccessToken().getTokenValue();
}
Though, it's not really utilizing the cached token, as every time it will be refreshing the token, but my issue has been rsolved for now.

How to handle timeout of AsynchronousResponse object in RestEasy

i am implementing a jax-rs service with RestEasy on JBoss AS 7.1.2 an i would like to use asynchronous HTTP processsing as described here: http://docs.jboss.org/resteasy/docs/1.0.0.GA/userguide/html/Asynchronous_HTTP_Request_Processing.html
For thr AsynchronousResponse I define a timeout of 10 seconds. When this period expires, the request is responded with a 200 OK and an empty body. I would like to modify this behaviour so i need to be notified about the timeout event.
In my solution, I would like to handle the timeout event in a NotificationManager object, which keeps the AsycnhronousResponse for the time being. Please see the code below for details.
So far, i could not figure out how to do that. Does anyone have more experience with the RestEasy Asynchronous HTTP processing?
#POST
#Path("/blabla")
#Produces({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
#Consumes({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
public void subscribeLongPolling (
final #Suspend(10000) AsynchronousResponse response,
JAXBElement<LongPollingRequestParameters> rqParam,
#Context HttpServletRequest req) throws Exception {
//do some stuff with req
Thread t = new Thread("ThreadSubscribeTo:" + channelID)
{
#Override
public void run() {
//hand over to Notification Manager to return notifications in case some exist
try {
NotificationManager nm = new NotificationManager();
nm.setAsyncResponseObject(response);
logger.info("Response object registered in NotificationManager");
}catch (Exception e){
e.printStackTrace();
}
}
};
t.start();
logger.info("Releasing Thread");
}
public class NotificationManager {
private AsynchronousResponse response;
private NotificationList nList;
public synchronized void setAsyncResponseObject(AsynchronousResponse response) {
this.response = response;
if (nList.getAny().size() > 0) {
logger.info("Stored notification send to web client: " + nList.getAny().get(0).toString());
sendNotification(nList.getAny().remove(0));
}
}
public synchronized void sendNotification(Object message){
if (response != null){
logger.info("Response object found. Send notification immediately: " + message.toString());
Response responseObject = Response.ok(message, MediaType.APPLICATION_XML).build();
response.setResponse(responseObject);
response = null;
}else{
logger.info("Response object not found notification will be stored");
addNotification(message);
}
}
}
Thanks in advance,
Alex

Is there a notification when ASP.NET Web API completes sending to the client

I'm using Web API to stream large files to clients, but I'd like to log if the download was successful or not. That is, if the server sent the entire content of the file.
Is there some way to get a a callback or event when the HttpResponseMessage completes sending data?
Perhaps something like this:
var stream = GetMyStream();
var response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContent(stream);
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
// This doesn't exist, but it illustrates what I'm trying to do.
response.OnComplete(context =>
{
if (context.Success)
Log.Info("File downloaded successfully.");
else
Log.Warn("File download was terminated by client.");
});
EDIT: I've now tested this using a real connection (via fiddler).
I inherited StreamContent and added my own OnComplete action which checks for an exception:
public class StreamContentWithCompletion : StreamContent
{
public StreamContentWithCompletion(Stream stream) : base (stream) { }
public StreamContentWithCompletion(Stream stream, Action<Exception> onComplete) : base(stream)
{
this.OnComplete = onComplete;
}
public Action<Exception> OnComplete { get; set; }
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
var t = base.SerializeToStreamAsync(stream, context);
t.ContinueWith(x =>
{
if (this.OnComplete != null)
{
// The task will be in a faulted state if something went wrong.
// I observed the following exception when I aborted the fiddler session:
// 'System.Web.HttpException (0x800704CD): The remote host closed the connection.'
if (x.IsFaulted)
this.OnComplete(x.Exception.GetBaseException());
else
this.OnComplete(null);
}
}, TaskContinuationOptions.ExecuteSynchronously);
return t;
}
}
Then I use it like so:
var stream = GetMyStream();
var response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContentWithCompletion(stream, ex =>
{
if (ex == null)
Log.Info("File downloaded successfully.");
else
Log.Warn("File download was terminated by client.");
});
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
return response;
I am not sure if there is direct signaling that all is ok, but you can use a trick to find out that the connection is exist just before you end it up, and right after you fully send the file.
For example the Response.IsClientConnected is return true if the client is still connected, so you can check something like:
// send the file, make a flush
Response.Flush();
// and now the file is fully sended check if the client is still connected
if(Response.IsClientConnected)
{
// log that all looks ok until the last byte.
}
else
{
// the client is not connected, so maybe have lost some data
}
// and now close the connection.
Response.End();
if the server sent the entire content of the file
Actually there is nothing to do :)
This might sound very simplistic but you will know if an exception is raised - if you care about server delivering and not client cancelling halfway. IsClientConnected is based on ASP.NET HttpResponse not the WebApi.

Why am I getting "(304) Not Modified" error on some links when using HttpWebRequest?

Any ideas why on some links that I try to access using HttpWebRequest I am getting "The remote server returned an error: (304) Not Modified." in the code?
The code I'm using is from Jeff's post here (the page seems to have disappeared, see an archive copy at the Wayback Machine).
Note the concept of the code is a simple proxy server, so I'm pointing my browser at this locally running piece of code, which gets my browsers request, and then proxies it on by creating a new HttpWebRequest, as you'll see in the code. It works great for most sites/links, but for some this error comes up. You will see one key bit in the code is where it seems to copy the http header settings from the browser request to it's request out to the site, and it copies in the header attributes. Not sure if the issue is something to do with how it mimics this aspect of the request and then what happens as the result comes back?
case "If-Modified-Since":
request.IfModifiedSince = DateTime.Parse(listenerContext.Request.Headers[key]);
break;
I get the issue for example from http://en.wikipedia.org/wiki/Main_Page
PS. UPDATE HERE
Still can't work this out. Basically I can identify 1 link which has an issue, and it seems to work fine, 2nd time it gets the error, 3rd time OK, 4th time gets the error, 5th time OK etc. As if there is some state not getting cleared or something in the code. I've tried to clean up the code a bit using "using" type statements etc.
Here's the code. If anyone can spot why every 2nd time I browse to a link like http://newsimg.bbc.co.uk/css/screen/1_0_16/nol/v4/story.css (starting at the 2nd time, not the first) via this proxy code I get the error I'd love to hear.
class Program
{
static void Main(string[] args)
{
Proxy p = new Proxy(8080);
Thread proxythread = new Thread(new ThreadStart(p.Start));
proxythread.Start();
Console.WriteLine("Proxy Started. Press Any Key To Stop...");
Console.ReadKey();
p.Stop();
}
}
public class Proxy
{
private HttpListener _listener;
private int _port;
public Proxy(int port)
{
int defaultport = 8080;
// Setup Thread Pool
System.Threading.ThreadPool.SetMaxThreads(50, 1000);
System.Threading.ThreadPool.SetMinThreads(50, 50);
// Sanitize Port Number
if (port < 1024 || port > 65535)
port = defaultport;
// Create HttpListener Prefix
string prefix = string.Format("http://*:{0}/", port);
_port = port;
// Create HttpListener
_listener = new HttpListener();
_listener.Prefixes.Add(prefix);
}
public void Start()
{
_listener.Start();
while (true)
{
HttpListenerContext request = null;
try
{
request = _listener.GetContext();
// Statistics (by Greg)
int availThreads = -1;
int compPortThreads = -1;
ThreadPool.GetAvailableThreads(out availThreads, out compPortThreads);
log("INFO", request.Request.Url.ToString(), "START - [" + availThreads + "]");
ThreadPool.QueueUserWorkItem(ProcessRequest, request);
}
catch (HttpListenerException ex)
{
log("ERROR", "NA", "INFO: HttpListenerException - " + ex.Message);
break;
}
catch (InvalidOperationException ex)
{
log("ERROR", "NA", "INFO: InvalidOperationException - " + ex.Message);
break;
}
}
}
public void Stop()
{
_listener.Stop();
}
private void log(string sev, string uri, string message)
{
Console.Out.WriteLine(Process.GetCurrentProcess().Id + " - " + sev + " (" + uri + "): " + message);
}
private void ProcessRequest(object _listenerContext)
{
#region local variables
HttpWebRequest psRequest; // Request to send to remote web server
HttpWebResponse psResponse; // Response from remote web server
List<byte> requestBody = new List<byte>(); // Byte array to hold the request's body
List<byte> responseBody = new List<byte>(); // Byte array to hold the response's body
byte[] buffer;
string uri = "";
#endregion
var listenerContext = (HttpListenerContext)_listenerContext;
uri = listenerContext.Request.Url.ToString().Replace(string.Format(":{0}", _port), "");
// Create Interent Request
HttpWebRequest internetRequest = (HttpWebRequest)WebRequest.Create(uri);
#region Build Request Up
internetRequest.Method = listenerContext.Request.HttpMethod;
internetRequest.ProtocolVersion = listenerContext.Request.ProtocolVersion;
internetRequest.UserAgent = listenerContext.Request.UserAgent;
foreach (string key in listenerContext.Request.Headers.AllKeys)
{
try
{
switch (key)
{
case "Proxy-Connection":
case "Connection":
internetRequest.KeepAlive = (listenerContext.Request.Headers[key].ToLower() == "keep-alive") ? true : false;
break;
case "Content-Length":
internetRequest.ContentLength = listenerContext.Request.ContentLength64;
break;
case "Content-Type":
internetRequest.ContentType = listenerContext.Request.ContentType;
break;
case "Accept":
internetRequest.Accept = listenerContext.Request.Headers[key];
break;
case "Host":
break;
case "Referer":
internetRequest.Referer = listenerContext.Request.Headers[key];
break;
case "If-Modified-Since":
internetRequest.IfModifiedSince = DateTime.Parse(listenerContext.Request.Headers[key]);
break;
default:
internetRequest.Headers.Add(key, listenerContext.Request.Headers[key]);
break;
}
}
catch (Exception ex)
{
Console.WriteLine("Error settup up psRequest object. Error = " + ex.Message + "\n" + ex.StackTrace);
}
}
#endregion
#region Copy content into request
buffer = new byte[1024];
using (Stream instream = listenerContext.Request.InputStream)
{
int incount = instream.Read(buffer, 0, buffer.Length);
while (incount > 0)
{
internetRequest.GetRequestStream().Write(buffer, 0, incount);
incount = instream.Read(buffer, 0, buffer.Length);
}
}
#endregion
// Get Internet Response
HttpWebResponse internetResponse = null;
try
{
using (internetResponse = (HttpWebResponse)internetRequest.GetResponse())
{
#region Configure Local Response Header Keys
foreach (string key in internetResponse.Headers.Keys)
{
try
{
switch (key)
{
case "Transfer-Encoding":
listenerContext.Response.SendChunked = (internetResponse.Headers[key].ToLower() == "chunked") ? true : false;
break;
case "Content-Length":
listenerContext.Response.ContentLength64 = internetResponse.ContentLength;
break;
case "Content-Type":
listenerContext.Response.ContentType = internetResponse.Headers[key];
break;
case "Keep-Alive":
listenerContext.Response.KeepAlive = true;
break;
default:
listenerContext.Response.Headers.Add(key, internetResponse.Headers[key]);
break;
}
}
catch (Exception ex)
{
log("ERROR", uri, "Error settup up listenerContext.Response objects. Error = " + ex.Message + "\n" + ex.StackTrace);
}
}
#endregion
try
{
// Transfer the body data from Internet Response to Internal Response
buffer = new byte[1024];
using (Stream inputStream = internetResponse.GetResponseStream())
{
int outcount = inputStream.Read(buffer, 0, buffer.Length);
while (outcount > 0)
{
listenerContext.Response.OutputStream.Write(buffer, 0, outcount);
outcount = inputStream.Read(buffer, 0, buffer.Length);
}
}
}
catch (Exception ex)
{
log("ERROR", uri, "Could not obtain response from URI: " + ex.Message);
}
finally
{
listenerContext.Response.OutputStream.Close();
}
}
}
catch (Exception ex)
{
//if (ex is InvalidOperationException ||
// ex is ProtocolViolationException ||
// ex is WebException)
//{
// log(uri, "Could not successfully get response: " + ex.GetType() + " - " + ex.Message);
// listenerContext.Response.Close();
// return;
//}
//else { throw; }
log("ERROR", uri, "Could not successfully get response: " + ex.GetType() + " - " + ex.Message);
listenerContext.Response.Close();
}
}
}
And here is an example of what I see - first hit is good, 2nd has error...
Proxy Started. Press Any Key To Stop...
2080 - INFO (http://newsimg.bbc.co.uk:8080/css/screen/1_0_16/nol/v4/story.css): START - [50]
2080 - INFO (http://newsimg.bbc.co.uk:8080/css/screen/1_0_16/nol/v4/story.css): START - [50]
2080 - ERROR (http://newsimg.bbc.co.uk/css/screen/1_0_16/nol/v4/story.css): Could not successfully get response: System.Net.WebException - The remote server returned an error: (304) Not Modified.
First, this is not an error. The 3xx denotes a redirection. The real errors are 4xx (client error) and 5xx (server error).
If a client gets a 304 Not Modified, then it's the client's responsibility to display the resouce in question from its own cache. In general, the proxy shouldn't worry about this. It's just the messenger.
This is intended behavior.
When you make an HTTP request, the server normally returns code 200 OK. If you set If-Modified-Since, the server may return 304 Not modified (and the response will not have the content). This is supposed to be your cue that the page has not been modified.
The authors of the class have foolishly decided that 304 should be treated as an error and throw an exception. Now you have to clean up after them by catching the exception every time you try to use If-Modified-Since.
Just pressing F5 is not always working.
why?
Because your ISP is also caching web data for you.
Solution: Force Refresh.
Force refresh your browser by pressing CTRL + F5 in Firefox or Chrome to clear ISP cache too, instead of just pressing F5
You then can see 200 response instead of 304 in the browser F12 developer tools network tab.
Another trick is to add question mark ? at the end of the URL string of the requested page:
http://localhost:52199/Customers/Create?
The question mark will ensure that the browser refresh the request without caching any previous requests.
Additionally in Visual Studio you can set the default browser to Chrome in Incognito mode to avoid cache issues while developing, by adding Chrome in Incognito mode as default browser, see the steps (self illustrated):
It is not an issue it is because of caching...
To overcome this add a timestamp to your endpoint call, e.g. axios.get('/api/products').
After timestamp it should be axios.get(/api/products?${Date.now()}.
It will resolve your 304 status code.
I think you have not installed these features. see below in picture.
I also suffered from this problem some days ago. After installing this feature then I solved it. If you have not installed this feature then installed it.
Install Process:
go to android studio
Tools
Android
SDK Manager
Appearance & Behavior
Android SDK

Resources