So I have to API's running on Kubernetes. One has a controller function as such:
string filePath = "/blobs/data/runsession/" + folderName;
if (!Directory.Exists(filePath))
return null;
var tempFile = Path.Combine(Path.GetTempPath(), guid.ToString());
logger.Info("GetTempPath()" + Path.GetTempPath());
logger.Info("Temp path is: " + tempFile);
ZipFile.CreateFromDirectory(filePath, (tempFile + ".zip"));
byte[] fileBytes = File.ReadAllBytes(tempFile + ".zip");
HelpMethods.GetNetworkTimeStamp("Stop", 2);
return fileBytes;
This controller method works fine and when I call it from postman it returns a file.
Now in the other API I want to call this function from one of my service classes as such:
public async Task<byte[]> DownloadRunSession(string foldername)
{
try
{
var request = new RestRequest($"blob/{foldername}")
{
Timeout = -1,
};
var response = await client.ExecuteGetAsync(request);
if (!response.IsSuccessful)
{
Console.WriteLine("File download failed");
Console.WriteLine(response.Content.ToString());
return null;
}
return response.RawBytes;
}catch(Exception e)
{
Console.WriteLine(e.Message);
Console.WriteLine(e.StackTrace);
return null;
}
Now the server logs on the API that sends the file Give me a status 200 but says "the application aborted the connection", and the API that is supposed to receive the file gets no response and the response variable is always "null".
Any one that has had a similar dilemma and knows how to solve it?
Related
This is the exact exception in the error logs.
I have a requirement where I should be creating a set of service calls and wait till all of them are completed successfully before moving on to do a different set of service calls.
I have two services Service1 and Service2.
There are 2 instances of Service2 in 2 different servers and we have set up an F5(Load Balancer) to distribute the load evenly.
Let's say I have 10 service calls to be made from Service1 to Service2 at a time and F5 will share those 10 calls among the 2 servers. i.e. 5 Calls to Service2 of each server.
But I observed that if any one of those 10calls is taking lot of time to complete the work it should do(The work to be done in Service2 has some heavy lifting) then i get a socket exception thrown and entire process gets stopped.
However when I dont use the F5 load balancer and just use 1 instance of the Service2. Then however long the process takes for any of those 10 calls it doesn't throw any exception.
I am not sure if this is an issue with F5 configuration or with the way connections are made with the F5 from .Net code.
Please go through the below code to get some idea of what i am trying to do and let me know if any code change would help me resolve it.
for (int i=0 ; i< ReqList.Count;i++)
{
maxTasks++;
ClassA reqList = new ClassA();
reqList = ReqList[i];
List<ClassA> recsByReq = recs.Where(x => x.ReqId == reqList.ReqtId).ToList();
ClassC service2Input = new ClassC();
service2Input.DetaiList = listofRecs;
service2Input.RecList = recsByReq;
taskList.Add(_service2.Service2MethodCall(service2Input, service2Resource));
if(maxTasks == 10 || ReqList.Count == 10 || i==ReqList.Count-1)
{
Task.WaitAll(taskList.ToArray(),-1);
maxTasks = 0;
taskList.Clear();
}
}
The Service2MethodCall is where I am creating a HttpClient to make connections with the 2nd service i.e. Service2,
public class Service2: IService2 {
private ServiceClient GetService2(string resource)
{
return new ServiceClient(_service2BaseUrl, TimeSpan.FromMinutes(60))
{
Resource = resource,
};
}
public async Task Service2MethodCall(ClassC service2Input, string resource)
{
try
{
var client = GetService2(resource);
await client.PostAsync(JsonConvert.SerializeObject(service2Input).ToString());
}
catch (Exception ex)
{
throw new Exception("The Service encountered an error during the Service2 call for Req ID : " +
service2Input.RecsList.Find(x => x.ReqId != "").ReqId.ToString(), ex);
}
}
}
The PostAsync() method creates a new HttpClient with HttpClientHandler object for each of the call.
public async Task PostAsync(object data) {
using(var httpClientHandler = new HttpClientHandler()) {
httpClientHandler.ServerCertificateCustomValidationCallback = (message, cert, chain, sslPolicyErrors) => {
if (sslPolicyErrors == SslPolicyErrors.None) {
return true; //Is valid
}
if (cert.GetCertHashString().ToUpper() == _acceptedThumbprint.ToUpper()) {
return true;
}
return false;
};
using(var client = new HttpClient(httpClientHandler)) {
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(ResponseFormatter.MediaType);
client.Timeout = Timeout.InfiniteTimeSpan;
Uri uri = BuildUri();
if (!String.IsNullOrWhiteSpace(SignatureKey)) {
client.DefaultRequestHeaders.TryAddWithoutValidation("Authorization",
RequestFormatter.GenerateAuthHeaderEncodedUserSignature(uri, AuthUserName, SignatureKey, data));
}
//_logger.WriteUsage(client.BaseAddress.ToString());
HttpResponseMessage response = await RequestFormatter.PostAsync(client, uri, data);
string content = await response.Content.ReadAsStringAsync();
try {
response.EnsureSuccessStatusCode();
}#
pragma warning disable CS0168 // The variable 'rex' is declared but never used
catch (HttpRequestException rex)# pragma warning restore CS0168 // The variable 'rex' is declared but never used
{
//_logger.WriteUsage("Response for POST to: {0} did not yield a successful status code. Message: {1}" + uri.ToString() + content);
throw new ApiHttpException(response.StatusCode, content);
}
}
}
}
Is there some thing I can do within the code to avoid this situation once and for all?
I am currently uploading a file via the kendo fileuploader to an api controller using ASP.NET core RC-1. I am receiving a periodic error of "object reference not set to instance of object" when attempting to read the stream following opening the stream with IFormFile.OpenReadStream().
My controller is:
[HttpPost]
[Route("api/{domain}/[controller]")]
public async Task<IActionResult> Post([FromRoute]string domain, [FromForm]IFormFile file, [FromForm]WebDocument document)
{
if (ModelState.IsValid)
{
if (file.Length > 0)
{
var userName =
Request.HttpContext.User.Claims
.FirstOrDefault(c => c.Type == ClaimTypesEx.FullName)?
.Value;
var uploadedFileName =
ContentDispositionHeaderValue.Parse(file.ContentDisposition).FileName.Trim('"');
document.Domain = domain;
document.MimeType = file.ContentType;
document.SizeInBytes = file.Length;
document.ChangedBy = userName;
document.FileName = (string.IsNullOrEmpty(document.FileName)) ? uploadedFileName : document.FileName;
try
{
document = await CommandStack.For<WebDocument>()
.AddOrUpdateAsync(document, file.OpenReadStream()).ConfigureAwait(false);
}
catch (Exception e)
{
return new HttpStatusCodeResult(500);
}
return Ok(document);
}
}
return new BadRequestResult();
}
And the error is being thrown when I actually try to read the stream when it is going into blob storage:
public async Task<Uri> CreateOrUpdateBlobAsync(string containerName, string fileName, string mimeType,
Stream fileStream)
{
var container = Client.GetContainerReference(containerName);
var blob = container.GetBlockBlobReference(fileName);
//Error HERE
await blob.UploadFromStreamAsync(fileStream);
blob.Properties.ContentType = mimeType;
await blob.SetPropertiesAsync();
return blob.Uri;
}
What I am having trouble with is this is sporadic and there seems to be no defined pattern of which files are accepted and which ones generate the error. At first I thought it might be a size issue but that is not the case as I have several larger files uploaded successfully and then one small file will throw the error. Images seem to work fine and it is hit or miss on other file types with no rhyme or reason that I can figure out.
The problem is this:
on the server have a video file;
The administrator runs it on the play (video broadcast begins);
user is connected to the server - must be given to the video stream that is currently playing. A live webcast in real time.
To implement this task, I took as a basis for the article:
http://www.strathweb.com/2013/01/asynchronously-streaming-video-with-asp-net-web-api/
It worked. To give the stream a video file independently and in parallel.
I was looking on.
Next it was necessary to solve the problem of broadcasting to multiple customers (paragraph 3 in the job). I took this article:
http://gigi.nullneuron.net/gigilabs/streaming-data-with-asp-net-web-api-and-pushcontentstream/
Since I have to give evidence in the video byte - I replaced the StreamWriter class to Stream.
It works for one of the first client.
I made a website Asp.Net WebForms + WebApi + HTML5.
Web page - to run a video manager and viewed by users.
WebApi gives the player for <video> (HTML5) video stream.
HTML5:
<video>
<source src="http://localhost:8080/SiteVideoStreaming/api/live/?filename=nameFile" />
</video>
WebApi controllers:
//Controllers
public class LiveController : ApiController
{
private static ConcurrentBag<Stream> clients; // List of clients who need to simultaneously deliver video data
static string fileName = "";
static LiveController()
{
clients = new ConcurrentBag<Stream>();
WriteToStream(); // The first call - start to play a video file
}
[HttpGet]
public HttpResponseMessage Subscribe(string filename)
{
fileName = HostingEnvironment.MapPath("~/Videos/") + filename;
var response = Request.CreateResponse();
response.Content = new PushStreamContent((a, b, c) => { OnStreamAvailable(a, b, c); }, "video/mp4");
return response;
}
private void OnStreamAvailable(Stream stream, HttpContent content, TransportContext context)
{
clients.Add(stream); // Add new client
}
//Class record a video file into a streams
public async static void WriteToStream()
{
var buffer = new byte[65536];
using (var video = File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.Read))
{
var length = (int)video.Length;
var bytesRead = 1;
while (length > 0 && bytesRead > 0)
{
bytesRead = video.Read(buffer, 0, Math.Min(length, buffer.Length));
foreach (var client in clients)// Each client in turn we return video data
{
try
{
await client.WriteAsync(buffer, 0, bytesRead); // ERROR - here !!! when you connect a second client
await client.FlushAsync();
}
catch (Exception ex)
{
Stream ignore;
clients.TryTake(out ignore);
}
}
length -= bytesRead;
}
}
}
}
If the request first came from one client - is given to video. Working.
If the request from the second client - when you try to start to give him a stream error occurs.
In this connection drops and the first client.
The error is as follows:
[System.Web.HttpException] = {"The remote host closed the connection.
The error code is 0x800704CD."}
As I understood after a search on the Internet is:
0x800704CD "An operation was attempted on a nonexistent network
connection."
Tell me that I'm not doing right?
Thank you.
I do so.
I use this controller:
public class VideoController : ApiController
{
// GET api/<controller>
public HttpResponseMessage Get(string filename)
{
if (filename == null)
return new HttpResponseMessage(HttpStatusCode.BadRequest);
string filePath = HostingEnvironment.MapPath("~/Videos/") + filename;
if (Request.Headers.Range != null)
{
//Range Specifc request: Stream video on wanted range.
try
{
//NOTE: ETag calculation only with file name is one approach (Not the best one though - GUIDs or DateTime is may required in live applications.).
Encoder stringEncoder = Encoding.UTF8.GetEncoder();
byte[] stringBytes = new byte[stringEncoder.GetByteCount(filePath.ToCharArray(), 0, filePath.Length, true)];
stringEncoder.GetBytes(filePath.ToCharArray(), 0, filePath.Length, stringBytes, 0, true);
MD5CryptoServiceProvider MD5Enc = new MD5CryptoServiceProvider();
string hash = BitConverter.ToString(MD5Enc.ComputeHash(stringBytes)).Replace("-", string.Empty);
HttpResponseMessage partialResponse = Request.CreateResponse(HttpStatusCode.PartialContent);
partialResponse.Headers.AcceptRanges.Add("bytes");
partialResponse.Headers.ETag = new EntityTagHeaderValue("\"" + hash + "\"");
var stream = new FileStream(filePath, FileMode.Open, FileAccess.Read);
partialResponse.Content = new ByteRangeStreamContent(stream, Request.Headers.Range, new MediaTypeHeaderValue("video/mp4"));
return partialResponse;
}
catch (Exception ex)
{
return new HttpResponseMessage(HttpStatusCode.InternalServerError);
}
}
else
{
return new HttpResponseMessage(HttpStatusCode.RequestedRangeNotSatisfiable);
}
}
}
On the client side - I run it <video> video player through technology SignalR.
I'm using Web API to stream large files to clients, but I'd like to log if the download was successful or not. That is, if the server sent the entire content of the file.
Is there some way to get a a callback or event when the HttpResponseMessage completes sending data?
Perhaps something like this:
var stream = GetMyStream();
var response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContent(stream);
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
// This doesn't exist, but it illustrates what I'm trying to do.
response.OnComplete(context =>
{
if (context.Success)
Log.Info("File downloaded successfully.");
else
Log.Warn("File download was terminated by client.");
});
EDIT: I've now tested this using a real connection (via fiddler).
I inherited StreamContent and added my own OnComplete action which checks for an exception:
public class StreamContentWithCompletion : StreamContent
{
public StreamContentWithCompletion(Stream stream) : base (stream) { }
public StreamContentWithCompletion(Stream stream, Action<Exception> onComplete) : base(stream)
{
this.OnComplete = onComplete;
}
public Action<Exception> OnComplete { get; set; }
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
var t = base.SerializeToStreamAsync(stream, context);
t.ContinueWith(x =>
{
if (this.OnComplete != null)
{
// The task will be in a faulted state if something went wrong.
// I observed the following exception when I aborted the fiddler session:
// 'System.Web.HttpException (0x800704CD): The remote host closed the connection.'
if (x.IsFaulted)
this.OnComplete(x.Exception.GetBaseException());
else
this.OnComplete(null);
}
}, TaskContinuationOptions.ExecuteSynchronously);
return t;
}
}
Then I use it like so:
var stream = GetMyStream();
var response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContentWithCompletion(stream, ex =>
{
if (ex == null)
Log.Info("File downloaded successfully.");
else
Log.Warn("File download was terminated by client.");
});
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
return response;
I am not sure if there is direct signaling that all is ok, but you can use a trick to find out that the connection is exist just before you end it up, and right after you fully send the file.
For example the Response.IsClientConnected is return true if the client is still connected, so you can check something like:
// send the file, make a flush
Response.Flush();
// and now the file is fully sended check if the client is still connected
if(Response.IsClientConnected)
{
// log that all looks ok until the last byte.
}
else
{
// the client is not connected, so maybe have lost some data
}
// and now close the connection.
Response.End();
if the server sent the entire content of the file
Actually there is nothing to do :)
This might sound very simplistic but you will know if an exception is raised - if you care about server delivering and not client cancelling halfway. IsClientConnected is based on ASP.NET HttpResponse not the WebApi.
I have been experimenting with WP7 apps today and have hit a bit of a wall.
I like to have seperation between the UI and the main app code but Ive hit a wall.
I have succesfully implemented a webclient request and gotten a result, but because the call is async I dont know how to pass this backup to the UI level. I cannot seem to hack in a wait for response to complete or anything.
I must be doing something wrong.
(this is the xbox360Voice library that I have for download on my website: http://www.jamesstuddart.co.uk/Projects/ASP.Net/Xbox_Feeds/ which I am porting to WP7 as a test)
here is the backend code snippet:
internal const string BaseUrlFormat = "http://www.360voice.com/api/gamertag-profile.asp?tag={0}";
internal static string ResponseXml { get; set; }
internal static WebClient Client = new WebClient();
public static XboxGamer? GetGamer(string gamerTag)
{
var url = string.Format(BaseUrlFormat, gamerTag);
var response = GetResponse(url, null, null);
return SerializeResponse(response);
}
internal static XboxGamer? SerializeResponse(string response)
{
if (string.IsNullOrEmpty(response))
{
return null;
}
var tempGamer = new XboxGamer();
var gamer = (XboxGamer)SerializationMethods.Deserialize(tempGamer, response);
return gamer;
}
internal static string GetResponse(string url, string userName, string password)
{
if (!string.IsNullOrEmpty(userName) && !string.IsNullOrEmpty(password))
{
Client.Credentials = new NetworkCredential(userName, password);
}
try
{
Client.DownloadStringCompleted += ClientDownloadStringCompleted;
Client.DownloadStringAsync(new Uri(url));
return ResponseXml;
}
catch (Exception ex)
{
return null;
}
}
internal static void ClientDownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
{
if (e.Error == null)
{
ResponseXml = e.Result;
}
}
and this is the front end code:
public void GetGamerDetails()
{
var xboxManager = XboxFactory.GetXboxManager("DarkV1p3r");
var xboxGamer = xboxManager.GetGamer();
if (xboxGamer.HasValue)
{
var profile = xboxGamer.Value.Profile[0];
imgAvatar.Source = new BitmapImage(new Uri(profile.ProfilePictureMiniUrl));
txtUserName.Text = profile.GamerTag;
txtGamerScore.Text = int.Parse(profile.GamerScore).ToString("G 0,000");
txtZone.Text = profile.PlayerZone;
}
else
{
txtUserName.Text = "Failed to load data";
}
}
Now I understand I need to place something in ClientDownloadStringCompleted but I am unsure what.
The problem you have is that as soon as an asynchronous operation is introduced in to the code path the entire code path needs to become asynchronous.
Because GetResponse calls DownloadStringAsync it must become asynchronous, it can't return a string, it can only do that on a callback
Because GetGamer calls GetResponse which is now asynchronous it can't return a XboxGamer, it can only do that on a callback
Because GetGamerDetails calls GetGamer which is now asynchronous it can't continue with its code following the call, it can only do that after it has received a call back from GetGamer.
Because GetGamerDetails is now asynchronous anything call it must also acknowledge this behaviour.
.... this continues all the way up to the top of the chain where a user event will have occured.
Here is some air code that knocks some asynchronicity in to the code.
public static void GetGamer(string gamerTag, Action<XboxGamer?> completed)
{
var url = string.Format(BaseUrlFormat, gamerTag);
var response = GetResponse(url, null, null, (response) =>
{
completed(SerializeResponse(response));
});
}
internal static string GetResponse(string url, string userName, string password, Action<string> completed)
{
WebClient client = new WebClient();
if (!string.IsNullOrEmpty(userName) && !string.IsNullOrEmpty(password))
{
client.Credentials = new NetworkCredential(userName, password);
}
try
{
client.DownloadStringCompleted += (s, args) =>
{
// Messy error handling needed here, out of scope
completed(args.Result);
};
client.DownloadStringAsync(new Uri(url));
}
catch
{
completed(null);
}
}
public void GetGamerDetails()
{
var xboxManager = XboxFactory.GetXboxManager("DarkV1p3r");
xboxManager.GetGamer( (xboxGamer) =>
{
// Need to move to the main UI thread.
Dispatcher.BeginInvoke(new Action<XboxGamer?>(DisplayGamerDetails), xboxGamer);
});
}
void DisplayGamerDetails(XboxGamer? xboxGamer)
{
if (xboxGamer.HasValue)
{
var profile = xboxGamer.Value.Profile[0];
imgAvatar.Source = new BitmapImage(new Uri(profile.ProfilePictureMiniUrl));
txtUserName.Text = profile.GamerTag;
txtGamerScore.Text = int.Parse(profile.GamerScore).ToString("G 0,000");
txtZone.Text = profile.PlayerZone;
}
else
{
txtUserName.Text = "Failed to load data";
}
}
As you can see async programming can get realy messy.
You generally have 2 options. Either you expose your backend code as an async API as well, or you need to wait for the call to complete in GetResponse.
Doing it the async way would mean starting the process one place, then return, and have the UI update when data is available. This is generally the preferred way, since calling a blocking method on the UI thread will make your app seem unresponsive as long as the method is running.
I think the "Silverlight Way" would be to use databinding. Your XboxGamer object should implement the INotifyPropertyChanged interface. When you call GetGamer() it returns immediately with an "empty" XboxGamer object (maybe with GamerTag=="Loading..." or something). In your ClientDownloadStringCompleted handler you should deserialize the returned XML and then fire the INotifyPropertyChanged.PropertyChanged event.
If you look at the "Windows Phone Databound Application" project template in the SDK, the ItemViewModel class is implemented this way.
Here is how you can expose asynchronous features to any type on WP7.