Hi people i'm trying to do a head request to get the content-length of the file (it's big >= 200MB) and simple just throw me an exception saying "Exception of type 'System.OutOfMemoryException' was thrown." since it's a HEAD method this shouldn't be a problem right?
Here is my code:
using (HttpClient client = new HttpClient())
{
using (var request = new HttpRequestMessage()
{
RequestUri = new Uri(streamLink.StreamLink),
Method = HttpMethod.Head
})
{
using (var response = await client.SendAsync(request))
{
var restatus = response.StatusCode == HttpStatusCode.OK;
if (restatus)
{
var filesize = ConvertBytesToMegabytes(response.Content.Headers.ContentLength);
CalculateStreamQuality(filesize, streamLink, runtime);
}
}
}
}
You need to call the SendAsync with the completion option parameter set to: HttpCompletionOption.ResponseHeadersRead - this will enable the SendAsync method to complete without trying to allocate a buffer to hold the (non-existent data payload).
Update your code to use the following:
using (var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead))
{
...
}
Related
Long story short, no matter what I try VeraCode continues to flag 8 lines of my code as flaws with CWE 918. This is old code so I'm not sure why it's suddenly being flagged.
Here's an example [offending] method with the flagged line in bold
public virtual async Task<HttpResponseMessage> Put(string controller = "", Dictionary<string, object> parameters = null, object body = null)
{
if (string.IsNullOrWhiteSpace(ApiBaseUrl)) return null;
HttpResponseMessage response = null;
using (var client = GetHttpClient())
{
client.BaseAddress = new Uri(ApiBaseUrl);
if (!string.IsNullOrEmpty(Token)) client.DefaultRequestHeaders.Add("Token-Key", Token);
if (!string.IsNullOrEmpty(DeviceId)) client.DefaultRequestHeaders.Add("DeviceId", DeviceId);
var url = GenerateUrl(controller, parameters);
var requestBody = GeneratedHttpContent(body);
if (requestBody == null) requestBody = new StringContent("");
**response = await client.PutAsync(url, requestBody);**
await LogError(response);
return response;
}
}
Here's my proposed fix that utilized an extension method to validate the URL
var url = GenerateUrl(controller, parameters);
var requestBody = GeneratedHttpContent(body);
if (requestBody == null) requestBody = new StringContent("");
**if (url.IsValidUrl())
{
response = await client.PutAsync(url, requestBody);
}
else
{
response = new HttpResponseMessage(HttpStatusCode.BadRequest);
}**
await LogError(response);
return response;
Here is the extension method with a VeraCode attribute
[RedirectUrlCleanser]
public static bool IsValidUrl(this string source)
{
return Uri.TryCreate(source, UriKind.RelativeOrAbsolute, out Uri uriResult) && Uri.IsWellFormedUriString(source, UriKind.RelativeOrAbsolute);
}
I can have VeraCode automatically mitigate based on the attribute, but our client will be performing their own scan and certainly won't have that setting enabled.
Any ideas on how I can resolve this would be appreciated.
The true source of the flaw is inside of your GenerateUrl method which is unfortunately not shown, but here is the general idea of what the Veracode is complaining about.
For CWE ID 918 it is hard to make Veracode recognize your fix unless you have static URL. You need to validate all your inputs that become parts of your request URL.
Below is what I found at the Veracode site:
https://community.veracode.com/s/question/0D52T00004i1UiSSAU/how-to-fix-cwe-918-veracode-flaw-on-webrequest-getresponce-method
The complete solution existed only for the case where you have single or some small number of possible input values (white list):
public WebResponse ProxyImage(string image_host, string image_path)
{
string validated_image_host = AllowedHosts.Host1;
if (image_host.Equals(AllowedHosts.Host2))
validated_image_host = AllowedHosts.Host2;
string validated_image = AllowedImages.Image1;
if (image_path.Equals(AllowedImages.Image2))
validated_image = AllowedImages.Image2;
string url = $"http://{validated_image_host}.example.com/{validated_image}";
return WebRequest.Create(url).GetResponse();
}
If the set of possible valid values is too large for that kind of validation then you need to fix the flaw by implementing dynamic validation of inputs using regular expressions. Unfortunately, Veracode is not smart enough to recognize that kind of fix, so "mitigation by design" is still required.
public WebResponse ProxyImage(string image_host, string image_path)
{
var image_host_regex = new System.Text.RegularExpressions.Regex("^[a-z]{1,10}$");
if (!image_host_regex.Match(image_host).Success)
throw new ArgumentException("Invalid image_host");
var image_path_regex = new System.Text.RegularExpressions.Regex("^/[a-z]{1,10}/[a-z]{1,255}.png$");
if (!image_path_regex.Match(image_path).Success)
throw new ArgumentException("Invalid image_host");
string url = $"http://{image_host}.example.com/{image_path}";
return WebRequest.Create(url).GetResponse();
}
Another way to fix this issue (which is kind of a hack) is to append your query string parameters in the baseAddress of the HttpClient, this way the veracode will not treat it like a flaw.
Here is how the solution would look like
public async Task<Data> GetData(string input)
{
try
{
var httpClient = new HttpClient();
//Appended the parameter in base address to
//to fix veracode flaw issue
httpClient.BaseAddress = new Uri($"https://someurl.com/somefunction/{input}");
//passing empty string in GetStringAsync to make sure
//veracode doesn't treat it like modifying url
var content = await httpClient.GetStringAsync("");
return JsonConvert.DeserializeObject<Data>(content);
}
}
I have built a REST API with a controller having a POST method with 4 parameters like this-
[HttpPost]
public void SaveSession([FromBody] string userId, [FromBody] DateTime issueDateTime, [FromBody] string browserType, [FromBody] string salt)
{
// Params need to be changed
_sessionService.SaveSession(userId, issueDateTime, browserType, salt);
}
How should I POST data on the client side, I mean what should be the format of the data to be sent?
I tried this format-
"userId=abc&DateTime=someDatetime&browserType=somebrowser&salt=somesalt"
Its not working if I try this, The web service method is not even being called
Could anyone tell me the correct format?
EDIT:
Here is how I am calling the API-
const string endPoint = #"http://localhost:85/session/Test";
var postData = "userId=abc&DateTime=someDatetime&browserType=somebrowser&salt=somesalt"
var request = (HttpWebRequest) WebRequest.Create(EndPoint + parameters);
request.Method = "POST";
request.ContentLength = 0;
request.ContentType = "application/x-www-form-urlencoded";
if (!string.IsNullOrEmpty(postData) && Method == HttpVerb.POST)
{
var encoding = new UTF8Encoding();
var bytes = Encoding.GetEncoding("iso-8859-1").GetBytes(postData);
request.ContentLength = bytes.Length;
using (var writeStream = request.GetRequestStream())
{
writeStream.Write(bytes, 0, bytes.Length);
}
}
using (var response = (HttpWebResponse) request.GetResponse())
{
var xmlDoc = new XmlDocument();
if (response.StatusCode != HttpStatusCode.OK)
{
var message = String.Format("Request failed. Received HTTP {0}", response.StatusCode);
throw new ApplicationException(message);
}
// grab the response
var responseStream = response.GetResponseStream();
if (responseStream != null)
{
xmlDoc.Load(responseStream);
}
return (xmlDoc);
}
Thanks!
I assume routing has been properly configured.
Said so.... DateTime parameter in the controller method has been named "issueDateTime" while within the request has been named "DateTime".
I got to know, what mistake I was doing. I was sending 4 parameters in a WebService method. We can only send one parameter while calling a web service method. If you want to send multiple data, just send it as an object. Like this -
[HttpPost]
public void SaveSession([FromBody] Values value)
{
var userId = values.userId,
var issueDateTime= values.issueDateTime,
var browserType= values.browserType,
var salt= values.salt,
_sessionService.SaveSession(userId, issueDateTime, browserType, salt);
}
I have been looking at MSDN and codeproject, but I am still a bit confused.
Synchronous Service vs an Asynchronous service.
I have a WCF service end point. This service has a 2way SSL applied to its web.config file.
The client end point is a Oracle based java Weblogic Suite. This has its own private key and public key. The client needs to communicate with our service both asynchronously and synchronously.
I CAN ONLY CHANGE THINGS ON THE SERVER SIDE
http://www.codeproject.com/Articles/91528/How-to-Call-WCF-Services-Synchronously-and-Asynchr
http://msdn.microsoft.com/en-us/library/ms731177.aspx
The following code is the synchronous part of the SVC, cs file:-
public getQuoteSyncResponse1 getQuoteSync(getQuoteSyncRequest request)
{
// Create new response
getQuoteSyncResponse1 res = new getQuoteSyncResponse1();
res.GetQuoteSyncResponse = new GetQuoteSyncResponse();
res.GetQuoteSyncResponse.Header = new GetQuoteResponseHeaderType();
res.GetQuoteSyncResponse.Response = new GetQuoteSyncResponseType();
// Create and populate header
res.GetQuoteSyncResponse.Header.MessageId = request.GetQuoteRequestSync.Header.MessageId;
res.GetQuoteSyncResponse.Header.Timestamp = request.GetQuoteRequestSync.Header.Timestamp;
res.GetQuoteSyncResponse.Header.QuoteId = request.GetQuoteRequestSync.Header.QuoteId;
res.GetQuoteSyncResponse.Header.CarrierId = request.GetQuoteRequestSync.Header.CarrierId;
List<RejectionType> rj = new List<RejectionType>();
string _sTotalEmployees = request.GetQuoteRequestSync.Request.Employer.TotalEmployees;
int _TotalEmployees = 0;
if (int.TryParse(_sTotalEmployees, out _TotalEmployees) == false)
{
RejectionType rt;
rt = new RejectionType();
rt.ReasonCode = "R01";
rt.ReasonDescription = "Invalid Number of Employees";
rj.Add(rt);
res.GetQuoteSyncResponse.Response.Rejections = rj.ToArray();
res.GetQuoteSyncResponse.Response.ReceiptStatus = AcceptanceContentType.Reject;
return res;
}
res.GetQuoteSyncResponse.Response.ReceiptStatus = AcceptanceContentType.Success;
List<QuoteType> q = new List<QuoteType>();
QuoteType qt;
qt = new QuoteType();
qt.PlanId = "P345678";
qt.EEPremium = 1220;
qt.EESPPremium = 2222;
qt.EEDepPremium = 3333;
qt.EEFamilyPremium = 4444;
qt.TotalMonthlyPremium = 3456;
qt.CoverageEffectiveDate = DateTime.Now;
q.Add(qt);
res.GetQuoteSyncResponse.Response.Quotes = q.ToArray();
return res;}
so this Synchronous part of the service is working. Now, how do I use this to transform it into the asynchronous equivalent?
Should I be starting the async method in the cs file? or in the svc file? I am confused...
public getQuoteAsyncResponse getQuoteAsync(getQuoteAsyncRequest request, AsyncCallback callback, Object state)
{
// Create new response
getQuoteAsyncResponse res = new getQuoteAsyncResponse();
return new getQuoteAsyncResponse();
}
I sort of understand about the callback deli-gator, object state and such, but can someone illustrate this further for me? How do I format the asynchronous part of the service? The web has so many examples... but all very confusing. I must have some inherent misunderstanding on this concept.
Edit:- I was told in the answer that, the server side needs no manipulation for async style of communication. However I found this:-
Implementing Asynchronous Operations in WCF
Just as the WCF plumbing enables clients to call server operations asynchronously, without the server needing to know anything about it, WCF also allows service operations to be defined asynchronously. So an operation like:
[OperationContract]
string DoWork(int value);
…might instead be expressed in the service contract as:
[OperationContract(AsyncPattern = true)]
IAsyncResult BeginDoWork(int value, AsyncCallback callback, object state);
string EndDoWork(IAsyncResult result);
Note that the two forms are equivalent, and indistinguishable in the WCF metadata: they both expose an operation called DoWork[1]:
The async part needs to be done in the client. This means you are probably doing something similar to:
var response = ServiceReference.GetSomething();
Instead, make a proxy to get the callback. Create and event (or delegate) that gets fired (or called) whenever the callback receives the response. In the above statement, you are obviously waiting for the response to be assigned into the variable before moving to the next line.
Instead, you could
On the Service contract, be sure to decorate with [OperationContract(IsOneWay = true)]
If you use ServiceReference or serviceutil, it will automatically create "incoming events" and do all the client side async work for you.
If you are using TCP, create an callback contract as well, then on client you can do something like
ServiceReference1.IncomingSomething += new eventHandler.
Now you can do ServiceReferecnce1.GetSomething(), and the response will go to the eventhandler function.
If this is RESTFUL:
public void MakeAsyncRequest(string url, string contentType)
{
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.ContentType = contentType;
request.Method = WebRequestMethods.Http.Get;
request.Timeout = 10000;
request.Proxy = null;
request.BeginGetResponse(new AsyncCallback(ReadCallback), request);
}
catch (Exception ex)
{
LogManager.LogException(ex);
}
}
private void ReadCallback(IAsyncResult asyncResult)
{
HttpWebRequest request = (HttpWebRequest)asyncResult.AsyncState;
string strContent = string.Empty;
string s;
try
{
using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asyncResult))
{
Stream responseStream = response.GetResponseStream();
using (StreamReader sr = new StreamReader(responseStream))
{
//Need to return this response
strContent = sr.ReadToEnd();
}
}
}
catch (Exception ex)
{
throw;
}
}
I created MassPayment using MassPaymentAPI But I got Error In this Method
Input For Methods :::
url="https://api.sandbox.paypal.com/nvp"
postdata="METHOD=MassPay&EMAILSUBJECT=You+have+money!&RECEIVERTYPE=EmailAddress&CURRENCYCODE=USD&L_EMAIL0=bhaumik50%40gmail.com&L_Amt0=1.00&L_UNIQUEID0=&L_NOTE0=&USER=rserasiya_api1.gmail.com&PWD=OneIsTheLonliestNumber&VERSION=1&SOURCE=1" ;
timeout= "3600" ;
X509certificate = "Certifcate Description"
public static string HttpPost(string url, string postData, int timeout, X509Certificate x509)
{
HttpWebRequest objRequest = (HttpWebRequest) WebRequest.Create(url);
objRequest.Timeout = timeout;
objRequest.Method = "POST";
objRequest.ContentLength = postData.Length;
if (null != x509)
{
objRequest.ClientCertificates.Add(x509);
}
using (StreamWriter myWriter = new StreamWriter(objRequest.GetRequestStream()))
{
myWriter.Write(postData);
}
using (WebResponse response = objRequest.GetResponse())
{
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
return sr.ReadToEnd();
}
}
}
In this method i succesfully sent request using StreamWriter object but i not got Response from paypal site so what should i do ? pls reply ..
my error is : The underlying connection was closed: The connection was closed unexpectedly.
I put image of my method which throws error and Browser Error.
pls give me your suggestion
The problem in your code is in Api url you are using
url="https://api.sandbox.paypal.com/nvp"
while you should use
url="https://api-3t.sandbox.paypal.com/nvp"
I'm using Web API to stream large files to clients, but I'd like to log if the download was successful or not. That is, if the server sent the entire content of the file.
Is there some way to get a a callback or event when the HttpResponseMessage completes sending data?
Perhaps something like this:
var stream = GetMyStream();
var response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContent(stream);
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
// This doesn't exist, but it illustrates what I'm trying to do.
response.OnComplete(context =>
{
if (context.Success)
Log.Info("File downloaded successfully.");
else
Log.Warn("File download was terminated by client.");
});
EDIT: I've now tested this using a real connection (via fiddler).
I inherited StreamContent and added my own OnComplete action which checks for an exception:
public class StreamContentWithCompletion : StreamContent
{
public StreamContentWithCompletion(Stream stream) : base (stream) { }
public StreamContentWithCompletion(Stream stream, Action<Exception> onComplete) : base(stream)
{
this.OnComplete = onComplete;
}
public Action<Exception> OnComplete { get; set; }
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
var t = base.SerializeToStreamAsync(stream, context);
t.ContinueWith(x =>
{
if (this.OnComplete != null)
{
// The task will be in a faulted state if something went wrong.
// I observed the following exception when I aborted the fiddler session:
// 'System.Web.HttpException (0x800704CD): The remote host closed the connection.'
if (x.IsFaulted)
this.OnComplete(x.Exception.GetBaseException());
else
this.OnComplete(null);
}
}, TaskContinuationOptions.ExecuteSynchronously);
return t;
}
}
Then I use it like so:
var stream = GetMyStream();
var response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContentWithCompletion(stream, ex =>
{
if (ex == null)
Log.Info("File downloaded successfully.");
else
Log.Warn("File download was terminated by client.");
});
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
return response;
I am not sure if there is direct signaling that all is ok, but you can use a trick to find out that the connection is exist just before you end it up, and right after you fully send the file.
For example the Response.IsClientConnected is return true if the client is still connected, so you can check something like:
// send the file, make a flush
Response.Flush();
// and now the file is fully sended check if the client is still connected
if(Response.IsClientConnected)
{
// log that all looks ok until the last byte.
}
else
{
// the client is not connected, so maybe have lost some data
}
// and now close the connection.
Response.End();
if the server sent the entire content of the file
Actually there is nothing to do :)
This might sound very simplistic but you will know if an exception is raised - if you care about server delivering and not client cancelling halfway. IsClientConnected is based on ASP.NET HttpResponse not the WebApi.