Read response from WebResponse Response Stream - http

I've been playing around with some basic HTTP request response operations in an F# interactive session to try and get a better feel for the language. Everything here seems to be working as expected except when I get the response stream for my HTTP request I can never seem to read anything out of it. In the following code I get the text from the printfn line every time but the value for responseString always seems to be empty. Am I missing anything obvious?
let httpListener = new HttpListener()
httpListener.Prefixes.Add("http://*:8020/")
let getContext = Async.FromBeginEnd(httpListener.BeginGetContext, httpListener.EndGetContext)
let processRequest = async {
while true do
let! context = getContext
let request = context.Request
use requestReader = new StreamReader(request.InputStream)
let requestString = requestReader.ReadToEnd()
let response = context.Response
use streamWriter = new StreamWriter(response.OutputStream)
streamWriter.WriteLine "A response!"
response.Close()
}
httpListener.Start()
let maxThreads = 10
for n = 0 to maxThreads do
Async.Start processRequest
printfn "HttpListener started with %d maximum threads processing requests." maxThreads
let sendRequest = async {
let request = WebRequest.Create("http://localhost:8020") :?> HttpWebRequest
let postBytes = Encoding.ASCII.GetBytes("Issuing HTTP")
request.Method <- "POST"
request.ContentType <- "application/x-www-form-urlencoded"
request.ContentLength <- int64 postBytes.Length
let requestStream = request.GetRequestStream()
requestStream.Write(postBytes, 0, postBytes.Length)
requestStream.Close()
let! response = Async.FromBeginEnd(request.BeginGetResponse, request.EndGetResponse)
let responseStream = response.GetResponseStream()
use streamReader = new StreamReader(responseStream)
let responseString = streamReader.ReadToEnd()
printfn "Received the following response: %s" responseString
}
for n = 0 to maxThreads do
Async.Start sendRequest

you need to reduce the scope of usage for StreamWriter so it can be flushed on Dispose.
let processRequest = async {
while true do
let! context = getContext
let request = context.Request
use requestReader = new StreamReader(request.InputStream)
let requestString = requestReader.ReadToEnd()
let response = context.Response
do
use streamWriter = new StreamWriter(response.OutputStream)
streamWriter.WriteLine "A response!"
response.Close()
}

Related

Asp.net core disable change string in Uri

I have a Encoded string like this:
https://xx.yyy.ir/xx/ff/addUser?name=%d8%b3%d9%84%d8%a7%d9%85
But when I use Uri to convert it to a URL and send it
result = "https://xx.yyy.ir/xx/ff/addUser?name=%d8%b3%d9%84%d8%a7%d9%85"
var client = new HttpClient
{
BaseAddress = new Uri(result.ToString()),
};
var response = await client.GetAsync("");
it send this request :
https://xx.yyy.ir/xx/ff/addUser?name=سلام
why this happen? how to prevent from this?
This is what's causing your problem: new Uri(result.ToString())
Let's try to do this in a proper manner and see what happens.
var builder = new UriBuilder("https://xx.yyy.ir/xx/ff/addUser") { Port = -1 };
var query = HttpUtility.ParseQueryString(builder.Query);
query["name"] = "سلام";
builder.Query = query.ToString();
using var httpClient = new HttpClient();
var response = await client.GetAsync(builder.ToString());
builder.ToString() returns https://xx.yyy.ir/xx/ff/addUser?name=%d8%b3%d9%84%d8%a7%d9%85
So basically, the above code boils down to this:
using var httpClient = new HttpClient();
var response = await client.GetAsync("https://xx.yyy.ir/xx/ff/addUser?name=%d8%b3%d9%84%d8%a7%d9%85");
Tested and verified on my computer.

System.Net.WebException when porting from C# to F#

I am trying to port some C# code over to F#.
The C# code has been taken from here (and slightly stripped back): https://github.com/joelpob/betfairng/blob/master/BetfairClient.cs
public bool Login(string p12CertificateLocation, string p12CertificatePassword, string username, string password)
{
var appKey = "APPKEY";
string postData = string.Format("username={0}&password={1}", username, password);
X509Certificate2 x509certificate = new X509Certificate2(p12CertificateLocation, p12CertificatePassword);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://identitysso.betfair.com/api/certlogin");
request.UseDefaultCredentials = true;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.Headers.Add("X-Application", appKey);
request.ClientCertificates.Add(x509certificate);
request.Accept = "*/*";
using (Stream stream = request.GetRequestStream())
using (StreamWriter writer = new StreamWriter(stream, Encoding.Default))
writer.Write(postData);
using (Stream stream = ((HttpWebResponse)request.GetResponse()).GetResponseStream())
using (StreamReader reader = new StreamReader(stream, Encoding.Default))
The C# code above works great. However, when trying to run (what I think is) F# equivalent code, without any real alterations, I get an error message.
The code is being run from the same computer, same VS installation and with exactly the same 4 arguments.
The error message I get is on the second to last line:
member x.Login(username, password,p12CertificateLocation:string, p12CertificatePassword:string) =
let AppKey = "APPKEY"
let url = "https://identitysso.betfair.com/api/certlogin"
let postData = "username=" + username + "&password=" + password
let x509certificate = new X509Certificate2(p12CertificateLocation, p12CertificatePassword)
let req = HttpWebRequest.Create(url) :?> HttpWebRequest
req.ClientCertificates.Add(x509certificate)|>ignore
req.UseDefaultCredentials <- true
req.Method <- "POST"
req.ContentType <- "application/x-www-form-urlencoded"
req.Headers.Add("X-Application",AppKey)
req.Accept <-"*/*"
use stream = req.GetRequestStream()
use writer =new StreamWriter(stream,Encoding.Default)
writer.Write(postData)
// fails on this line:
use stream = (req.GetResponse() :?> HttpWebResponse ).GetResponseStream()
// with System.Net.WebException: 'The remote server returned an error: (400) Bad Request.'
use reader = new StreamReader(stream,Encoding.Default)
I'm a bit lost here, as to my mind the two code implementations should be identical?
In this C# code:
using (Stream stream1 = request.GetRequestStream())
using (StreamWriter writer = new StreamWriter(stream1, Encoding.Default))
writer.Write(postData);
using (Stream stream2 = ((HttpWebResponse)request.GetResponse()).GetResponseStream())
using (StreamReader reader = new StreamReader(stream2, Encoding.Default))
writer and stream1 are flushed and closed immediately after the writer.Write call is finished, before you call request.GetResponse(). (This fact is somewhat obscured due to the, uhh.. interesting formatting of your code.)
In this F# code:
use stream1 = req.GetRequestStream()
use writer = new StreamWriter(stream1, Encoding.Default)
writer.Write(postData)
use stream2 = (req.GetResponse() :?> HttpWebResponse).GetResponseStream()
use reader = new StreamReader(stream2, Encoding.Default)
writer and stream1 stay alive and remain unflushed and unclosed when req.GetResponse() is called; you need to put them in an artificial scope to get the same behavior as C#:
do use stream1 = req.GetRequestStream()
use writer = new StreamWriter(stream1, Encoding.Default)
writer.Write(postData)
(* or
(use stream1 = req.GetRequestStream()
use writer = new StreamWriter(stream1, Encoding.Default)
writer.Write(postData))
*)
use stream2 = (req.GetResponse() :?> HttpWebResponse).GetResponseStream()
use reader = new StreamReader(stream2, Encoding.Default)
That's not "the C# way" to make an HTTP POST call. The typical way, in all supported .NET versions (ie 4.5.2 and later) is to use HttpClient. Even with HttpWebRequest, there are too many redundant or contradictory calls, like using default credentials (ie Windows authentication)
The C# way is this:
var client=new HttpClient("https://identitysso.betfair.com/api");
var values = new Dictionary<string, string>
{
{ "username", username },
{ "password", password }
};
var content = new FormUrlEncodedContent(values);
content.Headers.Add("X-Application",apiKey);
var response = await client.PostAsync("certlogin", content);
var responseString = await response.Content.ReadAsStringAsync();
In order to use a client certificate, you have to create the client instance using a custom HTTP Handler:
var handler = new WebRequestHandler();
var x509certificate = new X509Certificate2(certPath, certPassword);
handler.ClientCertificates.Add(certificate);
var client = new HttpClient(handler)
{
BaseAddress = new Uri("https://identitysso.betfair.com/api")
}
Writing the same code in F# is straight-forward:
let login username password (certPath:string) (certPassword:string) (apiKey:string) =
let handler = new WebRequestHandler()
let certificate = new X509Certificate2(certPath, certPassword)
handler.ClientCertificates.Add certificate |> ignore
let client = new HttpClient(handler,BaseAddress = Uri("https://identitysso.betfair.com"))
async {
let values = dict["username", username ; "password", password ]
let content = new FormUrlEncodedContent(values)
content.Headers.Add( "X-Application" ,apiKey)
let! response = client.PostAsync("api/certlogin",content) |> Async.AwaitTask
response.EnsureSuccessStatusCode() |> ignore
let! responseString = response.Content.ReadAsStringAsync() |> Async.AwaitTask
return responseString
}
The client, handler are thread safe and can be reused so they can be stored in fields. Reusing the same client means that the OS doesn't have to create a new TCP/IP connection each time, leading to improved performance. It's better to create the client separately. :
let buildClient (certPath:string) (certPassword:string) =
let handler = new WebRequestHandler()
let certificate = new X509Certificate2(certPath, certPassword)
handler.ClientCertificates.Add certificate |> ignore
new HttpClient(handler,BaseAddress = Uri("https://identitysso.betfair.com"))
let login (client:HttpClient) username password (apiKey:string) =
async {
let values = dict["username", username ; "password", password ]
let content = new FormUrlEncodedContent(values)
content.Headers.Add( "X-Application" ,apiKey)
let! response = client.PostAsync("api/certlogin",content) |> Async.AwaitTask
response.EnsureSuccessStatusCode() |> ignore
let! responseString = response.Content.ReadAsStringAsync() |> Async.AwaitTask
//Do whatever is needed here
return responseString
}

EndGetResponse can only be called once for each asynchronous operation?

Trying to implement a WebRequest and return to the caller synchronously.
I have tried various implementations and I think this would be the most appropriate so far.
Unfortunately the following code throws an InvalidOperationException with the message
EndGetResponse can only be called once for each asynchronous operation
I really struggled enough to make this happen and its really vital to the library I build to use the WebRequest like this.
The following code is intend to use in Windows Phone 8 and Windows 8 platforms.
I already understand the async/await pattern and used it, but it is REALLY vital for me to use the synchronous version of the web service request in a part of my library.
The code:
public void ExecuteRequest(string url, string requestData)
{
WebRequest request = WebRequest.Create(new Uri(url));
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.Headers["Header-Key"] = "AKey";
DTOWebRequest webRequestState = new DTOWebRequest
{
Data = requestData,
Request = request
};
ManualResetEventSlim resetEventSlim = new ManualResetEventSlim(false);
// Begin the request using a delegate
request.BeginGetRequestStream(ar =>
{
DTOWebRequest requestDataObj = (DTOWebRequest )ar.AsyncState;
HttpWebRequest requestStream = (HttpWebRequest)requestDataObj.Request;
string data = requestDataObj.Data;
// Convert the string into a byte array.
byte[] postBytes = Encoding.UTF8.GetBytes(data);
try
{
// End the operation
using (Stream endGetRequestStream = requestStream.EndGetRequestStream(ar))
{
// Write to the request stream.
endGetRequestStream.Write(postBytes, 0, postBytes.Length);
}
// Get the response using a delegate
requestStream.BeginGetResponse(result =>
{
DTOWebRequest requestDataObjResult = (DTOWebRequest )ar.AsyncState;
HttpWebRequest requestResult = (HttpWebRequest)requestDataObjResult.Request;
try
{
// End the operation
using (HttpWebResponse response = (HttpWebResponse)requestResult.EndGetResponse(ar)) // Here the exception is thrown.
{
HttpStatusCode rcode = response.StatusCode;
Stream streamResponse = response.GetResponseStream();
StreamReader streamRead = new StreamReader(streamResponse);
// The Response
string responseString = streamRead.ReadToEnd();
if (!string.IsNullOrWhiteSpace(requestDataObjResult.FileName))
{
FileRepository fileRepo = new FileRepository();
fileRepo.Delete(requestDataObjResult.FileName);
}
Debug.WriteLine("Response : {0}", responseString);
}
}
catch (WebException webEx)
{
WebExceptionStatus status = webEx.Status;
WebResponse responseEx = webEx.Response;
Debug.WriteLine(webEx.ToString());
}
resetEventSlim.Set(); // Signal to return handler
}, requestDataObj);
}
catch (WebException webEx)
{
WebExceptionStatus status = webEx.Status;
WebResponse responseEx = webEx.Response;
Debug.WriteLine(webEx.ToString());
}
}, webRequestState);
resetEventSlim.Wait(5000); // Wait either for Set() or a timeout 5 secs.
}
}
Thank you.
You can't do synchronous web calls in Windows Phone and that's why you aren't.
If you were, you'd be calling GetRequestStream instead of BeginGetRequestStram/EndGetRequestStream.
The only reason to be synchronous on Windows Phone is to block the UI which is a very bad idea.
You should use an HttpClient and àsync-await` instead.
But if you really think you should (and can) do asynchronous calls on Windows Phone, you can always try something like this:
public void ExecuteRequest(string url, string requestData)
{
try
{
WebRequest request = WebRequest.Create(new Uri(url));
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.Headers["Header-Key"] = "AKey";
// Convert the string into a byte array.
byte[] postBytes = Encoding.UTF8.GetBytes(requestData);
using (var requestStream = request.EndGetRequestStream(request.BeginGetRequestStream(null, null)))
{
// Write to the request stream.
endGetRequestStream.Write(postBytes, 0, postBytes.Length);
}
using (var response = request.EndGetResponse(request.BeginGetResponse(null, null)))
{
using (var streamRead = new StreamReader(response.GetResponseStream()))
{
// The Response
string responseString = streamRead.ReadToEnd();
if (!string.IsNullOrWhiteSpace(requestDataObjResult.FileName))
{
var fileRepo = new FileRepository();
fileRepo.Delete(request.FileName);
}
Debug.WriteLine("Response : {0}", responseString);
}
}
}
catch (WebException webEx)
{
WebExceptionStatus status = webEx.Status;
WebResponse responseEx = webEx.Response;
Debug.WriteLine(webEx.ToString());
}
}
But I really think you should revise your decision/need.

High response time from new StreamReader(webRsp.GetResponseStream())

I am trying to get response from web url.
but while we are throwing some load on it, lets say 100 user load. this line of code work very slowly. After reading the response from below code I have to send myXML to calling function for some use.
using (StreamReader rspStr = new StreamReader(webRsp.GetResponseStream()))
{
myXML = rspStr.ReadToEnd().Trim();
}
Is there any way to get good response time even after throwing 100 or 1000 users load.
I would try with an Async approach, just to avoid locking the execution on Stream opening or slow network waits, everything is explained here:
Making Asynchronous Requests
snippet:
WebRequest wreq = WebRequest.Create(httpSite);
// Create the state object.
RequestState rs = new RequestState();
// Put the request into the state object so it can be passed around.
rs.Request = wreq;
// Issue the async request.
IAsyncResult r = (IAsyncResult) wreq.BeginGetResponse(
new AsyncCallback(RespCallback), rs);
private static void RespCallback(IAsyncResult ar)
{
// Get the RequestState object from the async result.
RequestState rs = (RequestState) ar.AsyncState;
// Get the WebRequest from RequestState.
WebRequest req = rs.Request;
// Call EndGetResponse, which produces the WebResponse object
// that came from the request issued above.
WebResponse resp = req.EndGetResponse(ar);
// Start reading data from the response stream.
Stream ResponseStream = resp.GetResponseStream();
// Store the response stream in RequestState to read
// the stream asynchronously.
rs.ResponseStream = ResponseStream;
// Pass rs.BufferRead to BeginRead. Read data into rs.BufferRead
IAsyncResult iarRead = ResponseStream.BeginRead(rs.BufferRead, 0,
BUFFER_SIZE, new AsyncCallback(ReadCallBack), rs);
}
private static void ReadCallBack(IAsyncResult asyncResult)
{
// Get the RequestState object from AsyncResult.
RequestState rs = (RequestState)asyncResult.AsyncState;
// Retrieve the ResponseStream that was set in RespCallback.
Stream responseStream = rs.ResponseStream;
// Read rs.BufferRead to verify that it contains data.
int read = responseStream.EndRead( asyncResult );
if (read > 0)
{
// Prepare a Char array buffer for converting to Unicode.
Char[] charBuffer = new Char[BUFFER_SIZE];
// Convert byte stream to Char array and then to String.
// len contains the number of characters converted to Unicode.
int len =
rs.StreamDecode.GetChars(rs.BufferRead, 0, read, charBuffer, 0);
String str = new String(charBuffer, 0, len);
// Append the recently read data to the RequestData stringbuilder
// object contained in RequestState.
rs.RequestData.Append(
Encoding.ASCII.GetString(rs.BufferRead, 0, read));
// Continue reading data until
// responseStream.EndRead returns –1.
IAsyncResult ar = responseStream.BeginRead(
rs.BufferRead, 0, BUFFER_SIZE,
new AsyncCallback(ReadCallBack), rs);
}
else
{
if(rs.RequestData.Length>0)
{
// Display data to the console.
string strContent;
strContent = rs.RequestData.ToString();
}
// Close down the response stream.
responseStream.Close();
// Set the ManualResetEvent so the main thread can exit.
allDone.Set();
}
return;
}

SoapHttpClientProtocol Versus HttpWebRequest

I'm doing some performance testing and ran across something puzzling to myself and I was hoping someone could shed some light.
I'm comparing the performance between an HttpWebRequest and a SoapHttpClientProtocol. In my tests I see the SoapHttpClientProtocol class performing twice as fast. However, I expected the HttpWebRequest to performance better.
Thanks for any insight anyone can provide!
Sam
Here is the code for the HttpWebRequest
public string RetrieveValue()
{
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] payload = encoding.GetBytes("sIP=");
string Url = #"url/RetrieveValue";
HttpWebRequest wr = (HttpWebRequest)HttpWebRequest.Create(Url);
wr.Method = "POST";
wr.KeepAlive = false;
wr.ContentType = "application/x-www-form-urlencoded";
wr.ContentLength = payload.Length;
wr.Timeout = 30000;
HttpWebResponse webResponse;
Stream wrStream = wr.GetRequestStream();
wrStream.Write(payload, 0, payload.Length);
wrStream.Close();
webResponse = (HttpWebResponse)wr.GetResponse();
Stream baseStream = webResponse.GetResponseStream();
string result = null;
using (StreamReader sr = new StreamReader(baseStream))
result = sr.ReadToEnd();
return result;
}
Here is the Code for the SoapHttpClientProtocol
WebServiceBinding(Name = "Soap", Namespace = "http://namespace.com/")]
public class MyRetriever : SoapHttpClientProtocol
{
[SoapDocumentMethod("http://url.com/Retrieve", RequestNamespace = "http://url.com/", ResponseNamespace = "http://url.com/", Use = SoapBindingUse.Literal, ParameterStyle = SoapParameterStyle.Wrapped)]
public string RetrieveValue(string sVal)
{
return (string)base.Invoke("RetrieveValue",
new object[] { sVal })[0];
}
}
How are you invoking the two tests? RetrieveValue is establishing a new connection each time, and if you are using a single instance of your test soap client and calling GetNewSessionKey each time you are probably not incurring the same overhead.

Resources