I'm doing some performance testing and ran across something puzzling to myself and I was hoping someone could shed some light.
I'm comparing the performance between an HttpWebRequest and a SoapHttpClientProtocol. In my tests I see the SoapHttpClientProtocol class performing twice as fast. However, I expected the HttpWebRequest to performance better.
Thanks for any insight anyone can provide!
Sam
Here is the code for the HttpWebRequest
public string RetrieveValue()
{
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] payload = encoding.GetBytes("sIP=");
string Url = #"url/RetrieveValue";
HttpWebRequest wr = (HttpWebRequest)HttpWebRequest.Create(Url);
wr.Method = "POST";
wr.KeepAlive = false;
wr.ContentType = "application/x-www-form-urlencoded";
wr.ContentLength = payload.Length;
wr.Timeout = 30000;
HttpWebResponse webResponse;
Stream wrStream = wr.GetRequestStream();
wrStream.Write(payload, 0, payload.Length);
wrStream.Close();
webResponse = (HttpWebResponse)wr.GetResponse();
Stream baseStream = webResponse.GetResponseStream();
string result = null;
using (StreamReader sr = new StreamReader(baseStream))
result = sr.ReadToEnd();
return result;
}
Here is the Code for the SoapHttpClientProtocol
WebServiceBinding(Name = "Soap", Namespace = "http://namespace.com/")]
public class MyRetriever : SoapHttpClientProtocol
{
[SoapDocumentMethod("http://url.com/Retrieve", RequestNamespace = "http://url.com/", ResponseNamespace = "http://url.com/", Use = SoapBindingUse.Literal, ParameterStyle = SoapParameterStyle.Wrapped)]
public string RetrieveValue(string sVal)
{
return (string)base.Invoke("RetrieveValue",
new object[] { sVal })[0];
}
}
How are you invoking the two tests? RetrieveValue is establishing a new connection each time, and if you are using a single instance of your test soap client and calling GetNewSessionKey each time you are probably not incurring the same overhead.
Related
I am trying to port some C# code over to F#.
The C# code has been taken from here (and slightly stripped back): https://github.com/joelpob/betfairng/blob/master/BetfairClient.cs
public bool Login(string p12CertificateLocation, string p12CertificatePassword, string username, string password)
{
var appKey = "APPKEY";
string postData = string.Format("username={0}&password={1}", username, password);
X509Certificate2 x509certificate = new X509Certificate2(p12CertificateLocation, p12CertificatePassword);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://identitysso.betfair.com/api/certlogin");
request.UseDefaultCredentials = true;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.Headers.Add("X-Application", appKey);
request.ClientCertificates.Add(x509certificate);
request.Accept = "*/*";
using (Stream stream = request.GetRequestStream())
using (StreamWriter writer = new StreamWriter(stream, Encoding.Default))
writer.Write(postData);
using (Stream stream = ((HttpWebResponse)request.GetResponse()).GetResponseStream())
using (StreamReader reader = new StreamReader(stream, Encoding.Default))
The C# code above works great. However, when trying to run (what I think is) F# equivalent code, without any real alterations, I get an error message.
The code is being run from the same computer, same VS installation and with exactly the same 4 arguments.
The error message I get is on the second to last line:
member x.Login(username, password,p12CertificateLocation:string, p12CertificatePassword:string) =
let AppKey = "APPKEY"
let url = "https://identitysso.betfair.com/api/certlogin"
let postData = "username=" + username + "&password=" + password
let x509certificate = new X509Certificate2(p12CertificateLocation, p12CertificatePassword)
let req = HttpWebRequest.Create(url) :?> HttpWebRequest
req.ClientCertificates.Add(x509certificate)|>ignore
req.UseDefaultCredentials <- true
req.Method <- "POST"
req.ContentType <- "application/x-www-form-urlencoded"
req.Headers.Add("X-Application",AppKey)
req.Accept <-"*/*"
use stream = req.GetRequestStream()
use writer =new StreamWriter(stream,Encoding.Default)
writer.Write(postData)
// fails on this line:
use stream = (req.GetResponse() :?> HttpWebResponse ).GetResponseStream()
// with System.Net.WebException: 'The remote server returned an error: (400) Bad Request.'
use reader = new StreamReader(stream,Encoding.Default)
I'm a bit lost here, as to my mind the two code implementations should be identical?
In this C# code:
using (Stream stream1 = request.GetRequestStream())
using (StreamWriter writer = new StreamWriter(stream1, Encoding.Default))
writer.Write(postData);
using (Stream stream2 = ((HttpWebResponse)request.GetResponse()).GetResponseStream())
using (StreamReader reader = new StreamReader(stream2, Encoding.Default))
writer and stream1 are flushed and closed immediately after the writer.Write call is finished, before you call request.GetResponse(). (This fact is somewhat obscured due to the, uhh.. interesting formatting of your code.)
In this F# code:
use stream1 = req.GetRequestStream()
use writer = new StreamWriter(stream1, Encoding.Default)
writer.Write(postData)
use stream2 = (req.GetResponse() :?> HttpWebResponse).GetResponseStream()
use reader = new StreamReader(stream2, Encoding.Default)
writer and stream1 stay alive and remain unflushed and unclosed when req.GetResponse() is called; you need to put them in an artificial scope to get the same behavior as C#:
do use stream1 = req.GetRequestStream()
use writer = new StreamWriter(stream1, Encoding.Default)
writer.Write(postData)
(* or
(use stream1 = req.GetRequestStream()
use writer = new StreamWriter(stream1, Encoding.Default)
writer.Write(postData))
*)
use stream2 = (req.GetResponse() :?> HttpWebResponse).GetResponseStream()
use reader = new StreamReader(stream2, Encoding.Default)
That's not "the C# way" to make an HTTP POST call. The typical way, in all supported .NET versions (ie 4.5.2 and later) is to use HttpClient. Even with HttpWebRequest, there are too many redundant or contradictory calls, like using default credentials (ie Windows authentication)
The C# way is this:
var client=new HttpClient("https://identitysso.betfair.com/api");
var values = new Dictionary<string, string>
{
{ "username", username },
{ "password", password }
};
var content = new FormUrlEncodedContent(values);
content.Headers.Add("X-Application",apiKey);
var response = await client.PostAsync("certlogin", content);
var responseString = await response.Content.ReadAsStringAsync();
In order to use a client certificate, you have to create the client instance using a custom HTTP Handler:
var handler = new WebRequestHandler();
var x509certificate = new X509Certificate2(certPath, certPassword);
handler.ClientCertificates.Add(certificate);
var client = new HttpClient(handler)
{
BaseAddress = new Uri("https://identitysso.betfair.com/api")
}
Writing the same code in F# is straight-forward:
let login username password (certPath:string) (certPassword:string) (apiKey:string) =
let handler = new WebRequestHandler()
let certificate = new X509Certificate2(certPath, certPassword)
handler.ClientCertificates.Add certificate |> ignore
let client = new HttpClient(handler,BaseAddress = Uri("https://identitysso.betfair.com"))
async {
let values = dict["username", username ; "password", password ]
let content = new FormUrlEncodedContent(values)
content.Headers.Add( "X-Application" ,apiKey)
let! response = client.PostAsync("api/certlogin",content) |> Async.AwaitTask
response.EnsureSuccessStatusCode() |> ignore
let! responseString = response.Content.ReadAsStringAsync() |> Async.AwaitTask
return responseString
}
The client, handler are thread safe and can be reused so they can be stored in fields. Reusing the same client means that the OS doesn't have to create a new TCP/IP connection each time, leading to improved performance. It's better to create the client separately. :
let buildClient (certPath:string) (certPassword:string) =
let handler = new WebRequestHandler()
let certificate = new X509Certificate2(certPath, certPassword)
handler.ClientCertificates.Add certificate |> ignore
new HttpClient(handler,BaseAddress = Uri("https://identitysso.betfair.com"))
let login (client:HttpClient) username password (apiKey:string) =
async {
let values = dict["username", username ; "password", password ]
let content = new FormUrlEncodedContent(values)
content.Headers.Add( "X-Application" ,apiKey)
let! response = client.PostAsync("api/certlogin",content) |> Async.AwaitTask
response.EnsureSuccessStatusCode() |> ignore
let! responseString = response.Content.ReadAsStringAsync() |> Async.AwaitTask
//Do whatever is needed here
return responseString
}
I am trying to add order in Kraken through API call https://api.kraken.com/0/private/AddOrder. I found EAPI:Invalid nonce error on inserting new order in Kraken. Right Now I am inserting only one order at a time on button click, But there may be the situation when multiple order will be inserted. I have tried too many different solutions to generate a nonce But, still found the same issue. Does anybody know what's wrong?
private JsonObject QueryPrivate(string a_sMethod, string props = null)
{
// generate a 64 bit nonce using a timestamp at tick resolution
Int64 nonce = DateTime.Now.Ticks;
props = "nonce=" + nonce + props;
string path = string.Format("/{0}/private/{1}", _version, a_sMethod);
string address = _url + path;
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(address);
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.Method = "POST";
webRequest.Headers.Add("API-Key", _key);
byte[] base64DecodedSecred = Convert.FromBase64String(_secret);
var np = nonce + Convert.ToChar(0) + props;
var pathBytes = Encoding.UTF8.GetBytes(path);
var hash256Bytes = sha256_hash(np);
var z = new byte[pathBytes.Count() + hash256Bytes.Count()];
pathBytes.CopyTo(z, 0);
hash256Bytes.CopyTo(z, pathBytes.Count());
var signature = getHash(base64DecodedSecred, z);
webRequest.Headers.Add("API-Sign", Convert.ToBase64String(signature));
if (props != null)
{
using (var writer = new StreamWriter(webRequest.GetRequestStream()))
{
writer.Write(props);
}
}
//Make the request
try
{
//Wait for RateGate
_rateGate.WaitToProceed();
using (WebResponse webResponse = webRequest.GetResponse())
{
using (Stream str = webResponse.GetResponseStream())
{
using (StreamReader sr = new StreamReader(str))
{
string data = sr.ReadToEnd();
dynamic d = JObject.Parse(data);
return (JsonObject)JsonConvert.Import(data);
}
}
}
}
catch (WebException wex)
{
using (HttpWebResponse response = (HttpWebResponse)wex.Response)
{
using (Stream str = response.GetResponseStream())
{
using (StreamReader sr = new StreamReader(str))
{
string data = sr.ReadToEnd();
if (response.StatusCode != HttpStatusCode.InternalServerError)
{
throw;
}
return (JsonObject)JsonConvert.Import(sr);
}
}
}
}
}
Some exchange platforms allow the use of float type nonce, in your case as you are using Kraken and according to Kraken Api documentation,
nonce = always increasing unsigned 64 bit integer
Kraken will require an integer nonce.
Why invalid nonce ?
In my opinion, the issue occurs when you sent more than one request with the same nonce ...
When you convert the timestamp to an integer, you will will be allowed consequently to sent only 1 request per second (because the limitation relative to the integer nonce changing each second)
In order to have the capability to send more than one requests per seconds, one tips could be to multiply timestamp by 1000 then convert to integer and use this value as nonce.
nonce=integer(1000*timestamp)
In that case you may sent more than 1 request per second (because each nonce will be different) but remember that :
exchange platform have safeguards in place to protect against
abuse/DoS attacks.
I'd like to load an image directly from a URL but without saving it on the server, I want to upload it directly from memory to Amazon S3 server.
This is my code:
Dim wc As New WebClient
Dim fileStream As IO.Stream = wc.OpenRead("http://www.domain.com/image.jpg")
Dim request As New PutObjectRequest()
request.BucketName = "mybucket"
request.Key = "file.jpg"
request.InputStream = fileStream
client.PutObject(request)
The Amazon API gives me the error "Could not determine content length". The stream fileStream ends up as "System.Net.ConnectStream" which I'm not sure if it's correct.
The exact same code works with files from the HttpPostedFile but I need to use it in this way now.
Any ideas how I can convert the stream to become what Amazon API is expecting (with the length intact)?
I had the same problem when I'm using the GetObjectResponse() method and its propertie ResponseStream to copy a file from a folder to another in same bucket. I noted that the AWS SDK (2.3.45) have some faults like a another method called WriteResponseStreamToFile in GetObjectResponse() that simply doesn't work. These lacks of functions needs some workarounds.
I solved the problem openning the file in array of bytes and putting it in a MemoryStream object.
Try this (C# code)
WebClient wc = new WebClient();
Stream fileStream = wc.OpenRead("http://www.domain.com/image.jpg");
byte[] fileBytes = fileStream.ToArrayBytes();
PutObjectRequest request = new PutObjectRequest();
request.BucketName = "mybucket";
request.Key = "file.jpg";
request.InputStream = new MemoryStream(fileBytes);
client.PutObject(request);
The extesion method
public static byte[] ToArrayBytes(this Stream input)
{
byte[] buffer = new byte[16 * 1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}
You can also create a MemoryStream without an array of bytes. But after the first PutObject in S3, the MemoryStream will be discarted. If you need to put others objects, I recommend the first option
WebClient wc = new WebClient();
Stream fileStream = wc.OpenRead("http://www.domain.com/image.jpg");
MemoryStream fileMemoryStream = fileStream.ToMemoryStream();
PutObjectRequest request = new PutObjectRequest();
request.BucketName = "mybucket";
request.Key = "file.jpg";
request.InputStream = fileMemoryStream ;
client.PutObject(request);
The extesion method
public static MemoryStream ToMemoryStream(this Stream input)
{
byte[] buffer = new byte[16 * 1024];
int read;
MemoryStream ms = new MemoryStream();
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms;
}
I had the same problem in a similar scenario.
The reason for the error is that to upload an object the SDK needs to know the whole content length that is going to be uploaded. To be able to obtain stream length it must be seekable, but the stream returned from WebClient is not. To indicate the expected length set Headers.ContentLength in PutObjectRequest. The SDK will use this value if it cannot determine length from the stream object.
To make your code work, obtain content length from the response headers returned by the call made by WebClient. Then set PutObjectRequest.Headers.ContentLength. Of course this relies on the server returned content length value.
Dim wc As New WebClient
Dim fileStream As IO.Stream = wc.OpenRead("http://www.example.com/image.jpg")
Dim contentLength As Long = Long.Parse(client.ResponseHeaders("Content-Length"))
Dim request As New PutObjectRequest()
request.BucketName = "mybucket"
request.Key = "file.jpg"
request.InputStream = fileStream
request.Headers.ContentLength = contentLength
client.PutObject(request)
I came up with a solution that uses UploadPart when the length is not available by any other means, plus this does not load the entire file into memory.
if (args.DocumentContents.CanSeek)
{
PutObjectRequest r = new PutObjectRequest();
r.InputStream = args.DocumentContents;
r.BucketName = s3Id.BucketName;
r.Key = s3Id.ObjectKey;
foreach (var item in args.CustomData)
{
r.Metadata[item.Key] = item.Value;
}
await S3Client.PutObjectAsync(r);
}
else
{
// if stream does not allow seeking, S3 client will throw error:
// Amazon.S3.AmazonS3Exception : Could not determine content length
// as a work around, if cannot use length property, will chunk
// file into sections and use UploadPart, so do not have to load
// entire file into memory as a single MemoryStream.
var r = new InitiateMultipartUploadRequest();
r.BucketName = s3Id.BucketName;
r.Key = s3Id.ObjectKey;
foreach (var item in args.CustomData)
{
r.Metadata[item.Key] = item.Value;
}
var multipartResponse = await S3Client.InitiateMultipartUploadAsync(r);
try
{
var completeRequest = new CompleteMultipartUploadRequest
{
UploadId = multipartResponse.UploadId,
BucketName = s3Id.BucketName,
Key = s3Id.ObjectKey,
};
// just using this size, because it is the max for Azure File Share, but it could be any size
// for S3, even a configured value
const int blockSize = 4194304;
// BinaryReader gives us access to ReadBytes
using (var reader = new BinaryReader(args.DocumentContents))
{
var partCounter = 1;
while (true)
{
byte[] buffer = reader.ReadBytes(blockSize);
if (buffer.Length == 0)
break;
using (MemoryStream uploadChunk = new MemoryStream(buffer))
{
uploadChunk.Position = 0;
var uploadRequest = new UploadPartRequest
{
BucketName = s3Id.BucketName,
Key = s3Id.ObjectKey,
UploadId = multipartResponse.UploadId,
PartNumber = partCounter,
InputStream = uploadChunk,
};
// could call UploadPart on multiple threads, instead of using await, but that would
// cause more data to be loaded into memory, which might be too much
var part2Task = await S3Client.UploadPartAsync(uploadRequest);
completeRequest.AddPartETags(part2Task);
}
partCounter++;
}
var completeResponse = await S3Client.CompleteMultipartUploadAsync(completeRequest);
}
}
catch
{
await S3Client.AbortMultipartUploadAsync(s3Id.BucketName, s3Id.ObjectKey
, multipartResponse.UploadId);
throw;
}
}
I have a web API project with a controller like this:
namespace Api.Controllers
{
public class StudyController : ApiController
{
[Route("api/PostReviewedStudyData")]
[HttpPost]
public bool PostReviewedStudyData([FromBody]string jsonStudy)
{
ApiStudy study = JsonHelper.JsonDeserialize<ApiStudy>(jsonStudy);
BusinessLogics.BL.SaveReviewedStudyDataToDb(study);
return true;
}
[Route("api/GetStudyData/{studyUid}")]
[HttpGet, HttpPost]
public string GetStudyData(string studyUid)
{
ApiStudy study = BusinessLogics.BL.GetStudyObject(studyUid);
return JsonHelper.JsonSerializer<ApiStudy>(study);
}
}
}
I call it like this, from another application:
HttpWebRequest httpWReq = (HttpWebRequest)WebRequest.Create(#"http://localhost:60604/api/PostReviewedStudyData");
ASCIIEncoding encoding = new ASCIIEncoding();
string postData = Api.JsonHelper.JsonSerializer<ApiStudy>(s);
byte[] data = encoding.GetBytes(postData);
httpWReq.Method = "POST";
httpWReq.ContentType = "application/json; charset=utf-8";
httpWReq.ContentLength = data.Length;
httpWReq.Accept = "application/json";
using (Stream stream = httpWReq.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
HttpWebResponse response = (HttpWebResponse)httpWReq.GetResponse();
string responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
My breakpoint at the post method is hit, but the jsonStudy object is null. Any Ideas?
First of all what i notice is this:
HttpWebRequest httpWReq = (HttpWebRequest)WebRequest.Create(#"http://localhost:60604/api/PostReviewedStudy Data");
you have a space in the PostReviewedStudy Data also if that does not work try removing the content type line and see if it works
Try the following:
[Route("api/PostReviewedStudyData")]
[HttpPost]
public bool PostReviewedStudyData([FromBody]ApiStudy study)
{
BusinessLogics.BL.SaveReviewedStudyDataToDb(study);
return true;
}
WebApi supports fully typed parameters, there's no need to convert from a JSON string.
Trying to implement a WebRequest and return to the caller synchronously.
I have tried various implementations and I think this would be the most appropriate so far.
Unfortunately the following code throws an InvalidOperationException with the message
EndGetResponse can only be called once for each asynchronous operation
I really struggled enough to make this happen and its really vital to the library I build to use the WebRequest like this.
The following code is intend to use in Windows Phone 8 and Windows 8 platforms.
I already understand the async/await pattern and used it, but it is REALLY vital for me to use the synchronous version of the web service request in a part of my library.
The code:
public void ExecuteRequest(string url, string requestData)
{
WebRequest request = WebRequest.Create(new Uri(url));
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.Headers["Header-Key"] = "AKey";
DTOWebRequest webRequestState = new DTOWebRequest
{
Data = requestData,
Request = request
};
ManualResetEventSlim resetEventSlim = new ManualResetEventSlim(false);
// Begin the request using a delegate
request.BeginGetRequestStream(ar =>
{
DTOWebRequest requestDataObj = (DTOWebRequest )ar.AsyncState;
HttpWebRequest requestStream = (HttpWebRequest)requestDataObj.Request;
string data = requestDataObj.Data;
// Convert the string into a byte array.
byte[] postBytes = Encoding.UTF8.GetBytes(data);
try
{
// End the operation
using (Stream endGetRequestStream = requestStream.EndGetRequestStream(ar))
{
// Write to the request stream.
endGetRequestStream.Write(postBytes, 0, postBytes.Length);
}
// Get the response using a delegate
requestStream.BeginGetResponse(result =>
{
DTOWebRequest requestDataObjResult = (DTOWebRequest )ar.AsyncState;
HttpWebRequest requestResult = (HttpWebRequest)requestDataObjResult.Request;
try
{
// End the operation
using (HttpWebResponse response = (HttpWebResponse)requestResult.EndGetResponse(ar)) // Here the exception is thrown.
{
HttpStatusCode rcode = response.StatusCode;
Stream streamResponse = response.GetResponseStream();
StreamReader streamRead = new StreamReader(streamResponse);
// The Response
string responseString = streamRead.ReadToEnd();
if (!string.IsNullOrWhiteSpace(requestDataObjResult.FileName))
{
FileRepository fileRepo = new FileRepository();
fileRepo.Delete(requestDataObjResult.FileName);
}
Debug.WriteLine("Response : {0}", responseString);
}
}
catch (WebException webEx)
{
WebExceptionStatus status = webEx.Status;
WebResponse responseEx = webEx.Response;
Debug.WriteLine(webEx.ToString());
}
resetEventSlim.Set(); // Signal to return handler
}, requestDataObj);
}
catch (WebException webEx)
{
WebExceptionStatus status = webEx.Status;
WebResponse responseEx = webEx.Response;
Debug.WriteLine(webEx.ToString());
}
}, webRequestState);
resetEventSlim.Wait(5000); // Wait either for Set() or a timeout 5 secs.
}
}
Thank you.
You can't do synchronous web calls in Windows Phone and that's why you aren't.
If you were, you'd be calling GetRequestStream instead of BeginGetRequestStram/EndGetRequestStream.
The only reason to be synchronous on Windows Phone is to block the UI which is a very bad idea.
You should use an HttpClient and àsync-await` instead.
But if you really think you should (and can) do asynchronous calls on Windows Phone, you can always try something like this:
public void ExecuteRequest(string url, string requestData)
{
try
{
WebRequest request = WebRequest.Create(new Uri(url));
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.Headers["Header-Key"] = "AKey";
// Convert the string into a byte array.
byte[] postBytes = Encoding.UTF8.GetBytes(requestData);
using (var requestStream = request.EndGetRequestStream(request.BeginGetRequestStream(null, null)))
{
// Write to the request stream.
endGetRequestStream.Write(postBytes, 0, postBytes.Length);
}
using (var response = request.EndGetResponse(request.BeginGetResponse(null, null)))
{
using (var streamRead = new StreamReader(response.GetResponseStream()))
{
// The Response
string responseString = streamRead.ReadToEnd();
if (!string.IsNullOrWhiteSpace(requestDataObjResult.FileName))
{
var fileRepo = new FileRepository();
fileRepo.Delete(request.FileName);
}
Debug.WriteLine("Response : {0}", responseString);
}
}
}
catch (WebException webEx)
{
WebExceptionStatus status = webEx.Status;
WebResponse responseEx = webEx.Response;
Debug.WriteLine(webEx.ToString());
}
}
But I really think you should revise your decision/need.