I am trying to get AngleSharp to use both a proxy and set header properties like this:
var handler = new HttpClientHandler
{
Proxy = new WebProxy(ProxyMan.GetProxy()),
UseProxy = true,
PreAuthenticate = true,
UseDefaultCredentials = false
};
var requester = new DefaultHttpRequester();
requester.Headers["User-Agent"] = Tools.GetAgentString();
requester.Headers["Accept-Language"] = "en-US";
requester.Headers["Accept-Charset"] = "ISO-8859-1";
requester.Headers["Content-Type"] = "text/html; charset=UTF-8";
var config = Configuration.Default
.WithRequesters(handler)
.With(requester)
.WithTemporaryCookies()
.WithDefaultLoader();
var context = BrowsingContext.New(config);
var doc = await context.OpenAsync(Url);
When I added the Header requester, it stopped the proxy handler from working. I know there is some conflict between the .WithRequesters() and the .With() but I cannot locate the proper syntax for doing both in the same request.
Thanks.
Yeah I am not sure why you use both.
The DefaultHttpRequester is the requester from AngleSharp and uses WebClient underneath
WithRequesters comes from AngleSharp.Io and adds all available requesters (incl. HttpClientRequester)
WithDefaultLoader puts a loader into the config and registers the default HTTP requester, if no other requester has been registered yet
I think what you want to do should be actually done using the HttpClientRequester directly.
var handler = new HttpClientHandler
{
Proxy = new WebProxy(ProxyMan.GetProxy()),
UseProxy = true,
PreAuthenticate = true,
UseDefaultCredentials = false,
UseCookies = false,
AllowAutoRedirect = false
};
var client = new HttpClient(handler);
client.DefaultRequestHeaders.Append("User-Agent", Tools.GetAgentString());
// ... and the others if you want, even though `content-type` etc. should / will be determined by AngleSharp
var config = Configuration.Default
.With(new HttpClientRequester(client))
.WithTemporaryCookies()
.WithDefaultLoader();
var context = BrowsingContext.New(config);
var doc = await context.OpenAsync(Url);
Related
I have a Encoded string like this:
https://xx.yyy.ir/xx/ff/addUser?name=%d8%b3%d9%84%d8%a7%d9%85
But when I use Uri to convert it to a URL and send it
result = "https://xx.yyy.ir/xx/ff/addUser?name=%d8%b3%d9%84%d8%a7%d9%85"
var client = new HttpClient
{
BaseAddress = new Uri(result.ToString()),
};
var response = await client.GetAsync("");
it send this request :
https://xx.yyy.ir/xx/ff/addUser?name=سلام
why this happen? how to prevent from this?
This is what's causing your problem: new Uri(result.ToString())
Let's try to do this in a proper manner and see what happens.
var builder = new UriBuilder("https://xx.yyy.ir/xx/ff/addUser") { Port = -1 };
var query = HttpUtility.ParseQueryString(builder.Query);
query["name"] = "سلام";
builder.Query = query.ToString();
using var httpClient = new HttpClient();
var response = await client.GetAsync(builder.ToString());
builder.ToString() returns https://xx.yyy.ir/xx/ff/addUser?name=%d8%b3%d9%84%d8%a7%d9%85
So basically, the above code boils down to this:
using var httpClient = new HttpClient();
var response = await client.GetAsync("https://xx.yyy.ir/xx/ff/addUser?name=%d8%b3%d9%84%d8%a7%d9%85");
Tested and verified on my computer.
I'm trying to send some data using the example in the page of onesignal
var request = WebRequest.Create("https://onesignal.com/api/v1/notifications") as HttpWebRequest;
request.KeepAlive = true;
request.Method = "POST";
request.ContentType = "application/json; charset=utf-8";
request.Headers.Add("authorization", "Basic xxx");
var obj = new
{
app_id = "xxx",
contents = new { en = "English Message" },
included_segments = new string[] { "Active Users" }
};
var param = JsonConvert.SerializeObject(obj);
byte[] byteArray = Encoding.UTF8.GetBytes(param);
This coded works fine, but I'm using Flurl to make a request to onesignal like this:
var body = new
{
app_id = "xxx",
contents = new
{
es = "Mensaje prueba"
},
included_segments = new string[] { "All" }
};
string param = JsonConvert.SerializeObject(body);
var content = new System.Net.Http.ByteArrayContent(Encoding.UTF8.GetBytes(param));
var response = await new Flurl.Url(urlbase)
.AppendPathSegment("notifications")
.WithHeader("Content-Type", "application/json; charset=utf-8")
.WithHeader("Authorization", "Basic xxx")
.PostAsync(content)
.ReceiveString();
but I'm getting the "Bad request". Please someone could help to point how to make the same call with Flurl?
As mentioned in the first comment, you're doing more work than you need to. Flurl will serialize body for you, so remove these lines:
string param = JsonConvert.SerializeObject(body);
var content = new System.Net.Http.ByteArrayContent(Encoding.UTF8.GetBytes(param));
And post body directly using PostJsonAsync:
var response = await urlbase
...
.PostJsonAsync(body)
.ReceiveString();
I'm trying to write a middleware for batch requests i .net core 2.0.
So far the I have splitted the request, pipe each request on to the controllers.
The controllers return value, but for some reason the response on the created context that I parse to the controllers keeps giving me a NullStream in the body, so I think that there is something that I miss in my setup.
The code looks like this:
var json = await streamHelper.StreamToJson(context.Request.Body);
var requests = JsonConvert.DeserializeObject<IEnumerable<RequestModel>>(json);
var responseBody = new List<ResponseModel>();
foreach (var request in requests)
{
var newRequest = new HttpRequestFeature
{
Body = request.Body != null ? new MemoryStream(Encoding.ASCII.GetBytes(request.Body)) : null,
Headers = context.Request.Headers,
Method = request.Method,
Path = request.RelativeUrl,
PathBase = string.Empty,
Protocol = context.Request.Protocol,
Scheme = context.Request.Scheme,
QueryString = context.Request.QueryString.Value
};
var newRespone = new HttpResponseFeature();
var requestLifetimeFeature = new HttpRequestLifetimeFeature();
var features = CreateDefaultFeatures(context.Features);
features.Set<IHttpRequestFeature>(newRequest);
features.Set<IHttpResponseFeature>(newRespone);
features.Set<IHttpRequestLifetimeFeature>(requestLifetimeFeature);
var innerContext = _factory.Create(features);
await _next(innerContext);
var responseJson = await streamHelper.StreamToJson(innerContext.Response.Body);
I'm not sure what it is I'm missing in the setup, since innerContext.Response.Body isn't set.
One of the endpoints that I use for testing and that gets hit looks like this
[Route("api/[controller]")]
public class ValuesController : Controller
{
// GET api/values
[HttpGet]
public IEnumerable<string> Get()
{
return new string[] { "value1", "value2" };
}
}
I found the error, or two errors for it to work.
First I had to change my newResponse to
var newRespone = new HttpResponseFeature{ Body = new MemoryStream() };
Since HttpResponseFeature sets Body to Stream.Null in the constructor.
When that was done, then Body kept giving an empty string when trying to read it. That was fixed by setting the Position to Zero like
innerContext.Response.Body.Position = 0;
How to add the header in the WebRequest .
HttpWebRequest tRequest = (HttpWebRequest)WebRequest.Create("https://fcm.googleapis.com/fcm/send");
tRequest.Method = "POST";
tRequest.ContentType = "application/json";
var data = new
{
to = devicesId,
notification = new
{
body = "Fcm Test Notification",
title = "Test FCM",
sound = "Enabled"
},
priority = "high"
};
tRequest.Headers["Authorization: key={0}"] = appId;
tRequest.Headers["Sender: id={0}"] = senderId;
i need to add the header to create the web Request.
Thanks
Missing Add in HttpWebRequest is because have missing ISerializable in CoreFx (so you can not simply tRequest.Headers.Add("name", "value");). They working on it and you can follow at https://github.com/dotnet/corefx/issues/12669
I wish to get the data of Google analytics via service account.
When I launch first time the application, everything works correctly and I have access to the data. But When I launch second time the application I have the following error which appears: " the system cannot find the path specified ". Have you an idea? I thought it can be a lock.
This is my source code:
public static String GetAccessToken(string clientIdEMail, string keyFilePath, String scope)
{
// certificate
var certificate = new X509Certificate2(keyFilePath, "notasecret", X509KeyStorageFlags.MachineKeySet | X509KeyStorageFlags.Exportable);
// header
var header = new { typ = "JWT", alg = "RS256" };
// claimset
var times = GetExpiryAndIssueDate();
var claimset = new
{
iss = clientIdEMail,
scope = scope,
aud = "https://accounts.google.com/o/oauth2/token",
iat = times[0],
exp = times[1],
};
JavaScriptSerializer ser = new JavaScriptSerializer();
// encoded header
var headerSerialized = ser.Serialize(header);
var headerBytes = Encoding.UTF8.GetBytes(headerSerialized);
var headerEncoded = Convert.ToBase64String(headerBytes);
// encoded claimset
var claimsetSerialized = ser.Serialize(claimset);
var claimsetBytes = Encoding.UTF8.GetBytes(claimsetSerialized);
var claimsetEncoded = Convert.ToBase64String(claimsetBytes);
// input
var input = headerEncoded + "." + claimsetEncoded;
var inputBytes = Encoding.UTF8.GetBytes(input);
// signiture
var rsa = certificate.PrivateKey as RSACryptoServiceProvider;
var cspParam = new CspParameters
{
KeyContainerName = rsa.CspKeyContainerInfo.KeyContainerName,
KeyNumber = rsa.CspKeyContainerInfo.KeyNumber == KeyNumber.Exchange ? 1 : 2,
Flags = CspProviderFlags.UseMachineKeyStore
};
var aescsp = new RSACryptoServiceProvider(1024,cspParam) { PersistKeyInCsp = false };
var signatureBytes = aescsp.SignData(inputBytes, "SHA256");
var signatureEncoded = Convert.ToBase64String(signatureBytes);
// jwt
var jwt = headerEncoded + "." + claimsetEncoded + "." + signatureEncoded;
var client = new WebClient();
client.Encoding = Encoding.UTF8;
var uri = "https://accounts.google.com/o/oauth2/token";
var content = new NameValueCollection();
content["assertion"] = jwt;
content["grant_type"] = "urn:ietf:params:oauth:grant-type:jwt-bearer";
string response = Encoding.UTF8.GetString(client.UploadValues(uri, "POST", content));
JsonGoogleResponse result = (ser.Deserialize<JsonGoogleResponse>(response));
return result.access_token;
}
And this is the stack:
à System.Security.Cryptography.CryptographicException.ThrowCryptogaphicException(Int32 hr)
à System.Security.Cryptography.SafeProvHandle._FreeCSP(IntPtr pProvCtx)
à System.Security.Cryptography.SafeProvHandle.ReleaseHandle()
à System.Runtime.InteropServices.SafeHandle.InternalFinalize()
à System.Runtime.InteropServices.SafeHandle.Dispose(Boolean disposing)
à System.Runtime.InteropServices.SafeHandle.Finalize()
If you are running in IIS, you need to set "Load User Profile" to True in the application pool's advanced settings to be able to load a cert by filename & password.
So, I just had the exact same problem. I tried to solve it for almost 4 hours.
Problem was in passed path to key. Because I used the code from Google sample console application, where the path was just "key.p12" and the key was in the same directory as the exe file.
And when I wanted to create MVC application, I did not realize, that root of virtual server path can not be called just like "key.p12".
SOLUTION
Double check the path to the key. If it is MVC application (or another ASP web), then add the key file to the root and in code call the key by using Server.MapPath("key.p12").
I just had the same issue, in my case it was a space in the path. I have no idea why, but when I put the p12 file on c:\ root, it's working...