I have a class library to perform Rest API calls built in 4.6.1 framework. I have used System.Net.Http V4 HttpClient for managing calls. This library works in normal dotnet apps. Recently I tried it for a DotNet Core app, it failed with security error. Later I modified library as suggested in post. It had some progress but the app fails with error as shown below
Error message: An error occurred while sending the request.
Innerexception message: A security error occurred.
StackTrace:
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
at System.Net.Http.WinHttpHandler.d__105.MoveNext()
Library Code:
private readonly HttpClient _client = null;
private ICredentials somecred;
public Program()
{
HttpClientHandler clientHandler = new HttpClientHandler { Credentials = somecred, UseDefaultCredentials = false };
_client=new HttpClient(new MessageHandler(clientHandler, TimeSpan.FromSeconds(1), 1), false);
_client.Timeout = TimeSpan.FromSeconds(90);
_client.BaseAddress = new Uri("https://somedomain.com/");
ServicePointManager.ServerCertificateValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
ServicePointManager.DefaultConnectionLimit = 10;
ServicePointManager.Expect100Continue = true;
}
public async Task<IOperationResponse> GetData()
{
ServicePointManager.ServerCertificateValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
using (HttpResponseMessage httpResponse =
await _client.SendAsync(new HttpRequestMessage(HttpMethod.Head, "api/getdata"), HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false)
)
{
if (httpResponse != null) { ... }
}
}
internal class MessageHandler : DelegatingHandler
{
public MessageHandler(HttpMessageHandler innerHandler, TimeSpan retryInterval, int retryCount) : base(innerHandler)
{
}
protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
HttpResponseMessage response = null;
//!! security error in below line
response = await base.SendAsync(request, cancellationToken).ConfigureAwait(false);
}
}
Please let me know, what needs to be done for this library to run for DotNet Core apps. Thanks.
I have tested your code locally with a core project and an external 4.5 library and the only way I could get the security error to trigger was if the target site did not have the correct level of security (i.e was not correctly setup with SSL). When targeting a site that had SSL configured correctly the call went through.
Ensure that your target has SSL setup or target another known working SSL site to test against to see if you still encounter the issue.
I made following code change as per my colleague suggestion and that fixed it.
//original code
_client.BaseAddress = new Uri("https://xyz.abc.somedomain.com/");
//fix code
_client.BaseAddress = new Uri("https://abc.somedomain.com/");
_client.DefaultRequestHeaders.Host = "xyz.abc.somedomain.com";
I'm marking this as answer, though i do not completely understand this behavior.
Related
I am trying to implement the certificate authentication in .net core API(Server/target) and this API will be invoked in to another API(Client) .Here is the piece of code of client api which makes request to server/target api.But I'm facing an error on the server/target api .I'm running these two services from local and both certificates have already installed
Client side controller logic
[HttpGet]
public async Task<List<WeatherForecast>> Get()
{
List<WeatherForecast> weatherForecastList = new List<WeatherForecast>();
X509Certificate2 clientCert = Authentication.GetClientCertificate();
if (clientCert == null)
{
HttpActionContext actionContext = null;
actionContext.Response = new HttpResponseMessage(System.Net.HttpStatusCode.Forbidden)
{
ReasonPhrase = "Client Certificate Required"
};
}
HttpClientHandler requestHandler = new HttpClientHandler();
requestHandler.ClientCertificates.Add(clientCert);
requestHandler.ServerCertificateCustomValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;
HttpClient client = new HttpClient(requestHandler)
{
BaseAddress = new Uri("https://localhost:11111/ServerAPI")
};
client.DefaultRequestHeaders
.Accept
.Add(new MediaTypeWithQualityHeaderValue("application/xml"));//ACCEPT head
using (var httpClient = new HttpClient())
{
//httpClient.DefaultRequestHeaders.Accept.Clear();
var request = new HttpRequestMessage()
{
RequestUri = new Uri("https://localhost:44386/ServerAPI"),
Method = HttpMethod.Get,
};
request.Headers.Add("X-ARR-ClientCert", clientCert.GetRawCertDataString());
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));//ACCEPT head
//using (var response = await httpClient.GetAsync("https://localhost:11111/ServerAPI"))
using (var response = await httpClient.SendAsync(request))
{
if (response.StatusCode == System.Net.HttpStatusCode.OK)
{
string apiResposne = await response.Content.ReadAsStringAsync();
weatherForecastList = JsonConvert.DeserializeObject<List<WeatherForecast>>(apiResposne);
}
}
}
return weatherForecastList;
}
authentication class
public static X509Certificate2 GetClientCertificate()
{
X509Store userCaStore = new X509Store(StoreName.TrustedPeople, StoreLocation.CurrentUser);
try
{
string str_API_Cert_Thumbprint = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
userCaStore.Open(OpenFlags.ReadOnly);
X509Certificate2Collection certificatesInStore = userCaStore.Certificates;
X509Certificate2Collection findResult = certificatesInStore.Find(X509FindType.FindByThumbprint, str_API_Cert_Thumbprint, false);
X509Certificate2 clientCertificate = null;
if (findResult.Count == 1)
{
clientCertificate = findResult[0];
if(System.DateTime.Today >= System.Convert.ToDateTime(clientCertificate.GetExpirationDateString()))
{
throw new Exception("Certificate has already been expired.");
}
else if (System.Convert.ToDateTime(clientCertificate.GetExpirationDateString()).AddDays(-30) <= System.DateTime.Today)
{
throw new Exception("Certificate is about to expire in 30 days.");
}
}
else
{
throw new Exception("Unable to locate the correct client certificate.");
}
return clientCertificate;
}
catch (Exception ex)
{
throw;
}
finally
{
userCaStore.Close();
}
}
Server/target api code
[HttpGet]
public IEnumerable<WeatherForecast> Getcertdata()
{
IHeaderDictionary headers = base.Request.Headers;
X509Certificate2 clientCertificate = null;
string certHeaderString = headers["X-ARR-ClientCert"];
if (!string.IsNullOrEmpty(certHeaderString))
{
//byte[] bytes = Encoding.ASCII.GetBytes(certHeaderString);
//byte[] bytes = Convert.FromBase64String(certHeaderString);
//clientCertificate = new X509Certificate2(bytes);
clientCertificate = new X509Certificate2(WebUtility.UrlDecode(certHeaderString));
var serverCertificate = new X509Certificate2(Path.Combine("abc.pfx"), "pwd");
if (clientCertificate.Thumbprint == serverCertificate.Thumbprint)
{
//Valida Cert
}
}
var rng = new Random();
return Enumerable.Range(1, 5).Select(index => new WeatherForecast
{
Date = DateTime.Now.AddDays(index),
TemperatureC = rng.Next(-20, 55),
Summary = Summaries[rng.Next(Summaries.Length)]
}).ToArray();
//return new List<WeatherForecast>();
}
You have much more problems here, the code is significantly flawed and insecure in various ways. Let's explain each issue:
HttpClient in using clause in client side controller logic
Although you expect to wrap anything that implements IDisposable in using statement. However, it is not really the case with HttpClient. Connections are not closed immediately. And with every request to client controller action, a new connection is established to remote endpoint, while previous connections sit in TIME_WAIT state. Under certain constant load, your HttpClient will exhaust TCP port pool (which is limited) and any new attempt to create a new connection will throw an exception. Here are more details on this problem: You're using HttpClient wrong and it is destabilizing your software
Microsoft recommendation is to re-use existing connections. One way to do this is to Use IHttpClientFactory to implement resilient HTTP requests. Microsoft article talks a bit about this problem:
Though this class implements IDisposable, declaring and instantiating
it within a using statement is not preferred because when the
HttpClient object gets disposed of, the underlying socket is not
immediately released, which can lead to a socket exhaustion problem.
BTW, you have created a client variable, but do not use it in any way.
Ignore certificate validation problems
The line:
requestHandler.ServerCertificateCustomValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;
make you vulnerable to MITM attack.
you are doing client certificate authentication wrong
The line:
request.Headers.Add("X-ARR-ClientCert", clientCert.GetRawCertDataString());
It is not the proper way how to do client cert authentication. What you literally doing is passing certificate's public part to server. That's all. You do not prove private key possession which is required to authenticate you. The proper way to do so is:
requestHandler.ClientCertificates.Add(clientCert);
This will force client and server to perform proper client authentication and check if you possess the private key for certificate you pass (it is done in TLS handshake automatically). If you have ASP.NET on server side, then you read it this way (in controller action):
X509Certificate2 clientCert = Request.HttpContext.Connection.ClientCertificate
if (clientCert == null) {
return Unauthorized();
}
// perform client cert validation according server-side rules.
Non-standard cert store
In authentication class you open StoreName.TrustedPeople store, while normally it should be StoreName.My. TrustedPeople isn't designed to store certs with private key. It isn't a functional problem, but it is bad practice.
unnecessary try/catch clause in authentication class
If you purposely throw exceptions in method, do not use try/catch. In your case you simply rethrow exception, thus you are doing a double work. And this:
throw new Exception("Certificate is about to expire in 30 days.");
is behind me. Throwing exception on technically valid certificate? Really?
server side code
As said, all this:
IHeaderDictionary headers = base.Request.Headers;
X509Certificate2 clientCertificate = null;
string certHeaderString = headers["X-ARR-ClientCert"];
if (!string.IsNullOrEmpty(certHeaderString))
{
//byte[] bytes = Encoding.ASCII.GetBytes(certHeaderString);
//byte[] bytes = Convert.FromBase64String(certHeaderString);
//clientCertificate = new X509Certificate2(bytes);
clientCertificate = new X509Certificate2(WebUtility.UrlDecode(certHeaderString));
var serverCertificate = new X509Certificate2(Path.Combine("abc.pfx"), "pwd");
if (clientCertificate.Thumbprint == serverCertificate.Thumbprint)
{
//Valida Cert
}
}
must be replaced with:
X509Certificate2 clientCert = Request.HttpContext.Connection.ClientCertificate
if (clientCert == null) {
return Unauthorized();
}
// perform client cert validation according server-side rules.
BTW:
var serverCertificate = new X509Certificate2(Path.Combine("abc.pfx"), "pwd");
if (clientCertificate.Thumbprint == serverCertificate.Thumbprint)
{
//Valida Cert
}
This is another disaster in your code. You are loading the server certificate from PFX just to compare their thumbprints? So, you suppose that client will have a copy of server certificate? Client and server certificates must not be the same. Next thing is you are generating a lot of copies of server certificate's private key files. More private key files you generate, the slower the process is and you just generate a lot of garbage. More details on this you can find in my blog post: Handling X509KeyStorageFlags in applications
I'm having difficulties sending a certificate using HttpClientHandler because the certificate simply won't appear on Server's request. The certificate has the proper EKU for server and client authentication, and Key Usage of "Digital Signature". #davidsh regarded the issue 26531 for the lack of logging that HttpClient had but running my project in Visual Studio (with logs set to Trace and using dotnet 3.1.401) no output error came out. I'm not very familiar at all with logman but I ran it when the issue supposed to happen as I executed my code and nothing from the log stood out indicating what the problem could be. Running out of options to test the code I attempted to add a certificate without the private key on the client request to see if the httpClientHandler.ClientCertificates.Add ... would throw any error saying something like "You need a certificate with private key to sign your request", shouldn't it say anything?
On client:
services.AddHttpClient<ILetterManClient, LetterManClient.LetterManClient>()
.ConfigureHttpClient(client =>
{
client.BaseAddress = new Uri(configuration.GetValue<string>("Microservices:LetterManAPI"));
})
.ConfigurePrimaryHttpMessageHandler(() =>
{
HttpClientHandler httpClientHandler = new HttpClientHandler();
httpClientHandler.ServerCertificateCustomValidationCallback = ValidateServiceCertficate;
httpClientHandler.ClientCertificateOptions = ClientCertificateOption.Manual;
clientCertificate = new X509Certificate2("client_cert.pfx", "developer");
httpClientHandler.ClientCertificates.Add(clientCertificate);
return httpClientHandler;
});
On server:
public class ValidateClientCertificates : TypeFilterAttribute
{
public ValidateClientCertificates() : base(typeof(ValidateClientCertificatesImpl))
{
}
private class ValidateClientCertificatesImpl : IAsyncAuthorizationFilter
{
X509Certificate2 clientCertificate;
public ValidateClientCertificatesImpl(IConfiguration configuration, IWebHostEnvironment webHostEnvironment)
{
clientCertificate = new X509Certificate2("client_cert.crt");
}
public async Task OnAuthorizationAsync(AuthorizationFilterContext context)
{
var certificate = await context.HttpContext.Connection.GetClientCertificateAsync();
if ((certificate == null) || (!certificate.Thumbprint.Equals(clientCertificate.Thumbprint)))
{
context.Result = new UnauthorizedObjectResult("");
return;
}
}
}
}
Side note:
I've been also trying to debug my project using code compiled from corefx repo to see what's going but Visual Studio insists reference the code from local installed sdk instead of the project from corefx that it's referencing it but this is another issue.
I've created this project that simulates the issue. It creates the certificates and it has two projects with one service and another client implemented.
Any help will be very welcomed.
These are the guidelines for Kestrel to require Client certificate but it assumes that the CA is installed in the machine otherwise you have to specify the client certificate directly when configuring Kestrel server as follows:
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
webBuilder.ConfigureKestrel(o =>
{
o.ConfigureHttpsDefaults(o => {
o.ClientCertificateMode = ClientCertificateMode.RequireCertificate;
o.ClientCertificateValidation = ValidateClientCertficate;
});
});
});
public static Func<X509Certificate, X509Chain, SslPolicyErrors, bool> ValidateClientCertficate =
delegate (X509Certificate serviceCertificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
X509Certificate2 clientCertificate;
clientCertificate = new X509Certificate2("client.crt");
if (serviceCertificate.GetCertHashString().Equals(clientCertificate.Thumbprint))
{
return true;
}
return false;
};
Unfortunately, you can't require Client certificates for a specific route as I intended.
I have created two Web API in Asp.net Core one is Wrapper Service to be deployed on DMZ Server and other is core service which have access to our DB Non DMZ. The problem i am facing is that i am unable to communicate between two services.
Both are running on local machine through dockers. When i try to hit core service running on url: https://localhost:56788/Rewards thorugh HttpClient/WebRequest i got message
No connection could be made because the target machine actively refused it.
DMZ Controller Logic
[Route("[controller]")]
[ApiController]
public class RewardsController : ControllerBase
{
[HttpGet]
public string Get()
{
string response = string.Empty;
//using (var client = new HttpClient())
//{
// client.BaseAddress = new Uri("http://localhost:5000/");
// //HTTP GET
// var responseTask = client.GetAsync("Rewards");
// responseTask.Wait();
// var result = responseTask.Result;
// if (result.IsSuccessStatusCode)
// {
// response = result.ToString();
// }
// else //web api sent error response
// {
// //log response status here..
// response = "Error";
// }
//}
string sURL = "https://localhost:56788/Rewards";
WebRequest wrPostURL = WebRequest.Create(sURL);
wrPostURL.Method = "GET";
ServicePointManager.ServerCertificateValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;
using (HttpWebResponse webresponse = wrPostURL.GetResponse() as HttpWebResponse)
{
Encoding enc = System.Text.Encoding.GetEncoding("utf-8");
StreamReader loResponseStream = new StreamReader(webresponse.GetResponseStream(), enc);
var jsonResponse = loResponseStream.ReadToEnd();
loResponseStream.Close();
webresponse.Close();
}
return response;
}
}
NON-DMZ Controller Logic
[Route("[controller]")]
[ApiController]
public class RewardsController : ControllerBase
{
[HttpGet]
public string Get()
{
return "Hello";
}
}
Update
This problem is due to dockers. When i deploy both APIs to IIS then it will work fine but i have to do it with Dockers
The problem occurs due to the fact that i was calling localhost to communicate with other API running on another container. When we call localhost the first container start looking for the service that is running on same container. By using VM/Local Machine IP instead of localhost the problem can be avoided.
I am creating a Middleware for logging Request and Response, as I am new to this thing, its not clear how to implement it. As I checked some examples I have written the following thing
public RequestResponseLogging(RequestDelegate next, ILogger<RequestResponseLogging> logger)
{
if (next == null) throw new ArgumentNullException(nameof(next));
_next = next;
_logger = logger;
}
I found the below example, this is logging whole request and response, but i need to log some of the properties and i need to log it in Database.
public async Task Invoke(HttpContext context)
{
_logger.LogInformation(await FormatRequest(context.Request));
var originalBodyStream = context.Response.Body;
using (var responseBody = new MemoryStream())
{
context.Response.Body = responseBody;
await _next(context);
_logger.LogInformation(await FormatResponse(context.Response));
await responseBody.CopyToAsync(originalBodyStream);
}
}
I tried something like this:
RequestData requestData = new RequestData();
requestData.UserName = context.Request.Headers[UserKey];
requestData.Token = context.Request.Headers[TokenKey];
_logger.LogInformation("Request",requestData);
await _next(context);
ResponseData responseData = new ResponseData();
responseData.Code = context.Response.Headers["Code"];
responseData.Descirption = context.Response.Headers["Desciption"];
_logger.LogInformation("Response", responseData);
not sure if this is the right way, do I need to add EF in my middleware? All the examples I have seen for nlog are asking to configure it in startup file, but as I need to implement it in Middleware I dont have any startup file its a class library project not API, from API i am calling my Middleware.
Can anybody help me??
I'm trying to make HTTP requests via a WebProxy in a .net core 2.0 web application. The code I've got works fine in .net framework so I know (believe) its not an environmental issue. I've also tried to make the request using both HttpWebRequest and HttpClient but both mechanisms always result in 407 (Proxy Authentication Required) http error in .net core. Its as if in .net core the credentials I'm supplying are always being ignored.
Here is the code I've been using:
public void Test()
{
var cred = new NetworkCredential("xxxxx", "yyyyyy");
var proxyURI = new Uri("http://xxx.xxx.xxx.xxx:80");
var destinationURI = new Uri("http://www.bbc.co.uk");
WebProxy proxy = new WebProxy(proxyURI, false) { UseDefaultCredentials = false, Credentials = cred };
MakeProxyRequestViaHttpWebRequest(proxy, destinationURI);
MakeProxyRequestViaHttpClient(proxy, destinationURI);
}
private void MakeProxyRequestViaHttpClient(WebProxy proxy, Uri destination)
{
HttpClientHandler handler = new HttpClientHandler()
{
Proxy = proxy,
UseProxy = true,
PreAuthenticate = true,
UseDefaultCredentials = false
};
HttpClient client = new HttpClient(handler);
HttpResponseMessage response = client.GetAsync(destination).Result;
if (response.IsSuccessStatusCode)
{
HttpContent content = response.Content;
string htmlData = content.ReadAsStringAsync().Result;
}
else
{
HttpStatusCode code = response.StatusCode;
}
}
private void MakeProxyRequestViaHttpWebRequest(WebProxy proxy, Uri destination)
{
HttpWebRequest req = HttpWebRequest.Create(destination) as HttpWebRequest;
req.UseDefaultCredentials = false;
req.Proxy = proxy;
req.PreAuthenticate = true;
using (WebResponse response = req.GetResponse())
{
using (StreamReader responseStream = new StreamReader(response.GetResponseStream()))
{
string htmlData = responseStream.ReadToEnd();
}
}
}
I've tried the following in .net core but the result is always 407:
Run the code in a console app
Implement IWebProxy and use that as the proxy
Set default values for other properties on WebProxy, HttpClient, etc. (removed on the example above because it works fine on .net standard)
I've run out of ideas and things to try. I have the following questions:
Does the code need to be different between .net core vs .net framework
Are there additional things that need to go into appsettings.json (ie. the config that would have gone into web.config)
Is there any additional configuration code required in Startup.cs
Should I be looking to use an external library
How would I trouble shoot what the issue is? Fiddler doesn't seem to be helping but then I haven't looked to hard at configuring it.