I have WCF REST web service hosted by IIS, it works on HTTPS, I generate Certificate on IIS and assign Https to a port
I generate cer through IE browser. I create a test application and regardless Add a client certificate or not or even add a wrong certificate the connection take place and a I get correct response. I am wondering how the message was decrypted if there is no certificate sent.
Either the destination is not secured or I misunderstand the whole thing.
also
The error I have from the callback "CheckValidationResult()" is either
CertCN_NO_MATCH = 0x800B010F
or
"Unknown Certificate Problem" , the certificateProblem (parameter of CheckValidationResult) is 0 for this case
What is CertCN_NO_MATCH eror, what is CN?
See code below.
ServicePointManager.CertificatePolicy = new CertPolicy();
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(String.Format("https://{0}/uri", ip));
//request.ClientCertificates.Add(new X509Certificate("D:\\ThePubKey.cer"));
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
using (StreamWriter stream = new StreamWriter(request.GetRequestStream()))
{
stream.Write("RequestType=CheckStatus&ReportType=Fulfillment&ReportID=5");
}
using (StreamReader stream = new StreamReader(request.GetResponse().GetResponseStream()))
{
Response.ContentType = "text/xml";
Response.Output.Write(stream.ReadToEnd());
Response.End();
}
class CertPolicy : ICertificatePolicy
{
public enum CertificateProblem : uint
{
CertEXPIRED = 0x800B0101,
CertVALIDITYPERIODNESTING = 0x800B0102,
CertROLE = 0x800B0103,
CertPATHLENCONST = 0x800B0104,
CertCRITICAL = 0x800B0105,
CertPURPOSE = 0x800B0106,
CertISSUERCHAINING = 0x800B0107,
CertMALFORMED = 0x800B0108,
CertUNTRUSTEDROOT = 0x800B0109,
CertCHAINING = 0x800B010A,
CertREVOKED = 0x800B010C,
CertUNTRUSTEDTESTROOT = 0x800B010D,
CertREVOCATION_FAILURE = 0x800B010E,
CertCN_NO_MATCH = 0x800B010F,
CertWRONG_USAGE = 0x800B0110,
CertUNTRUSTEDCA = 0x800B0112
}
public bool CheckValidationResult(ServicePoint srvPoint, X509Certificate certificate, WebRequest request, int certificateProblem)
{
// You can do your own certificate checking.
// You can obtain the error values from WinError.h.
// Return true so that any certificate will work with this sample.
String error = "";
using (StringWriter writer = new StringWriter())
{
writer.WriteLine("Certificate Problem with accessing " + request.RequestUri);
writer.Write("Problem code 0x{0:X8},", (int)certificateProblem);
writer.WriteLine(GetProblemMessage((CertificateProblem)certificateProblem));
error = writer.ToString();
}
return true;
}
private String GetProblemMessage(CertificateProblem Problem)
{
String ProblemMessage = "";
CertificateProblem problemList = new CertificateProblem();
String ProblemCodeName = Enum.GetName(problemList.GetType(), Problem);
if (ProblemCodeName != null)
ProblemMessage = ProblemMessage + "-Certificateproblem:" +
ProblemCodeName;
else
ProblemMessage = "Unknown Certificate Problem";
return ProblemMessage;
}
}
I've just replied to this similar question (in Java).
CN is the "Common Name". It ought to be the hostname of the server to which you're connecting (unless it's in the subject alternative name). I guess from your code sample that you're using the IP address directly. In this case, the CN should be that IP address (it tends to be better to use a hostname rather than an IP address). See RFC 2818 (sec 3.1) for the specifications.
Note that the CN or subject alternative name is from the point of view of the client, so if you connect to https://some.example.com/, then the name in the cert should be some.example.com, if you connect to https://localhost/, then the name in the cert should be localhost, even if some.example.com and localhost may be the same server effectively.
(I guess that by default, IIS might generate a certificate for the external name, but you'd have to look at the certificate to know; this should be visible in the certificate properties somewhere.)
Related
I am trying to implement the certificate authentication in .net core API(Server/target) and this API will be invoked in to another API(Client) .Here is the piece of code of client api which makes request to server/target api.But I'm facing an error on the server/target api .I'm running these two services from local and both certificates have already installed
Client side controller logic
[HttpGet]
public async Task<List<WeatherForecast>> Get()
{
List<WeatherForecast> weatherForecastList = new List<WeatherForecast>();
X509Certificate2 clientCert = Authentication.GetClientCertificate();
if (clientCert == null)
{
HttpActionContext actionContext = null;
actionContext.Response = new HttpResponseMessage(System.Net.HttpStatusCode.Forbidden)
{
ReasonPhrase = "Client Certificate Required"
};
}
HttpClientHandler requestHandler = new HttpClientHandler();
requestHandler.ClientCertificates.Add(clientCert);
requestHandler.ServerCertificateCustomValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;
HttpClient client = new HttpClient(requestHandler)
{
BaseAddress = new Uri("https://localhost:11111/ServerAPI")
};
client.DefaultRequestHeaders
.Accept
.Add(new MediaTypeWithQualityHeaderValue("application/xml"));//ACCEPT head
using (var httpClient = new HttpClient())
{
//httpClient.DefaultRequestHeaders.Accept.Clear();
var request = new HttpRequestMessage()
{
RequestUri = new Uri("https://localhost:44386/ServerAPI"),
Method = HttpMethod.Get,
};
request.Headers.Add("X-ARR-ClientCert", clientCert.GetRawCertDataString());
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));//ACCEPT head
//using (var response = await httpClient.GetAsync("https://localhost:11111/ServerAPI"))
using (var response = await httpClient.SendAsync(request))
{
if (response.StatusCode == System.Net.HttpStatusCode.OK)
{
string apiResposne = await response.Content.ReadAsStringAsync();
weatherForecastList = JsonConvert.DeserializeObject<List<WeatherForecast>>(apiResposne);
}
}
}
return weatherForecastList;
}
authentication class
public static X509Certificate2 GetClientCertificate()
{
X509Store userCaStore = new X509Store(StoreName.TrustedPeople, StoreLocation.CurrentUser);
try
{
string str_API_Cert_Thumbprint = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
userCaStore.Open(OpenFlags.ReadOnly);
X509Certificate2Collection certificatesInStore = userCaStore.Certificates;
X509Certificate2Collection findResult = certificatesInStore.Find(X509FindType.FindByThumbprint, str_API_Cert_Thumbprint, false);
X509Certificate2 clientCertificate = null;
if (findResult.Count == 1)
{
clientCertificate = findResult[0];
if(System.DateTime.Today >= System.Convert.ToDateTime(clientCertificate.GetExpirationDateString()))
{
throw new Exception("Certificate has already been expired.");
}
else if (System.Convert.ToDateTime(clientCertificate.GetExpirationDateString()).AddDays(-30) <= System.DateTime.Today)
{
throw new Exception("Certificate is about to expire in 30 days.");
}
}
else
{
throw new Exception("Unable to locate the correct client certificate.");
}
return clientCertificate;
}
catch (Exception ex)
{
throw;
}
finally
{
userCaStore.Close();
}
}
Server/target api code
[HttpGet]
public IEnumerable<WeatherForecast> Getcertdata()
{
IHeaderDictionary headers = base.Request.Headers;
X509Certificate2 clientCertificate = null;
string certHeaderString = headers["X-ARR-ClientCert"];
if (!string.IsNullOrEmpty(certHeaderString))
{
//byte[] bytes = Encoding.ASCII.GetBytes(certHeaderString);
//byte[] bytes = Convert.FromBase64String(certHeaderString);
//clientCertificate = new X509Certificate2(bytes);
clientCertificate = new X509Certificate2(WebUtility.UrlDecode(certHeaderString));
var serverCertificate = new X509Certificate2(Path.Combine("abc.pfx"), "pwd");
if (clientCertificate.Thumbprint == serverCertificate.Thumbprint)
{
//Valida Cert
}
}
var rng = new Random();
return Enumerable.Range(1, 5).Select(index => new WeatherForecast
{
Date = DateTime.Now.AddDays(index),
TemperatureC = rng.Next(-20, 55),
Summary = Summaries[rng.Next(Summaries.Length)]
}).ToArray();
//return new List<WeatherForecast>();
}
You have much more problems here, the code is significantly flawed and insecure in various ways. Let's explain each issue:
HttpClient in using clause in client side controller logic
Although you expect to wrap anything that implements IDisposable in using statement. However, it is not really the case with HttpClient. Connections are not closed immediately. And with every request to client controller action, a new connection is established to remote endpoint, while previous connections sit in TIME_WAIT state. Under certain constant load, your HttpClient will exhaust TCP port pool (which is limited) and any new attempt to create a new connection will throw an exception. Here are more details on this problem: You're using HttpClient wrong and it is destabilizing your software
Microsoft recommendation is to re-use existing connections. One way to do this is to Use IHttpClientFactory to implement resilient HTTP requests. Microsoft article talks a bit about this problem:
Though this class implements IDisposable, declaring and instantiating
it within a using statement is not preferred because when the
HttpClient object gets disposed of, the underlying socket is not
immediately released, which can lead to a socket exhaustion problem.
BTW, you have created a client variable, but do not use it in any way.
Ignore certificate validation problems
The line:
requestHandler.ServerCertificateCustomValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;
make you vulnerable to MITM attack.
you are doing client certificate authentication wrong
The line:
request.Headers.Add("X-ARR-ClientCert", clientCert.GetRawCertDataString());
It is not the proper way how to do client cert authentication. What you literally doing is passing certificate's public part to server. That's all. You do not prove private key possession which is required to authenticate you. The proper way to do so is:
requestHandler.ClientCertificates.Add(clientCert);
This will force client and server to perform proper client authentication and check if you possess the private key for certificate you pass (it is done in TLS handshake automatically). If you have ASP.NET on server side, then you read it this way (in controller action):
X509Certificate2 clientCert = Request.HttpContext.Connection.ClientCertificate
if (clientCert == null) {
return Unauthorized();
}
// perform client cert validation according server-side rules.
Non-standard cert store
In authentication class you open StoreName.TrustedPeople store, while normally it should be StoreName.My. TrustedPeople isn't designed to store certs with private key. It isn't a functional problem, but it is bad practice.
unnecessary try/catch clause in authentication class
If you purposely throw exceptions in method, do not use try/catch. In your case you simply rethrow exception, thus you are doing a double work. And this:
throw new Exception("Certificate is about to expire in 30 days.");
is behind me. Throwing exception on technically valid certificate? Really?
server side code
As said, all this:
IHeaderDictionary headers = base.Request.Headers;
X509Certificate2 clientCertificate = null;
string certHeaderString = headers["X-ARR-ClientCert"];
if (!string.IsNullOrEmpty(certHeaderString))
{
//byte[] bytes = Encoding.ASCII.GetBytes(certHeaderString);
//byte[] bytes = Convert.FromBase64String(certHeaderString);
//clientCertificate = new X509Certificate2(bytes);
clientCertificate = new X509Certificate2(WebUtility.UrlDecode(certHeaderString));
var serverCertificate = new X509Certificate2(Path.Combine("abc.pfx"), "pwd");
if (clientCertificate.Thumbprint == serverCertificate.Thumbprint)
{
//Valida Cert
}
}
must be replaced with:
X509Certificate2 clientCert = Request.HttpContext.Connection.ClientCertificate
if (clientCert == null) {
return Unauthorized();
}
// perform client cert validation according server-side rules.
BTW:
var serverCertificate = new X509Certificate2(Path.Combine("abc.pfx"), "pwd");
if (clientCertificate.Thumbprint == serverCertificate.Thumbprint)
{
//Valida Cert
}
This is another disaster in your code. You are loading the server certificate from PFX just to compare their thumbprints? So, you suppose that client will have a copy of server certificate? Client and server certificates must not be the same. Next thing is you are generating a lot of copies of server certificate's private key files. More private key files you generate, the slower the process is and you just generate a lot of garbage. More details on this you can find in my blog post: Handling X509KeyStorageFlags in applications
We use a WebMethod on our IIS Webservice, so that users can download a file.
Our client runs into an exception when connected to one of our customers webservices, because the key "Content-Length" cannot be found in the header (KeyNotFoundException). The method does work for all other customers.
The customer installed a fresh version of Windows Server 2016, one of my colleagues then installed the IIS roles and features. We double and triple checked: the configuration is the same as the one on our internal webservice and as far as we know as on all the webservices other customers run.
After debugging and searching on the internet for the past two days I found out that instead of the "Content-Length" header a header named "Transfer-Encoding" is send, value is "chunked".
It seems that this only occurs when we call the method via POST, but I'm not completely sure about that.
What we have tried so far:
Disabling chunked-encoding with this script: cscript adsutil.vbs set
/W3SVC/AspEnableChunkedEncoding "TRUE"
Disabling chunked-encoding via appcmd: appcmd set config /section:asp /enableChunkedEncoding:False
Setting system.webServer/asp/enableChunkedEncoding to false via the iis configuration manager of the server AND the site.
We restarted the whole machine after each of these steps.
IIS Webmethod Code (shortened):
[WebMethod]
public void Download(string uniqueID)
{
Byte[] data;
var path = GetPath(uniqueID);
data = File.ReadAllBytes(path);
if (data != null)
{
string sExtension = Path.GetExtension(path);
string MimeType = GetMIMEType(sExtension);
HttpContext.Current.Response.ClearContent();
HttpContext.Current.Response.ClearHeaders();
HttpContext.Current.Response.ContentType = "application/octet-stream";
HttpContext.Current.Response.AddHeader("Content-Disposition", "attachment;filename=" + path.Replace(" ", "_"));
HttpContext.Current.Response.AddHeader("Content-Type", MimeType);
HttpContext.Current.Response.AddHeader("Content-Length", data.Length.ToString());
HttpContext.Current.Response.BinaryWrite(data);
HttpContext.Current.Response.Flush();
HttpContext.Current.Response.SuppressContent = true;
HttpContext.Current.ApplicationInstance.CompleteRequest();
}
}
Client Code (shortened, written in Xamarin.Android as an example, same error occurs on iOS)
Stream stream = null;
Java.IO.DataOutputStream dos = null;
var urlConnection = CreateUrlConnection();
urlConnection.DoInput = true; // Allow Inputs
urlConnection.DoOutput = true; // Allow Outputs
urlConnection.RequestMethod = "POST";
urlConnection.SetRequestProperty("Content-Type", "application/x-www-form-urlencoded");
dos = new Java.IO.DataOutputStream(urlConnection.OutputStream);
string encodedParameters = "";
bool first = true;
foreach (Parameter param in parameters)
{
if (first)
first = false;
else
encodedParameters += "&";
encodedParameters += HttpUtility.UrlEncode(param.Name);
encodedParameters += "=";
encodedParameters += HttpUtility.UrlEncode(param.Value);
}
dos.WriteBytes(encodedParameters);
dos.Flush();
dos.Close();
stream = urlConnection.InputStream;
var header = urlConnection.HeaderFields;
var bytesToRead = int.Parse(header["Content-Length"][0]); // Exception gets thrown here
Exported Qlik Sense certificate using QMC (client.pfx, root.cer, server.pfx).
Imported certificates into IIS web server using MMC. Server and client certificates to store Personal->Certificates, root to store Trusted Root Certification Authorities.
Requested QRS API from ASP.NET controller using QlikClient certificate from store (code below). Tried various user IDs and directories, including INTERNAL/sa_repository, but in all cases got an error "An error occurred while sending the request. The client certificate credentials were not recognized".
Endpoint for test : https://server:4242/qrs/about
I've searched the web but I haven't managed to find what I'm doing wrong, what credentials I should provide.
On the other hand, as I converted exported certificates to separate .key/.crt files (using https://www.markbrilman.nl/2011/08/howto-convert-a-pfx-to-a-seperate-key-crt-file/) and used them in the Postman from web server, it worked without any problem, actually with any UserId in header (i guess it's ignored in that case).
ASP.NET controller:
public X509Certificate2 LoadQlikCertificate()
{
X509Certificate2 certificate = null;
try
{
// Open certification store (MMC)
X509Store store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
store.Open(OpenFlags.ReadOnly);
// Get certiface based on the friendly name
certificate = store.Certificates.Cast<X509Certificate2>().FirstOrDefault(c => c.FriendlyName == "QlikClient");
// Logging for debugging purposes
if (certificate != null)
{
logger.Log(LogLevel.Warning, $"Certificate: {certificate.FriendlyName} {certificate.GetSerialNumberString()}");
}
else
{
logger.Log(LogLevel.Warning, $"Certificate: No certificate");
}
// Close certification store
store.Close();
// Return certificate
return certificate;
}
catch (Exception e)
{
...
}
}
/* Get Qlik API response
***********************/
[HttpGet("getqlikapi")]
public IActionResult GetQlikAPI()
{
// Get Qlik certificate
var certificate = this.LoadQlikCertificate();
try
{
ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
// Set server name
string server = "server";
// HARDCODED USER AND DIRECTORY FOR TESTING
string userID = "sa_repository"; // tried also other user ids
string userDirectory = "INTERNAL";
// Set Xrfkey header to prevent cross-site request forgery
string xrfkey = "abcdefg123456789";
// Create URL to REST endpoint
string url = $"https://{server}:4242/qrs/about?xrfkey={xrfkey}";
// The JSON object containing the UserId and UserDirectory
string body = $"{{ 'UserId': '{userID}', 'UserDirectory': '{userDirectory}', 'Attributes': [] }}";
// Encode the json object and get the bytes
byte[] bodyBytes = Encoding.UTF8.GetBytes(body);
// Create the HTTP Request
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
// Add the method to authentication the user
request.ClientCertificates.Add(certificate);
// POST request will be used
request.Method = "POST";
// The request will accept responses in JSON format
request.Accept = "application/json";
// A header is added to validate that this request contains a valid cross-site scripting key (the same key as the one used in the url)
request.Headers.Add("X-Qlik-Xrfkey", xrfkey);
request.ContentType = "application/json";
request.ContentLength = bodyBytes.Length;
Stream requestStream = request.GetRequestStream();
requestStream.Write(bodyBytes, 0, bodyBytes.Length);
requestStream.Close();
// Make the web request and get response
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream stream = response.GetResponseStream();
// Return string in response
//return new OkObjectResult(stream != null ? new StreamReader(stream).ReadToEnd() : string.Empty);
return new OkObjectResult("test");
}
catch (Exception e)
{
...
}
}
I ran into this issue on a system we are building.
The problem was that the user did not have rights to the certificate.
Open certificate manager (Start > Manage Computer Certificates)
Find the required certificate.
Right-click cert > All Tasks > Manage Private Keys > Add > [Select the appropriate user]
Note: Manage User Certificates does not have the Manage Private Keys option.
I am a newbie into async programming and am trying to use the httpclient to fire bulk URL requests for the page content.
Here is my attempt:
private async void ProcessUrlAsyncWithHttp(HttpClient httpClient, string purl)
{
Stopwatch sw = new Stopwatch();
sw.Start();
HttpResponseMessage response = null;
try
{
Interlocked.Increment(ref _activeRequestsCount);
var request = new HttpRequestMessage()
{
RequestUri = new Uri(purl),
Method = HttpMethod.Get,
};
request.Headers.TryAddWithoutValidation("User-Agent", "MozillaMozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36");
request.Headers.TryAddWithoutValidation("Accept", "text/html,*.*");
request.Headers.TryAddWithoutValidation("Connection", "Keep-Alive");
request.Headers.TryAddWithoutValidation("Accept-Encoding", "gzip, deflate, sdch");
request.Headers.TryAddWithoutValidation("Accept-Language", "en-US,en;q=0.8");
response = await httpClient.SendAsync(request).ConfigureAwait(false);
string html = await response.Content.ReadAsStringAsync().ConfigureAwait(false);
response.Dispose();
if (IsCaptcha(html)) throw new Exception("Captcha was returned");
request.Dispose();
Interlocked.Increment(ref _successfulCalls);
}
catch (HttpRequestException hex)
{
Console.WriteLine("http:" + hex.Message);
Interlocked.Increment(ref _failedCalls);
}
catch (Exception ex)
{
Console.WriteLine(ex.GetType().AssemblyQualifiedName + " " + ex.Message);
Interlocked.Increment(ref _failedCalls);
}
finally
{
Interlocked.Decrement(ref _activeRequestsCount);
Interlocked.Decrement(ref _itemsLeft);
if (response != null) response.Dispose();
if (httpClient != null) httpClient.Dispose();
sw.Stop();
DateTime currentTime = DateTime.UtcNow;
TimeSpan elapsedTillNow = (currentTime - _overallStartTime).Duration();
Console.WriteLine("Left:" + _itemsLeft + ", Current execution:" + sw.ElapsedMilliseconds + " (ms), Average execution:" + Math.Round((elapsedTillNow.TotalMilliseconds / (_totalItems - _itemsLeft)), 0) + " (ms)");
lock(_syncLock)
{
if (_itemsLeft == 0)
{
_overallEndTime = DateTime.UtcNow;
this.DisplayTestResults();
}
}
}
}
As you can see I am passing an httpclient to the function and it gets destroyed everytime the URL is downloaded. I know this is an overkill and ideally we should be reusing the httpclient. But since I cant use a single httpclient with different proxies for each URL (the handler needs to be passed to the constructor of httpclient and cannot be changed, hence a fresh proxy cant be given without recreating the httpclient object), I needed to use this approach.
At the caller side, I have a pretty basic code:
public async void TestAsyncWithHttp()
{
ServicePointManager.DefaultConnectionLimit = 10;
//ServicePointManager.UseNagleAlgorithm = false;
List<string> urlList = SetUpURLList();
urlList = urlList.GetRange(1, 50);
_itemsLeft = urlList.Count();
_totalItems = _itemsLeft;
List<string> proxies = new List<string>();
proxies.Add("124.161.94.8:80");
proxies.Add("183.207.228.8:80");
proxies.Add("202.29.97.5:3128");
proxies.Add("210.75.14.158:80");
proxies.Add("203.100.80.81:8080");
proxies.Add("218.207.172.236:80");
proxies.Add("218.59.144.120:81");
proxies.Add("218.59.144.95:80");
proxies.Add("218.28.35.234:8080");
proxies.Add("222.88.236.236:83");
Random rnd = new Random();
foreach (string url in urlList)
{
int ind = rnd.Next(0, proxies.Count-1);
var httpClientHandler = new HttpClientHandler
{
Proxy = new WebProxy(proxies.ElementAt(ind), false),
UseProxy = true
};
HttpClient httpClient = new HttpClient(httpClientHandler);
//HttpClient httpClient = new HttpClient();
httpClient.Timeout = TimeSpan.FromMinutes(2);
ProcessUrlAsyncWithHttp(httpClient, url);
}
}
Question is:
1) Why the TCP ports gets closed for each request. I wanted to open the max connections number of ports and reuse them across calls. e.g in the example above I can have 10 concurrent connections. Hence I wanted this to open 10 TCP ports and the rest of the 40 requests could then use these 10 ports in tandem. This is a normal behaviour expected in httpwebrequest. I have a working code for using httpwebrequest that depicts this behavior of reusing ports. Can post the code of that on demand for anyone who might want to have a look. So its kind of weird that httpclient does not mimic this behaviour although it is based on httpwebrequest.
2) How do we assign autoredirect to false for such calls?
3) I intend to use this function for multiple calls - say around 50K. Anything wrong in the way the code is written that might need a correction
4) Lets assume that I somehow manage to use a single httpclient object instead of one object per request. What is the way to ensure I read cookies for all these individual requests and also alter them if necessary all the while remembering that I have a single httpclient class for the whole set of URL requests?
Tks
Kallol
In my experience (I once had a similar problem with TCP port congestion, because of ports always getting closed, when I was hitting a server with around 6000 connections a minute) it suffices to reuse the HttpClientHandler objects, which actually manage the connection pooling, and always recreate the HttpClient objects for each request (using the constructor with HttpClientManager parameter).
Hope this helps.
Matthias
have you tried putting the HttpClient code in class and create 10 classes, each with a HttpClient?
I previously asked on StackOverflow how to parse XML downloaded programmatically by my ASP.net application. By this, I mean that the user visits https://www.example.com/page1.aspx. The code-behind for page1.aspx is supposed to somehow download and parse an xml file located at https://www.example.com/foo.xml.
I received good answers about how to parse the XML. However, I've been out of luck with being able to retrieve XML from my secure HTTPS server.
I am looking at a situation where https://www.example.com/foo.xml authenticates requests with a cookie. (third party system, not Forms Authentication). The answer I received to my question about how to download and parse XML suggested that I use the System.Net.WebClient class. I read that the WebClient class must be customized to work with cookies. Therefore, I wrote the following code:
public class WebClientWithCookies : WebClient
{
private CookieContainer m_container = new CookieContainer();
public CookieContainer CookieContainer
{
get { return m_container; }
set { m_container = value; }
}
public void addCookie(Cookie cookie)
{
m_container.Add(cookie);
}
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
if ( request is HttpWebRequest)
{
(request as HttpWebRequest).CookieContainer = m_container;
}
return request;
}
} // end class
However, when the request is received at https://www.example.com/foo.xml, there are no cookies in the request, and so it doesn't work.
How can I work around this problem?
Where are you creating the cookie? That seems to be a missing part from the code you are displaying. There is an "HttpCookie" class as part of the System.Web name space that may be useful.
Here's the code that I eventually wrote that solved the problem:
private XmlDocument getXmlData(string url)
{
System.Net.HttpWebRequest rq = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(url);
System.Net.CookieContainer container = new System.Net.CookieContainer();
for (int i = 0; i < System.Web.HttpContext.Current.Request.Cookies.Count; i++)
{
System.Web.HttpCookie httpcookie = System.Web.HttpContext.Current.Request.Cookies[i];
string name = httpcookie.Name;
string value = httpcookie.Value;
string path = httpcookie.Path;
string domain = "my.domain";
System.Net.Cookie cookie = new System.Net.Cookie(name, value, path, domain);
container.Add(cookie);
}
rq.CookieContainer = container;
rq.Timeout = 10000;
rq.UserAgent = "Asset Tracker Server Side Code";
System.Net.HttpWebResponse rs = (System.Net.HttpWebResponse)rq.GetResponse();
System.Text.Encoding enc = System.Text.Encoding.GetEncoding(1252);
System.IO.StreamReader reader = new System.IO.StreamReader(rs.GetResponseStream());
System.Xml.XmlDocument xml = new System.Xml.XmlDocument();
xml.Load(rs.GetResponseStream());
return xml;
}