HTML scraping: Forms authentication failed for the request. The ticket supplied has expired - asp.net

The ActiveForums module we're using as part of our DotNetNuke system has a bug in the XML for it's RSS feed. It doesn't correctly encode ampersands, it leaves them as & rather than encoding them as &
I've reported the bug to the company, but in the mean time I need a fix. So what I've done is create an intermediary page that makes a request to the RSS feed via a System.Net.HttpWebRequest.Create(url) and them performs a Regex.Replace to replaces any unencoded ampersands.
The problem is that when I run the code on our production server I get an exception: The remote server returned an error: (500) Internal Server Error.
The only reason I could think of was around authentication (As the server requires NTLM), however as far as I can tell I'm doing this part of it correctly. My code is shown below:
string html = string.Empty;
string url = "http://intranet.nt.avs/dnn/Default.aspx?tabid=130";
WebResponse response;
WebRequest request = System.Net.HttpWebRequest.Create(url);
request.PreAuthenticate = true;
request.Credentials = System.Net.CredentialCache.DefaultCredentials;
response = request.GetResponse();
using (StreamReader sr = new StreamReader(response.GetResponseStream()) )
{
html = sr.ReadToEnd();
}
// Clean invalid XML
html = Regex.Replace( html, "&(?!amp;|gt;|lt;|quot;|apos;)", "&", RegexOptions.Multiline | RegexOptions.IgnoreCase );
Response.ContentType = "text/xml";
Response.Write( html );
Updated: Here's what the event log says
Error code: 4005
Event message: Forms authentication failed for the request. Reason: The ticket supplied has expired

Related

WooCommerce rest api access failing with httpWebRequest

I am trying to get WooCommerce data using WooCommerce v3 API from c#.
When I use below url in browser I get the products data as json;
https://MyUsername:MyPassword#mydomain.com/wp-json/wc/v3/products?consumer_key=ck_12345678901234567890&consumer_secret=cs_12345678901234567890
When I use url
https://MyUsername:MyPassword#mydomain.com/wp-json/wc/v3/products
and
consumer_key = ck_12345678901234567890 and consumer_secret = cs_12345678901234567890
as Params in Postman then I get the products data as well. So the credentials and url are valid.
However when I try to do this using HttpWebRequest it fails with "The remote server returned an error: (401) Unauthorized" exception. My code is below and the exception appears on the last line of code.
The vales in HttpWebRequest before the GetResponseAsync call can be seen here.
What am I doing wrong?
Thanks
Regards
string wc_url = "https://MyUsername:MyPassword#mydomain.com/wp-json/wc/v3/products?consumer_key=ck_12345678901234567890&consumer_secret=cs_12345678901234567890";
HttpWebRequest httpWebRequest = null;
httpWebRequest = (HttpWebRequest)WebRequest.Create(wc_url);
httpWebRequest.AllowReadStreamBuffering = false;
WebResponse wr = await httpWebRequest.GetResponseAsync().ConfigureAwait(false);

Get HTTP Response from a url by using C#

Environment: ASP.Net MVC 4 using C#
I need to get image by using GET request to a URL /inbound/faxes/{id}/image
I used the code below
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("/inbound/faxes/238991717/image");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
System.IO.StreamReader stream = new StreamReader(response.GetResponseStream());
but it flags "URL not valid"
I used the complete URL www.interfax.net/inbound/faxes/{id}/image
but the result is same
I want to follow this article to receive faxes
Accepting incoming fax notifications by callback
Can anyone help me to get fax...?
Try like this:
using (var client = new WebClient())
{
byte[] imageData = client.DownloadData("http://www.interfax.net/inbound/faxes/{id}/image");
}
Notice how the url is prefixed with the protocol (HTTP in this case). Also make sure you have replaced the {id} part of the url with the actual id of the image you are trying to retrieve.

Executing a webrequest without redirecting a page

I ran into a weird problem while using openid in asp.net. I wanted a server side logout for gmail account but without redirecting to another page.
I thought executing a web request would do that. This is my code
HttpWebRequest loHttp =
(HttpWebRequest)WebRequest.Create("https://www.google.com/accounts/Logout");
// *** Set properties
loHttp.Timeout = 10000; // 10 secs
loHttp.UserAgent = "Code Sample Web Client";
// *** Retrieve request info headers
HttpWebResponse loWebResponse = (HttpWebResponse)loHttp.GetResponse();
Encoding enc = Encoding.GetEncoding(1252); // Windows default Code Page
StreamReader loResponseStream =
new StreamReader(loWebResponse.GetResponseStream(), enc);
string lcHtml = loResponseStream.ReadToEnd();
loWebResponse.Close();
loResponseStream.Close();
But it doesn't seem to work. The gmail account is still signed in.
Is it possible to execute a webrequest with such URL?
Thanks
I think that's because HttpWebRequest is made at the server level and you are logged in in the client.
You should use an iframe to load the URL

PayPal Express Checkout returns - Security header is not valid

This has probably been asked a thousand times... But i'm sure i have all the endpoints and credentials right. It was working yesterday.
Security error:
Error no: 10002
Error message: Security header is not valid
Right now i'm testing against the sandbox server.
Whenever i try the testURL in my browser i get ACK=Success with a token, but i get "Security header is not valid" error when it runs through the code.
It was working a few hours ago but for some reason i keep getting the same error now.
using user/pwd and signature from a sandbox account
https://api-3t.sandbox.paypal.com/nvp
USER=xxxxxxxxxxxxxxxxxxxx
PWD=XXXXXXXXXXXXX
SIGNATURE=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
I have the following test code:
String testURL = "https://api-3t.sandbox.paypal.com/nvp?USER=xxxxxxxxxxxxxxxxxxxx&PWD=XXXXXXXXXXXXX&SIGNATURE=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX&VERSION=84.0-2276209&PAYMENTREQUEST_0_PAYMENTACTION=Sale&PAYMENTREQUEST_0_AMT=15&RETURNURL=https%3a%2f%2fdomain.com%2fCheckout.aspx&CANCELURL=https%3a%2f%2fdomain.com%2fCheckout.aspx&METHOD=SetExpressCheckout";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(testURL);
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
//Send the request to PayPal and get the response
StreamWriter streamOut = new StreamWriter(req.GetRequestStream(), System.Text.Encoding.ASCII);
streamOut.Write(testURL);
streamOut.Close();
// get resposne
StreamReader streamIn = new StreamReader(req.GetResponse().GetResponseStream());
string strResponse = HttpUtility.UrlDecode(streamIn.ReadToEnd());
streamIn.Close();
HTTPREsponse when code sends it
TIMESTAMP=2011-11-22T23:25:34Z&CORRELATIONID=392047cb78388&ACK=Failure&VERSION=84.000000&BUILD=2271164&L_ERRORCODE0=10002&L_SHORTMESSAGE0=Security error&L_LONGMESSAGE0=Security header is not valid&L_SEVERITYCODE0=Error
HttpResponse if i just copy paste the testURL in the browser
TOKEN=EC%2d4JW15968AV8121546&TIMESTAMP=2011%2d11%2d22T22%3a59%3a27Z&CORRELATIONID=c790299fd9ac7&ACK=Success&VERSION=84%2e000000&BUILD=2271164
I've tried not to UrlEncode the variables in the test url... same problem
thanks in advance
OK this might be a bug on Paypal's side.
if i change the testUrl from
String testURL = "https://api-3t.sandbox.paypal.com/nvp?USER=XXX&PWD=YYY&SIGNATURE=ZZZ&VERSION=.......&METHOD=SetExpressCheckout";
to
String testURL = "https://api-3t.sandbox.paypal.com/nvp?&x=y&USER=XXX&PWD=YYY&SIGNATURE=ZZZ&VERSION=.......&METHOD=SetExpressCheckout";
it works
See the bolded part with random first querystring variable.
PayPal seems to ignore the first querystring parameter when it's sent from codebehind (which would be user=xxx if there wasn’t x=y before it).

ASP.NET HttpWebRequest with Kerberos Authentication

I am trying to connect to a web service that uses Kerberos Authentication to authorize the user, but all I get is a 401 unauthorized everytime I try to make the request. Below is the code that I am using. Thanks in advance for any help you can provide!
public XPathNavigator GSASearch(string url, string searchString)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url + searchString);
request.CookieContainer = new CookieContainer();
request.Credentials = CredentialCache.DefaultCredentials;
request.ContentType = "text/xml";
request.Method = "POST";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream receiveStream = response.GetResponseStream();
XPathDocument doc = new XPathDocument(receiveStream);
return doc.CreateNavigator();
}
EDIT: I feel I should explain a bit more what I am attempting to do. I have been tasked with providing a new interface for my company's Google Search Appliance. I am using an ASP.NET page, which does some things like choose a Collection depending on where a user is located, etc. and then sends the appropriate search string the the GSA. This was all working well until they decided to turn authentication on, and now I can't get any results (I either get a 401 unauthorized, or a message stating that 'Data at the root level is invalid'). If I take the search string and provide it directly to the GSA, it authenticates fine, and displays the results, I just can't seem to get it through the HttpWebRequest.
EDIT 2: I did a little more looking (ran the request through Fiddler) and it looks like the request is only attempting Negotiate and not Kerberos. I set the credentials to use Kerberos explicitly as below, but it didn't help...
public XPathNavigator GSASearch(string url, string searchString)
{
CredentialCache credCache = new CredentialCache();
credCache.Add(new Uri(url), "Kerberos", CredentialCache.DefaultNetworkCredentials);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url + searchString);
request.CookieContainer = new CookieContainer();
request.PreAuthenticate = true;
request.Credentials = credCache;
request.ContentType = "text/xml";
request.Method = "POST";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream receiveStream = response.GetResponseStream();
//StreamReader readStream = new StreamReader(receiveStream);
XPathDocument doc = new XPathDocument(receiveStream);
return doc.CreateNavigator();
}
EDIT 3: Ok, looking closer again, the CredentialCache.DefaultCredentials doesn't appear to have my network credentials in it...
1) Have you done a wireshark trace of a successful session to the GSA using the browser? Does that work?
2) If #1 works, what is the WWW-Authenticate header that is sent by the GSA on the first unauthenticated request?
3) Is the machine on which the ASPX app is running a part of the same AD domain that the GSA is in? AFAIK this is probably required for a successful auth.
4) Next, since it is the ASPX app that is doing the request, you cannot use the DefaultCredentials because you actually need the credentials of a user that is trusted by the GSA. For this you should either create a special user account for the app that is talking to the GSA, or have each user be a trusted user on the GSA and have the ASPX page authenticate the user first, then pass those credentials to the GDA using Delegation. For this you will also have to make the server running the ASPX app trusted for delegation.
In my opinion, you should first model your code into a console app that you run, and debug. Then port it to ASPX page. That way you will be able to know if the failure is due to the host (ASPX vs console) or something else.

Resources