I am attempting to get the new Google reCaptcha working in my ASP.NET project and I am having problems getting it to be the new one "I'm not a robot".
I had the old one in there and after doing much research on the developers.google.com web site, everything looks the same (they even point me to a download of the same dll - 1.0.5). So, I got the new keys and put them in and it works but it looks just like the old reCaptcha.
Has anyone gotten the new one to work with their ASP.Net? What am I missing?
EDIT:
So playing around in a test app and searching some other web sites I found that if I create a page like this:
<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
<title>reCAPTCHA demo: Simple page</title>
<script src="https://www.google.com/recaptcha/api.js" async defer></script>
</head>
<body>
<form id="form1" runat="server" action="?" method="POST">
<div>
<div class="g-recaptcha" data-sitekey="My Public Key"></div>
<br/>
<asp:Button ID="Button1" runat="server" Text="Submit" />
</div>
</form>
</body>
</html>
And then in my code-behind (Button1_Click), I do this:
Dim Success As Boolean
Dim recaptchaResponse As String = request.Form("g-recaptcha-response")
If Not String.IsNullOrEmpty(recaptchaResponse) Then
Success = True
Else
Success = False
End If
The recaptchaResponse will either be empty or filled in depending on if they are a bot or not. The issue is, I now need to take this response and send it to google with my private key so I can verify that the response was not provided by a bot, in my code-behind, but I cannot figure out how. I tried this (in place of Success = True):
Dim client As New System.Net.Http.HttpClient()
client.BaseAddress = New Uri("https://www.google.com/recaptcha/")
client.DefaultRequestHeaders.Accept.Clear()
client.DefaultRequestHeaders.Accept.Add(New Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"))
Dim response As Net.Http.HttpResponseMessage = Await client.GetAsync("api/siteverify?secret=My Private key&response=" + recaptchaResponse)
If (response.IsSuccessStatusCode) Then
Dim CaptchResponse As ReCaptchaModel = Await response.Content.ReadAsAsync(Of ReCaptchaModel)()
Success = CaptchResponse.success
Else
Success = False
End If
But, I could not figure out how to get the async stuff working and I cannot find anything on what ReCaptchaModel is, so I found another way to call a web service and get a json response and tried this instead:
Dim request As Net.WebRequest = Net.WebRequest.Create("https://www.google.com/recaptcha/")
Dim Data As String = "api/siteverify?secret=My Private Key&response=" + recaptchaResponse
request.Method = "POST"
request.ContentType = "application/json; charset=utf-8"
Dim postData As String = "{""data"":""" + Data + """}"
'get a reference to the request-stream, and write the postData to it
Using s As IO.Stream = request.GetRequestStream()
Using sw As New IO.StreamWriter(s)
sw.Write(postData)
End Using
End Using
'get response-stream, and use a streamReader to read the content
Using s As IO.Stream = request.GetResponse().GetResponseStream()
Using sr As New IO.StreamReader(s)
'decode jsonData with javascript serializer
Dim jsonData = sr.ReadToEnd()
Stop
End Using
End Using
But, this just gives me the content of the web page at https://www.google.com/recaptcha. Not what I want. The Google page isn't very useful and I am stuck on where to go. I need some help either calling the Google verify service or if anyone has found another way to do this from ASP.NET.
I had just about given up when I ran across something unrelated that made me think about it again and in a different way. In my last attempt above, I was attempting to pass the private key and recaptcha response as the data, so I tried it in the create of the WebRequest and it worked. Here is the final solution:
Using the same HTML posted above, I created a function that I can call in the button click event where I check the Page.IsValid and call this function:
Private Function IsGoogleCaptchaValid() As Boolean
Try
Dim recaptchaResponse As String = Request.Form("g-recaptcha-response")
If Not String.IsNullOrEmpty(recaptchaResponse) Then
Dim request As Net.WebRequest = Net.WebRequest.Create("https://www.google.com/recaptcha/api/siteverify?secret=My Private Key&response=" + recaptchaResponse)
request.Method = "POST"
request.ContentType = "application/json; charset=utf-8"
Dim postData As String = ""
'get a reference to the request-stream, and write the postData to it
Using s As IO.Stream = request.GetRequestStream()
Using sw As New IO.StreamWriter(s)
sw.Write(postData)
End Using
End Using
''get response-stream, and use a streamReader to read the content
Using s As IO.Stream = request.GetResponse().GetResponseStream()
Using sr As New IO.StreamReader(s)
'decode jsonData with javascript serializer
Dim jsonData = sr.ReadToEnd()
If jsonData = "{" & vbLf & " ""success"": true" & vbLf & "}" Then
Return True
End If
End Using
End Using
End If
Catch ex As Exception
'Dont show the error
End Try
Return False
End Function
I'm sure there are improvements to be made to the code, but it works. I couldn't see adding references to some JSON libraries for reading one thing I just check the string.
Thank you for sharing this. It worked for me. I went ahead and converted it to C# (since that's what I was using) and added a few things.
I changed the validation step. I split the JSON string and evaluated if success was found where it should be.
I used the ConfigurationManager to store the ReCaptcha Keys.
Finally, I changed it from using a WebRequest to using and HttpClient. This cut the code in half because I don't need to read the stream now.
Feel free to use this code as well.
private static bool IsReCaptchaValid(string response)
{
if (string.IsNullOrWhiteSpace(response))
{
return false;
}
var client = new HttpClient();
string result =
client.GetStringAsync(string.Format("{0}?secret={1}&response={2}", ConfigurationManager.AppSettings["ReCaptchaValidationLink"],
ConfigurationManager.AppSettings["ReCaptchaSecretKey"], response)).Result;
string[] split = result.Split('\"');
return split[1] == "success";
}
I took a slightly different approach, using the data-callback option and a Session parameter. The following sits within the MainContent block of the .aspx file:
<asp:ScriptManager ID="scrEnablePage" EnablePageMethods="true" runat="server" />
<asp:Panel ID="pnlCaptcha" runat="server" Visible="true">
<div class="g-recaptcha"
data-sitekey='<asp:Literal ID="litKey" runat="server" Text="<%$ AppSettings:recaptchaPublicKey%>" />'
data-callback="handleCaptcha"></div>
</asp:Panel>
<script src="https://www.google.com/recaptcha/api.js" async defer></script>
<script type="text/javascript">
function handleCaptcha(e) {
PageMethods.RecaptchaValid(e);
location.reload(true);
}
</script>
Then in the code-behind:
Private Const GoogleUrl As String = "https://www.google.com/recaptcha/api/siteverify?secret={0}&response={1}"
Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
pnlCaptcha.Visible = Not (Session("VerifiedHuman") = "True")
...
End Sub
<System.Web.Services.WebMethod(EnableSession:=True)> _
Public Shared Sub RecaptchaValid(response As String)
Dim client As New System.Net.WebClient()
Dim outcome As Dictionary(Of String, String)
Dim result As String = String.Join(vbCrLf,
{"{", """success"": true", "}"})
Dim serializer As New System.Web.Script.Serialization.JavaScriptSerializer()
Dim url As String = String.Format(GoogleUrl,
ConfigurationManager.AppSettings.Get("recaptchaPrivateKey"),
response)
Try
result = client.DownloadString(url)
Catch ex As System.Net.WebException
Exit Sub ' Comment out to default to passing
End Try
outcome = serializer.Deserialize(Of Dictionary(Of String, String))(result)
HttpContext.Current.Session("VerifiedHuman") = outcome("success")
End Sub
Now in Page_Load you can check Session("VerifiedHuman") = "True" and update your page controls accordingly, hiding the panel with the Captcha control and showing the other appropriate items.
Note that this takes the keys from Web.config, i.e.
<configuration>
<appSettings>
<add key="recaptchaPublicKey" value="..." />
<add key="recaptchaPrivateKey" value="..." />
...
</appSettings>
...
</configuration>
This adds a few things. It converts the response from Google into a Json object, it adds a timeout on the verification request, and it adds a verification of the hostname (required by Google if sending requests from multiple domains and the domains aren't listed in the Google Admin area).
Imports Newtonsoft.Json
Public Class Google
Public Class ReCaptcha
Private Const secret_key = "YOUR_SECRET_KEY"
Public Shared Function Validate(Request As HttpRequest, hostname As String) As Boolean
Dim g_captcha_response = Request.Form("g-recaptcha-response")
If Not String.IsNullOrEmpty(g_captcha_response) Then
Dim response = ExecuteVerification(g_captcha_response)
If Not response.StartsWith("ERROR:") Then
Dim json_obj = JsonConvert.DeserializeObject(Of ValidateResponse)(response)
If json_obj.success Then
If json_obj.hostname.ToLower = hostname.ToLower Then Return True
End If
End If
End If
Return False
End Function
Private Shared Function ExecuteVerification(g_captcha_response As String) As String
Dim request As Net.WebRequest = Net.WebRequest.Create("https://www.google.com/recaptcha/api/siteverify?secret=" & secret_key & "&response=" & g_captcha_response)
request.Timeout = 5 * 1000 ' 5 Seconds to avoid getting locked up
request.Method = "POST"
request.ContentType = "application/json"
Try
Dim byteArray As Byte() = Encoding.UTF8.GetBytes("")
request.ContentLength = byteArray.Length
Dim dataStream As Stream = request.GetRequestStream()
dataStream.Write(byteArray, 0, byteArray.Length)
dataStream.Close()
Dim response As Net.WebResponse = request.GetResponse()
dataStream = response.GetResponseStream()
Dim reader As New StreamReader(dataStream)
Dim responseFromServer As String = reader.ReadToEnd()
reader.Close()
response.Close()
Return responseFromServer
Catch ex As Exception
Return "ERROR: " & ex.Message
End Try
End Function
Public Class ValidateResponse
Public Property success As Boolean
Public Property challenge_ts As DateTime
Public Property hostname As String
<JsonProperty("error-codes")>
Public Property error_codes As List(Of String)
End Class
End Class
End Class
So in the button's Click event, just call:
If Google.ReCaptcha.Validate(Request, Request.Url.Host) Then
' good to go
Else
' validation failed
End If
Related
I am working in an "old" VB.Net application. I wish to make a call to an API, check if I got a 404, parse the JSON result and all the other usual things you can do with an API call.
What is the cleanest way of doing this? I thought I can use the HttpClient class, but I can't apparently! VS 2017 is not giving me the option of adding System.Net.Http as an Import.
CODE
Right now I'm doing this which seems messy.
Public Function GetUserInfo(ByVal authTokenBytes As Byte()) As WebPayWS.GetUserInfoResult
Dim token As New token
token.token1 = authTokenBytes
Dim userInformation As GetUserInfoResult = _myService.GetUserInfo(token)
If AccountExists(userInformation.accountno) Then
Dim response As String = GetAccountInfoFromApi(userInformation.accountno)
End If
Return userInformation
End Function
Private Function GetAccountInfoFromApi(accountno As String) As String
Dim accountInformationUrl As String = "URL"
Dim webClient As WebClient = New WebClient()
Return webClient.DownloadString(New Uri(accountInformationUrl))
End Function
Am I stuck with WebClient? If yes, how do I check for a 404 using WebClient?
You can wrap it in an exception with the WebClient check this out
Private Function GetAccountInfoFromApi(accountno As String) As String
Dim accountInformationUrl As String = "URL"
Dim webClient As WebClient = New WebClient()
'Return webClient.DownloadString(New Uri(accountInformationUrl))
Dim retString As String
Try
retString = webClient.DownloadString(New Uri(accountInformationUrl))
Catch ex As WebException
If ex.Status = WebExceptionStatus.ProtocolError AndAlso ex.Response IsNot Nothing Then
Dim resp = DirectCast(ex.Response, HttpWebResponse)
If resp.StatusCode = HttpStatusCode.NotFound Then
' HTTP 404
'other steps you want here
End If
End If
'throw any other exception - this should not occur
Throw
End Try
Return retString
End Function
Add a reference first
Right click on project
Add...
Reference...
Browse
C:\Program Files (x86)\Microsoft ASP.NET\ASP.NET MVC 4\Assemblies\System.Net.Http.dll
Ok
Now, you can import
Imports System.Net.Http
You can also use NuGet, see https://stackoverflow.com/a/13668810/832052
I'm trying to scrape some schedules off of a website. the information is displayed in a GridView with paging.
The url is:
http://www.landmarkworldwide.com/when-and-where/register/search-results.aspx?prgid=0&pgID=270&crid=0&ctid=&sdt=0
My Issue is when I want to scrape pages other then #1 in the grid view.
The best post I found so far was This One, but it doesn't work and that topic is not complete. I tried to use Fiddler and Chrome to get the post data and use it, but I can't get it to work for me. Can you guys see what's missing?
Here's the code I am using. it's in VB, but you can answer in C# and I'll translate -) (sorry)
Protected Sub Page_Load(sender As Object, e As System.EventArgs) Handles Me.Load
Dim lcUrl As String = "http://www.landmarkworldwide.com/when-and-where/register/search-results.aspx?prgid=0&pgID=270&crid=0&ctid=&sdt=0"
' first, request the login form to get the viewstate value
Dim webRequest__1 As HttpWebRequest = TryCast(WebRequest.Create(lcUrl), HttpWebRequest)
Dim responseReader As New StreamReader(webRequest__1.GetResponse().GetResponseStream())
Dim responseData As String = responseReader.ReadToEnd()
responseReader.Close()
' extract the viewstate value and build out POST data
Dim viewState As String = ExtractViewState(responseData)
Dim loHttp As HttpWebRequest = DirectCast(WebRequest.Create(lcUrl), HttpWebRequest)
' *** Send any POST data
Dim lcPostData As String = [String].Format("__VIEWSTATE={0}&__EVENTTARGET={1}&__EVENTARGUMENT={2}", viewState, HttpUtility.UrlEncode("contentwrapper_0$maincontent_0$maincontentfullwidth_0$ucSearchResults$gvPrograms"), HttpUtility.UrlEncode("Page$3"))
loHttp.Method = "POST"
Dim lbPostBuffer As Byte() = System.Text.Encoding.GetEncoding(1252).GetBytes(lcPostData)
loHttp.ContentLength = lbPostBuffer.Length
Dim loPostData As Stream = loHttp.GetRequestStream()
loPostData.Write(lbPostBuffer, 0, lbPostBuffer.Length)
loPostData.Close()
Dim loWebResponse As HttpWebResponse = DirectCast(loHttp.GetResponse(), HttpWebResponse)
Dim enc As Encoding = System.Text.Encoding.GetEncoding(1252)
Dim loResponseStream As New StreamReader(loWebResponse.GetResponseStream(), enc)
Dim lcHtml As String = loResponseStream.ReadToEnd()
loWebResponse.Close()
loResponseStream.Close()
Response.Write(lcHtml)
End Sub
Private Function ExtractViewState(s As String) As String
Dim viewStateNameDelimiter As String = "__VIEWSTATE"
Dim valueDelimiter As String = "value="""
Dim viewStateNamePosition As Integer = s.IndexOf(viewStateNameDelimiter)
Dim viewStateValuePosition As Integer = s.IndexOf(valueDelimiter, viewStateNamePosition)
Dim viewStateStartPosition As Integer = viewStateValuePosition + valueDelimiter.Length
Dim viewStateEndPosition As Integer = s.IndexOf("""", viewStateStartPosition)
Return HttpUtility.UrlEncodeUnicode(s.Substring(viewStateStartPosition, viewStateEndPosition - viewStateStartPosition))
End Function
To make it work you need to send all input fields to the page, not only viewstate. Other critical data is the __EVENTVALIDATION for example that you do not handle it. So:
First you need to make scrape on the #1 page. So load it and use the Html Agility Pack to convert it to a usable struct.
Then extract from that struct the input data that you need to post. From this answer HTML Agility Pack get all input fields here is a code sniped on how you can do that.
foreach (HtmlNode input in doc.DocumentNode.SelectNodes("//input"))
{
// use this to create the post string
// input.Attributes["value"];
}
Then when you have the post data that is needed to be a valid post, you move to the next step. Here is an example How to pass POST parameters to ASP.Net web request?
You can also read: How to use HTML Agility pack
In .NET by default the client side ID's, for server side controls, get concatenated with generated text.
For example:
<asp:TextBox ID="txtUser" runat="server">
would become...
<input type="text" id="ctl00_body_txbUser">
However when I use HttpWebRequest.GetResponse(objReq.Getresponse, HttpWebResponse) to request the same page the item comes back without the auto generated text.
<input type="text" id="txbUser">
Is it possible to use an HttpWebRequest object and GetResponse in such a way that it returns a response with the Auto generated ID's .NET uses for server side controls?
I am working with a 3rd party that has previously set up translation rules specific to ID, now we are attempting have the same rules work against an API call passed a string generated from the page. However, the string generated from the page does not have the same IDs.
Below is code being used to return the Web Page as a string.
Public Shared Function GetWebPageAsString(ByVal strURI As String, ByVal strPostData As String) As String
' Declare our variables. '
Dim objHttpRequest As HttpWebRequest
Dim PostBuffer() As Byte
Dim PostDataStream As Stream = Nothing
Dim objHttpResponse As HttpWebResponse = Nothing
Dim objStreamReader As StreamReader = Nothing
Dim strResponseText As String = ""
Try
' Create a new request. '
objHttpRequest = CType(WebRequest.Create(strURI), HttpWebRequest)
objHttpRequest.Timeout = 3000000
objHttpRequest.Method = "POST"
PostBuffer = Encoding.ASCII.GetBytes(strPostData)
objHttpRequest.ContentType = "application/x-www-form-urlencoded"
objHttpRequest.ContentLength = PostBuffer.Length
PostDataStream = objHttpRequest.GetRequestStream
PostDataStream.Write(PostBuffer, 0, PostBuffer.Length)
PostDataStream.Close()
' Get the response to our request as a stream object. '
objHttpResponse = CType(objHttpRequest.GetResponse, HttpWebResponse)
' Create a stream reader to read the data from the stream. '
objStreamReader = New StreamReader(objHttpResponse.GetResponseStream, Encoding.UTF8)
' Copy the text retrieved from the stream to a variable. '
strResponseText = objStreamReader.ReadToEnd()
' Close our objects. '
objStreamReader.Close()
objHttpResponse.Close()
Catch ex As Exception
strResponseText = strURI & " | " & strPostData
Throw (ex)
Finally
If Not objStreamReader Is Nothing Then
objStreamReader.Close()
End If
If Not PostDataStream Is Nothing Then
PostDataStream.Close()
End If
If Not objHttpResponse Is Nothing Then
objHttpResponse.Close()
End If
objHttpRequest = Nothing
PostBuffer = Nothing
PostDataStream = Nothing
objHttpResponse = Nothing
objStreamReader = Nothing
End Try
' Set return value. '
Return strResponseText
End Function
EDIT: Just to Clarify, I need the IDs to continue to be Auto generated by .NET. I understand that I could make them equal by setting the mode to Static. Unfortunately the 3rd Party we are working with has already created the rules for our current pages based on the IDs that were generated by .NET. Requesting the same page using the HTTPRequest object and pushing data into a stream. I am not seeing the Auto Generated IDs anymore, even though its the same page.
Create a clean master page and put your page in it. That should fix the IDs issue.
I am developing app where I need to capture an Information from Webpage after giving credentials automatically. Some how I managed to do Automatic login and redirection of page. Here is my code :
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("https://abcd.com.au/categories/A_dfn/sdf");
HttpWebResponse res = req.GetResponse() as HttpWebResponse;
StringBuilder sb = new StringBuilder();
byte[] buf = new byte[10000];
Stream resStream = res.GetResponseStream();
string s = null;
int c = 0;
do
{
c = resStream.Read(buf, 0, buf.Length);
if (c != 0) {
s = ASCIIEncoding.ASCII.GetString(buf, 0, c);
sb.Append(s);
}
} while (c > 0);
string oldhead = "class=\"login_button\">";
string newhead = "class=\"login_button\"> <script type=\"text/javascript\">document.getElementById('btn').click()</script>";
sb.Replace(oldhead, newhead);
string oldbtn = "value=\"Submit\"";
string newbtn = "value=\"Submit\" id=\"btn\" ";
sb.Replace(oldbtn, newbtn);
string oldAction = "<form action=\"/login\" method=\"post\">";
string newAction = "<form action=\"https://abcd.com.au/login?orig_req_url=%2Fcategories/A_dfn/sdf\" method=\"post\">";
sb.Replace(oldAction, newAction);
string oldUsername = "<input id=\"login_email\" type=\"text\" name=\"user[email_address]\" class=\"textBox\" value=\"\">";
string newUserName = "<input id=\"login_email\" type=\"text\" name=\"user[email_address]\" class=\"textBox\" value=\"abc#xyz.com.au\">";
sb.Replace(oldUsername, newUserName);
string oldPass = "<input id=\"login_password\" type=\"password\" name=\"user[password]\" class=\"textBox\" value=\"\">";
string newPass = "<input id=\"login_password\" type=\"password\" name=\"user[password]\" class=\"textBox\" value=\"abc\">";
sb.Replace(oldPass,newPass);
Response.Write(sb);
This is show me expected output as I want by rendering page(Response.write(sb)). But, now I want to do same thing without redirecting to "https://abcd.com.au/login?orig_req_url=%2Fcategories/A_dfn/sdf" and want to do more stuff on this. I expect to get output of Response.Write(sb) in some buffer. Is it possible to do?
Here is example, that explains exactly what I want to do.
I am looking for an product's qty say name : Screw 15mm, this resides in page https://abcd.com.au/%2Fcategories/A_dfn/sdf.
So, I am requesting this url first, but as need login to access that page, its redirecting me to login page, filling username and password, pressing login button by using javascript,and then redirected to Originally Requested page. And on this page I want to find for that product, and return information to my web app.
All this I want to do without showing to user.
I just want to show retrieved information.
Thanks.
What you are looking for is a persisted session. Your approach towards this problem is incorrect. You are triggering the submit on the client-side. What you are trying to achieve should be done on the server-side.
The key to your scenario is to persist (store) the session & cookies set by the login page; then before your next request for the product info, inject the credential into the requesting webRequest.
Use the WebRequest object to load the login page.
Store any info (cookies) sent by the login page Response header.
create a new WebRequest object with the provided Response header, inject in userid/password.
Store any credentials returned by the Response.
Proceed to request for the quote info.
There is no generic way to do this without knowing the website you are trying to screen-scrap from. But the general step is as above. Basically, you need to create a custom class for this.
Also, you need HTMLAgilityPack to parse the HTML nodes. It is the correct method.
EDIT: Added my codes. Just so happen that I've created this class before sometime ago. So, you're in luck. However, you will need HTMLAgilityPack installed & referenced to use it. You can download HAP at: http://htmlagilitypack.codeplex.com/ If you want to do any serious screen-scraping, HAP is the de-facto standard.
Public Class clsBrowserSession
'=================================================================================================================================
'This is a special Browser Post class
' Instead of just POST to a URL as per the clsWeb.fnsPostResponse()
' clsBrowserSession allows us to LOAD a page first, persist all the cookies and variables, and then only POST to the target URL.
' The reason is that some program will drop (lets say) a SessionID as an input when you first load the page.
' and when you post, without the SessionID (variable), it will reject the POST. Thus clsBrowserSession can solve this problem.
'=================================================================================================================================
' USAGE:
' Dim voBrowserSession As New clsBrowserSession
' voBrowserSession.sbLoadPage("https://xxx.yyy.net.my/publicncdenq/index.htm")
' voBrowserSession.proFormElements("UserID") = "myID"
' voBrowserSession.proFormElements("Password") = "myPassword"
' Dim vsResponseHTML As String = voBrowserSession.Post("https://xxx.yyy.net.my/publicncdenq/index.htm")
Private vbIsPostingInProgress As Boolean
Public voCookies As System.Net.CookieCollection
Public proHTMLDoc As HtmlAgilityPack.HtmlDocument
Public proFormElements As clsFormElementCollection
Public Sub sbLoadPage(pvsURL As String)
vbIsPostingInProgress = False
fnoCreateWebRequestObject().Load(pvsURL)
End Sub
Public Function Post(pvsURL As String) As String
vbIsPostingInProgress = True
fnoCreateWebRequestObject().Load(pvsURL, "POST")
Return proHTMLDoc.DocumentNode.InnerHtml
End Function
Private Function fnoCreateWebRequestObject() As HtmlAgilityPack.HtmlWeb
Dim voWeb As New HtmlAgilityPack.HtmlWeb
voWeb.UseCookies = True
voWeb.PreRequest = New HtmlAgilityPack.HtmlWeb.PreRequestHandler(AddressOf event_OnPreRequest)
voWeb.PostResponse = New HtmlAgilityPack.HtmlWeb.PostResponseHandler(AddressOf event_OnAfterResponse)
voWeb.PreHandleDocument = New HtmlAgilityPack.HtmlWeb.PreHandleDocumentHandler(AddressOf event_OnPreHandleDocument)
Return voWeb
End Function
Private Sub sbAddPostDataTo(pvoRequest As Net.HttpWebRequest)
Dim vsPayload As String = proFormElements.fnsAssemblePostPayload()
Dim vabyteBuffer As Byte() = Text.Encoding.UTF8.GetBytes(vsPayload.ToCharArray())
pvoRequest.ContentLength = vabyteBuffer.Length
pvoRequest.ContentType = "application/x-www-form-urlencoded"
pvoRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11"
pvoRequest.GetRequestStream().Write(vabyteBuffer, 0, vabyteBuffer.Length)
End Sub
Private Sub sbAddvoCookiesTo(pvoRequest As Net.HttpWebRequest)
If (Not IsNothing(voCookies)) Then
If voCookies.Count > 0 Then pvoRequest.CookieContainer.Add(voCookies)
End If
End Sub
Private Sub sbSaveCookiesFrom(pvoResponse As Net.HttpWebResponse)
If pvoResponse.Cookies.Count > 0 Then
If IsNothing(voCookies) Then voCookies = New Net.CookieCollection
voCookies.Add(pvoResponse.Cookies)
End If
End Sub
Private Sub sbSaveHtmlDocument(pvoHTMLDocument As HtmlAgilityPack.HtmlDocument)
proHTMLDoc = pvoHTMLDocument
proFormElements = New clsFormElementCollection(proHTMLDoc)
End Sub
Protected Function event_OnPreRequest(pvoRequest As Net.HttpWebRequest) As Boolean
sbAddvoCookiesTo(pvoRequest)
If vbIsPostingInProgress Then sbAddPostDataTo(pvoRequest)
Return True
End Function
Protected Sub event_OnAfterResponse(pvoRequest As System.Net.HttpWebRequest, pvoResponse As Net.HttpWebResponse)
sbSaveCookiesFrom(pvoResponse)
End Sub
Protected Sub event_OnPreHandleDocument(pvoHTMLDocument As HtmlAgilityPack.HtmlDocument)
sbSaveHtmlDocument(pvoHTMLDocument)
End Sub
'-----------------------------------------------------------------------------------------------------
'Form Elements class
' Note: This element class will only capture (any) INPUT elements only, which should be enough
' for most cases. It can be easily modified to add other SELECT, TEXTAREA, etc voInputs
'-----------------------------------------------------------------------------------------------------
Public Class clsFormElementCollection
Inherits Dictionary(Of String, String)
Public Sub New(htmlDoc As HtmlAgilityPack.HtmlDocument)
Dim voInputs As Collections.Generic.IEnumerable(Of HtmlAgilityPack.HtmlNode) = htmlDoc.DocumentNode.Descendants("input")
For Each voInput As HtmlAgilityPack.HtmlNode In voInputs
Dim vsName = voInput.GetAttributeValue("name", "undefined")
Dim vsValue = voInput.GetAttributeValue("value", "")
If vsName <> "undefined" Then Add(vsName, vsValue)
Next
End Sub
Public Function fnsAssemblePostPayload() As String
Dim sb As New Text.StringBuilder
For Each voKeyValuePair In Me
Dim vsValue = System.Web.HttpUtility.UrlEncode(voKeyValuePair.Value)
sb.Append("&" & voKeyValuePair.Key & "=" & vsValue)
Next
Return sb.ToString.Substring(1)
End Function
End Class
End Class
Just make the above into a class object and instantiate it. The usage example is in the comment. You want the vsResponseHTML string.
I'm trying to make an asynchronous upload operation but I got this error message:
Error occurred, info=An exception
occurred during a WebClient request`.
Here's the upload function:
Private Sub UploadFile()
Dim uploads As HttpFileCollection
uploads = HttpContext.Current.Request.Files
Dim uri As Uri = New Uri("C:\UploadedUserFiles\")
Dim client = New WebClient
AddHandler client.UploadFileCompleted, AddressOf UploadFile_OnCompleted
For i As Integer = 0 To (uploads.Count - 1)
If (uploads(i).ContentLength > 0) Then
Dim c As String = System.IO.Path.GetFileName(uploads(i).FileName)
Try
client.UploadFileAsync(uri, c)
Catch Exp As Exception
End Try
End If
Next i
End Sub
Public Sub UploadFile_OnCompleted(ByVal sender As Object, ByVal e As System.ComponentModel.AsyncCompletedEventArgs)
Dim client As WebClient = CType(e.UserState, WebClient)
If (e.Cancelled) Then
labMessage.Text = "upload files was cancelled"
End If
If Not (e.Error Is Nothing) Then
labMessage.Text = "Error occured, info=" + e.Error.Message
Else
labMessage.Text = "File uploaded successfully"
End If
End Sub
Update 1:
Private Sub UploadFile()
Dim uploads As HttpFileCollection
Dim fileToUpload = "C:\Demo\dummy.doc"
Dim uri As Uri = New Uri("C:\UploadedUserFiles\")
Dim client = New WebClient
AddHandler client.UploadFileCompleted, AddressOf UploadFile_OnCompleted
client.UploadFileAsync(uri, fileToUpload)
End Sub
client.UploadFileAsync(uri, fileToUpload) is throwing this error message
Error occured, info=System.Net.WebException: The request was aborted: The request was canceled. at System.Net.FileWebRequest.EndGetResponse(IAsyncResult asyncResult) at System.Net.WebClient.GetWebResponse(WebRequest request, IAsyncResult result) at System.Net.WebClient.DownloadBitsResponseCallback(IAsyncResult result)
By calling GetFileName you're truncating whatever path information was there. You should provide UploadFileAsync with a full path name.
Also, replace e.Error.Message with just e.Error so you get the full error details including inner exceptions. This will provide more info and probably lead you to the answer.
Dim uri As Uri = New Uri("C:\UploadedUserFiles\")
The above statement is wrong, how come you have Uri as a file system path, it should be "http://" or "https://" , if you are trying to upload it to your local asp.net web site project then you must have a url something like http://localhost:PORT/UploadedUserFiles ... and you will know port number when you execute project.
Dim uri As Uri = New Uri("http://localhost:PORT/Upload.aspx")
Try this:
1) Create a new class MyWebClient.
Class MyWebClient
Inherits WebClient
Protected Overrides Function GetWebRequest(address As Uri) As WebRequest
Dim req = MyBase.GetWebRequest(address)
Dim httpReq = TryCast(req, HttpWebRequest)
If httpReq IsNot Nothing Then
httpReq.KeepAlive = False
End If
Return req
End Function
End Class
2) Use this class instead of the default WebRequest.
Private Sub UploadFile()
Dim uploads As HttpFileCollection
uploads = HttpContext.Current.Request.Files
Dim uri As Uri = New Uri("C:\UploadedUserFiles\")
Dim client = New MyWebClient
AddHandler ...
I hope this helps.
From the reference to HttpContext.Current.Request.Files I am assuming that your are running the code in a web project. You don't have to use WebClient to save the file to your disk. All you have to do is this:
Private Sub UploadFile()
Dim uploads As HttpFileCollection
uploads = HttpContext.Current.Request.Files
Dim path As String = "C:\UploadedUserFiles\"
For i As Integer = 0 To (uploads.Count - 1)
If (uploads(i).ContentLength > 0) Then
Dim p As String = System.IO.Path.Combine(path, System.IO.Path.GetFileName(uploads(i).FileName))
uploads(i).SaveAs(p)
End If
Next i
End Sub
I'm not that good with VB.. so there might be syntax problems :) bear with me..
You can't upload files from the ASP.NET web request asyncronously via this mechanism. The file is actually in the body of the http request so it is already on the server. Request.Files provides a stream to the files that are already on the server.
You could use something like Silverlight or Flash from the client side.
You might want to investigate the AJAX Control Toolkit and use the AsyncFileUpload control.
The control has a server side event when upload is complete. (UploadedComplete)
It also has a SaveAs(string filename) method for the uploaded file.
Simple Usage:
Markup
<asp:AsyncFileUpload ID="upload" runat="server" OnUploadedComplete="displayFile"/>
Code Behind
protected void displayFile(object sender, AjaxControlToolkit.AsyncFileUploadEventArgs e)
{
upload.SaveAs("C:\\text.txt");//server needs permission
}
Try to add Header information to the WebClient Object as below :
With webClient
.Headers.Add("Content-Type", Web.MimeMapping.GetMimeMapping("C:\filename.txt"))
.Headers.Add("Keep-Alive", "true")
.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)")
.Credentials = New Net.NetworkCredential("UserName", "password")
End With