EVENTVALIDATION error while scraping asp.net page - asp.net

I need to get some values from this website. Basically I need to get the Area for every city. I am using Python and beautifulsoup for this. What I am doing is :
First making a Get request to this page and getting __VIEWSTATE AND __EVENTVALIDATION to make a POST request to get cities for a particular state.Till here its working and I am getting cities for every states.
To get Area I need to make another POST with new __VIEWSTATE AND __EVENTVALIDATION and this time i need to send city as well with other parameters.But I am getting error here:
505|error|500|Invalid postback or callback argument. Event validation
is enabled using <pages enableeventvalidation="true"> in configuration
or <%# Page EnableEventValidation="true" %> in a page. For security
purposes, this feature verifies that arguments to postback or callback
events originate from the server control that originally rendered
them. If the data is valid and expected, use the
ClientScriptManager.RegisterForEventValidation method in order to
register the postback or callback data for validation.|</pages>
I have checked each and every argument/parameter in firebug that I need to send to get areas for corresponding cities but no success. Maybe according to me problem is with __VIEWSTATE or __EVENTVALIDATION.
please help me

you have to get and pass the __VIEWSTATE and __EVENTVALIDATION variables in one session call, otherwise when doing the second POST call while reopening connection to the server it expects to receive back already another __VIEWSTATE and __EVENTVALIDATION!
So basically the logics should be as follows:
curl_init()
curl GET __VIEWSTATE and __EVENTVALIDATION
*some logics here *
curl POST parsed out __VIEWSTATE and __EVENTVALIDATION
* some logics here *
curl_close();

Try to add __REQUESTDIGEST form field, it is a validation field used by Sharepoint.
Also, add name of the select element with states as the value of __EVENTTARGET field:
__EVENTTARGET ctl00$ctl25$g_4828177f_5427_4b0f_a968_ad02dc3674ec$ctl01$ddlState
__REQUESTDIGEST 0xDF6A6E14C550004C4A8533A9F5D...,14 Jun 2012 10:13:34 -0000

Related

How to eliminate false success messages when implementing post-redirect-get pattern?

When implementing the post-redirect-get pattern in a web application, it is common for the final step in your server code to look something like this (pseudocode):
if (postSuccessful)
{
redirect("/some-page?success=true")
}
That is, the redirect URL has some kind of success parameter in the query string so that you know when to display a nice looking "Your form has been submitted!" message on your page. The problem with this is that the success=true persists in the query string when it's only needed to initialize the page. If the user refreshes the page or bookmarks it, they will receive a false success message even though no additional POST has taken place.
Is there an elegant solution to this that doesn't involve using JavaScript to eliminate success=true from both the query string and the browser history? This solution works, but definitely adds complexity to a page's load process.
You can use server side technology to implement this feature, without any JavaScript. The stes are listed below:
When post is successful, redirect to /some-page with current timestamp information:
if (postSuccessful)
{
redirect("/some-page?success=true&timestamp=1559859090747")
}
When server receives GET /some-page?success=true&timestamp=1559859090747 request, compare the timestamp parameter with the current timestamp, check whether it is within the last 3 seconds (or you can change this number according to the network environment).
If the timestamp parameter is within last 3 seconds, then it means this GET /some-page?success=true request is a result of server redirect. If not, then it's more like a result of "user refreshes the page or bookmarks it".
In server code that handling GET /some-page, render different HTML according to the result of step 3. Display the success message only when current access is a result of server redirect.

Submit ASP.NET forms in parallel

I need to submit one form multiple times in parallel. The server accepts the parameter _ASYNCPOST.
I can explain in an abstract way how the page works
Login
Submit form search (POST)
POST same form with new data (all these need to be done in parallel)
In the last step, I yield all the requests with every parameter I could find (including __VIEWSTATE, EVENTTARGET, etc)
The problem is that the first post works, but the rest return an error saying "The server data does not match the browser data, hit refresh"
Is what I'm trying to achieve possible?
I followed this doc https://blog.scrapinghub.com/2016/04/20/scrapy-tips-from-the-pros-april-2016-edition/

Passing data between different URLs

I need to check where did the incoming request came from before loading a page
ex:
for user to view www.mysite/page1.aspx
request should come through www.othersite/page1.aspx
so on page1 load in mysite i need to check whether the request came from page1 in othersite.
i have tried Page.Request.UrlReferrer but i saw there some posts which tells every browser might not support Page.Request.UrlReferrer.
i can not pass visible parameters on URL.
This is a common issue when you do not want to allow request from arbitrary sites.
What you can do is, create a variable in session and put this variable in the Page1.aspx. When the page posts back, you should get that variable back and it should also match the one stored in the session. If it does not, you can be sure that the request is from some other server.
You can use PostBackUrl on the start page
And access your parameters with PreviousPage in the arrived page
if (this.PreviousPage != null)
{
var control = Page.PreviousPage.FindControl("..."); //Adjust your Id and add cast
}
Nota : This was also created to provide greater security redirection setting.

How to add and access Custom Header in C#

I need to add a custom header something like
MYName: Balaji
which i need to access from .aspx file through
Request.ServerVariables["HTTP_MYName"];
should return "Balaji". I need so many variables like this it will added dynamically.
Kindly help.
Also, I cannot persist this varaibles in any of the .Net controls or objects like
cookies, sessions, application, hidden variable etc., or cannot store this in d/b and get it back whenever is required, I NEED IT ONLY IN HTTP HEADERS.
Kindly send the C# code how to add this variable and get the value back in .aspx file.
What do you mean by "get the value back in .aspx file"? HTTP headers are intended to be used as directives to a browser, how to interpret the given content. You don't have access to these values in your document.
Setting a custom HTTP header is quite easy, however:
Page.Response.AddHeader("MyCustomHeader", "VerySecretValue")
Updated my answer as per your comment.
If you need to transfer information between a HTTPModule and an ASPX page, you can use HTTPContext.Current, since this stays the same in both places.
So, you add it by
HttpContext.Current.Items.Add("SecretKey", "SecretValue");
and read it as
string s = HttpContext.Current.Items["SecretKey"];

Validate Origin of FORM POST to ensure it came from same server/app

I want find a platform/language agnostic solution to ensuring the origin of a FORM POST is from an expected source. I.e. Page1.aspx posting to Page2.php within the same web site.
Specifically what I am attempting to do here is to prevent request forgery.
Use a hidden field in your form, which contains a token your app generated. Store the token in the user session. When the form is submitted, your app will check that the value of the hidden field is identical to the value stored in the user session.
If it is identical, then you know the submitted form comes from where it is expected to come.
Old Thread, but might still be useful.
If you do not have session info set (best option) then you can include a hidden field with an encrypted timestamp then compare it (after de-crypt) to the current time on the process end to make sure it is relatively close and thus as recent as you deem necessary.
You could include into the form a hidden field which would be the SHA1Hash("some-secret" + Remote_IP + PerSessionSecret).
The PerSessionSecret is something you autogenerate in the beginning of the session. "some-secret" is a global secret value - which will help a little bit in case the randomly generated PerSessionSecret turns out not to be very random enough.
Then do the same calculation upon the form submission and you know it's most probably submitted from the same client that it was sent to. (Of course, if you have multiple clients behind the single address, like a proxy or a NAT, you can not distinguish between them reliably).

Resources