Crawler for Deep-Web calling ASP.NET page - asp.net

Introduction
I'm developing a web-crawler and I need get the return of ASP.NET forms.
I know about the difficult to try that, but my crawler can get the return of PHP forms or languages like that. I'm using HtmlAgilityPack and call with POST parameters. That works fine.
Problem
My problem starts when I need crawler ASP.NET Forms, with a lot of hiddens fields and other the information (like __VIEWSTATE or __EVENTARGUMENT) for ASP.NET Kernel.
Question
How Can I put for my crawler send a POST call and receive the correct return with the data base filters? Somebody have information about that? Somebody tried something like that?
Thank's everybody.
I'm waiting for a good response.

Related

ASP.NET (C#) do a post to page from code and go to that page

I am very new to .NET and am now in need to modify an existing ASP.NET website done with C# long back by someone else (with .NET version 2.0).
There are lot of process involved in this website and I need to get few data from the database and then do a POST to another page from the code itself. I am able to do a redirection to required page with necessary parameters, but need to implement the same with POST.
Can anyone please guide me on how to implement this? Thank you.
After few more searches on my requirement, came across another Stackoverflow post - https://stackoverflow.com/a/2802848/2451196.
This was a much straight forward approach for me and it worked like a charm. Thanks to Pavlo Neiman (https://stackoverflow.com/users/164001/pavlo-neiman)

How to display a web service (SOAP) JSON result in a Drupal View

I'm a Drupal beginner, and I would like to display a web service (SOAP) result (in JSON) in a page.
I would like to display the result in a table (so I think Drupal View would be nice to display it...), but I don't know how to bring the Web Service result to display it into a View.
I am using the module "wsclient" (http://drupal.org/project/wsclient) together with "Rules" to execute a web service method, and I am able to get its result into a message, but that's all I was able to do until now.
I'm new to Drupal. Can someone help me please, to perform this action in the best way?
I am using Drupal 7.
Thank you!
There's a feature request in the wsclient issue queue for views integration: http://drupal.org/node/1215570
EDIT: as pointed out in the comments below, the following does not apply to the original question:
Have you considered using the Services module instead? There is already views integration for it.

Login to asp.net site from another app, then receive filestream

I have an ASP.NET application with pages that use reportviewer. Can someone give me a hint on how to approach the following requirement:
I want to get the report as PDF file from the page, without user interaction. I know I can render the report to a filestream, but since there's no user opening it in a browser, I need to collect the filestream from another application that might run during the night.
There might be other approaches, like a webservice for example that could return the filestream to me, but this would also mean, I have to modify the setup of the datasources that the report receives it's data from. There are a lot of controls on the page, for supplying filter parameters. By using the page life cycle I can use what's already there.
I thought about wget, but haven't tried it yet, and I'm not sure how complicated logging in will be with cookies. I do have full control over the asp.net application though, so if I can modify something there to make it easier, I'd do it.
You can use the "WebClient" in .net application to get the response from the site.
Please refer the below link:
http://msdn.microsoft.com/en-us/library/system.net.webclient(v=vs.80).aspx

How to prevent users from manipulating html content "badly" and posting it back to server?

I have not found the same as or related to my question above.
If it is already asked, please let me know, I will delete mine.
I am learning asp.net mvc 2.
After downloading the rendered page, the visitors have a chance to manipulate the html contents ILLEGALLY and submit it back to the server. In MVC, how can I avoid this issue?
You cannot prevent anything on the user side. Therefore you must ensure that the server reacts correctly to manipulated input.
Note that the automatic model binding is tricky: both over- and under-posting can cause trouble.
This is an excellent post on that topic: http://bradwilson.typepad.com/blog/2010/01/input-validation-vs-model-validation-in-aspnet-mvc.html
Take all the efforts you can to validate the user input at the server. MVC offers the AntiForgeryToken which helps verify that the page being posted back is the correct page but as with any client-side verification, it's not foolproof.
You can't do anything about this.
Use the mantra "never trust anything from your users" and validate everything on server side.

Legitamate cross site communication

I am building a website, within a large intranet, that wraps and adds functionality to another site within the same intranet. I do not have access to the other site's source and they do not provide any api's for the functionality they provide. I need to, somehow, have my server-side code go to that site, fill in some forms, then press a submit button.
Is this possible? If so, how can I accomplish this?
Note: I am working in asp.NET if that matters at all.
Not the most efficient, but maybe WatiN can get you started:
http://watin.sourceforge.net/
Just look at the URL the form is supposed to submit to and the method it employs (POST or GET) and then send a request to that URL using the same method and put the field you want as parameters
Your server-side code is basically a web client to the other web site. You will need to write the code to send the HTML form data to the other web site and process the response. I would start with the System.Net.WebClient class. Take a look at System.Net.WebClient.UploadValues. That class/method will enable you to POST the form data to the web site via a NameValueCollection.

Resources