.NET library to interact with a webpage programmatically - asp.net

I need to parse some information from a website.
I know there are some testing tools that do that, but I need it to be done in real time, from ASP.NET webpage code. So I guess, I need a .NET library.
Previously I was just able to generate a specific URL and parse the HTML but now I have to get information from a page that is generated by AJAX calls after form clicks. So I have to generate those clicks, right?
BTW I only have (limited) experience with server side ASP.NET. Maybe the optimal solution is quite different to what I suspect?

You may take a look at WatiN.
From the homepage:
Automate all major HTML elements with ease
Find elements by multiple attributes
Native support for Page and Control model.
Supports AJAX website testing
Supports creating screenshots of webpages
Supports frames (cross domain) and iframes
Handles popup dialogs like alert, confirm, login etc..
Supports HTML dialogs (modal and modeless)
Easy to integrate with your favorit (unit) test tool
Works with Internet Explorer 6, 7, 8, 9 and FireFox 2 and 3
Can be used with any .Net Language
Licensed under the Apache License 2.0
Downloaded more than 120.000 times.
And since its open source you can add and contribute new features yourself!

Take a look at Coypu. It wraps WatiN and Selenium giving you helper methods that make this kind of work a lot easier.

Related

how complex can an ASP.NET web page be?

I'm building a Bridge game.
The main page, where you play the game, boast some 200 controls, like buttons that represents the 52 cards, 26 others for won / lost tricks, plenty of text fields... You name it, total some 200.
Actually its been already built and runs under Winforms model, but with the unavoidable question of implementing .NET 4.0 onto user's machine that must run Windows, and installing the software.
Hence I'm considering creating an ASP version (no PHP please, since I'm fluent in VB, and ignorant in C, Java, PHP and the likes).
Is that a "real" option or am I dreaming?
Will it run smoothly for users with a 2MB DSL connection?
Oh, by the way, no MVC please, I'm too old for that sort of sh...t :->
Can it be done? Sure.
The problem for you might be that to do it effectivly you will need to use HTML, CSS, Javascript and the like.
What most likely would be a good idea for you here would be to go the route of Silverlight. It will give you a more rich UI experience and will allow you to code in VB entirely.
I don't see using ASP.BET directly as a good option here. To make it look smooth, you would have to use javascript heavily. I thing you would be better off with Flash.
If you prefer to use pure HTML/Javascript/CSS you can create rich Web UI's by modifying the Document's Object Model (DOM). With little knowledge of Javascript and the usage of some library like: scriptaculus, prototype, sarissa, jquery, you can get rich interfaces. You should take a look at some of these libraries mainly scriptaculus because the set of animations it provides. Another javascript library made for game development is gameQuery.
With that many controls on an ASP.NET page your view state will be gigantic. If you plan on only serving up a couple of games at once then it shouldn't be a problem but you have to actually start thinking about bandwidth with that large of a viewstate. You would be better off with a DOM/jQuery type interface which displayed static images for the cards. All interaction can be handled client side and validated with a server side call.
This might be the perfect opportunity to learn javascript :). Remember each language you learn is another tool in your toolbox. The engineer with the most tools usually wins :).

Interactive Graph Visualisations in ASP.NET Website (Drag/Select/Link/Unlink)

We have a requirements to create a Website (ASP.NET v4.0) which displays a Graph. It should be able to do:
Display nodes (with names and colours)
Display links between the nodes, with text on the link (e.g. '85%')
Interact with nodes/links to drag/move/select
Layouts out the nodes in a clear manner automatically
Can add/remove nodes (asynchronously) and link/unlink easily
Javascript interaction with events (onClick, onDrag)
Events must provide identification of selected nodes/links (Javascript).
Ability to zoom in/out (ideally)
Updates Asynchronously (rather than full postbacks)
Responsive when displaying >100 nodes
Flash is not supported
MUST support IE6 (just don't ask...!)
Development is Visual Studio 2010 on .Net Framework v4.0
We were currently using the Syncfusion Diagram tool (v. 6.1.0.34) running on v2.0, but recently upgraded to v4.0 and a breaking change in System.Web ViewState management means we need to find an alternative. Its possible the latest version (v8.3) is much better, but we're reclutant to fork out a few thousand pounds for the licenses if its just as bad.
We found the Syncfusion tool ok, but it was very difficult to code against (without manual hacks) and it performed quite badly with large graphs when it loads 200 images from the server.
Really looking for some inspiration from your guys. Any suggestions or experiences shared would be most helpful.
Thanks in advance.
mxGraph is designed for this type of functionality (disclaimer - I do work for them). It does support IE 6 and is entirely written in JavaScript. It comes with .NET backend server classes to perform comms with the JavaScript client. In order to get responsive behaviour with over 100 nodes on IE 6, you need to switch to using a server side image over about 50-60 nodes, since IE 6 does perform very badly. We include an example to demonstrate how to do this. Give it a try, if you require evaluation support, there is a forum for that.

Do I need AJAX and ASP.NET for what I'm doing?

Easy question. I used to develop websites back in the days of "classic" ASP, and when I'm asked to do a quick and dirty website for family or friends now, I still resort to direct HTML/ASP and some basic CSS and Javascript - I can get the sites up pretty quickly this way. However, I've had a few requests to design and develop some sites for pay, and thought I should catch up on my web skills. I have been using .NET 3.5, XAML/WPF, etc. for Windows apps, so I'm up on .NET, I'm just behind on the web end.
To the question: If I want to design/code a site that looks identical on all (at least somewhat recent) browsers and platforms, should I be using ASP.NET and AJAX? There might be a little database activity on the site, but not much, so I don't need an enterprise level, multi-tier extendable architecture... just something that looks good and works on multiple platforms without having to code all variations for each browser. After looking at all the ASP.NET books at the bookstore, it seems they all focus mostly on data and postback stuff. Is it still a legit option to use some basic, boring html and javascript with some Flash embedded where needed?
Let me know if I need to clarify the question. Thanks for your advice in advance!
Your question is more loaded than you think, but let me try to address a few points that I think are relevant.
First, how a site looks is almost completely dependent on the HTML/CSS you use and how you code the front end of the site and only slightly dependent on the server technology. So if you want your site to function across browsers and platforms, learn to code following web standards, with semantic markup. (Search on those terms for more info).
Also, ASP.NET comes in two flavors now: ASP.NET MVC and normal ASP.NET. I highly recommend, if you are going to get into ASP.NET, that you follow the MVC platform. It closely follows similar technologies (like Ruby on Rails) and will make the transition to other MVC platforms easier on you. Also, the MVC platform doesn't try to output as much pre-made HTML as straight ASP.NET will when you use their "drag and drop" controls.
Secondly, it really depends on the sites you are building, but straight JS (or JS + jQuery), CSS, and HTML -- and please don't use Flash unless you are embedding a video -- will actually work for a number of basic sites. If you need some things to happen on the server, PHP makes for a great platform. If you are working with advanced database access, and program flow, and since you are already familiar with .NET, then stick with it... MS has some great tools and resources to help you out.
Finally, a lot of developers use a favorite CMS or blogging platform as the backend of simple sites that still need the ability to manage the content easily. Expression Engine (CMS) and WordPress (Blog/Lite-CMS) are often used (both PHP based) but there are tons out there.
Good luck stepping up your game!
I would recommend learning jQuery. This will give you a browser independent abstraction for your JavaScript.
ASP.NET controls will render it's controls in a browser independent way, but that doesn't mean your site will automagically be browser independent. You still need to know how elements are rendered differently in different browsers.
I'd also recommend using a CSS Reset sheet as a starting point for your CSS.
All in all, if you've been developing old school ASP, you'll probably really love ASP.NET as it will save you a lot of time and looping. You may want to jump right in to ASP.NET MVC too.
To the root of your question, I'd learn ASP.NET if you're doing anything more than a simple brochure site. If you have .NET experience, and classic web development experience, then learning ASP.NET is not going to be a big hurdle and will be well worth the effort.
I agree with Aaron Daniels' answer about learning jQuery. jQuery helps a lot with cross-browser compatibility in JavaScript and some CSS-based effects.
However, you should also look into ensuring your site uses well-formed, valid HTML, and doesn't use too many CSS 2+ features. This should ensure that your site is standards compliant, which will mean it will play well with Firefox, Safari, Opera, and even later versions of IE to an extent. You will still need to do manual tweaking for IE - it's been too broken for too long for MS to be able to fix it properly in one go - so look into conditional comments for applying a separate stylesheet for IE users.
AJAX is a handy technology for "desktopifying" your web app. It provides a mechanism for asynchronous callbacks to the web server, so you can pass data to and fro without reloading the page in the browser window. This is how the voting buttons work on StackOverflow, for example.
Lastly, ASP.NET doesn't really have much effect on the end user's experience in terms of the look and feel of the site. It is a server technology that provides for writing complex applications to be delivered over the web to a browser. Having said that, MS have put in some extra goodies to make working with AJAX a little easier.
Hope that helps!

Which Javascript history back implementation is the best?

There are implementations for history.back in Micrososft AJAX and jQuery (http://www.asual.com/jquery/address/).
I already have jQuery and asp.net ajax included in my project but I am not sure which implementation of history.back is better.
Better for me is:
Already used by some large projects
Wide browser support
Easy to implement
Little footprint
Does anybody know which one is better?
EDIT:
Another jquery plugin is http://plugins.jquery.com/project/history It is recommmended in the book JQuery Cookbook. This one worked well so far.
One alternative to jQuery Address is the nice jQuery history plugin. There are also URL Utils.
Reference: AJAX History and Bookmarks.
If you are building an ASP.NET application then using ASP.NET Ajax Framework gives you many advantages and a nice-simple API to use server-side.
Below you can find an example that uses Browser History with ASP.NET Ajax
Create a Facebook-like AJAX image gallery
Both have a wide range support in browsers.
For me is easier to integrate Microsoft AJAX Framework in an ASP.NET page so again if you have an .aspx page it might be easier to work with ASP.NET AJAX
If you don't need exactly AJAX, i.e. updating only parts of the site on request is sufficient for you, then you can use invisible iframe as target for loading generated HTML file containing only JS script that updates/resets "updateable" parts of the site. This is cross-browser solution and doesn't need address polling.
Example, but not in ASP: kociszkowo.pl (Polish site)
When you click there in the section icon and your browser supports javascript, link is modified before being fetched - target is changed to iframe and href is suffixed with .dhtml to inform server, that we're interested in a special version of the page. If you press Back in your js-equipped browser, then previously fetched iframe page is loaded from the cache. Simple, but requires some decisions at architectural level.
This link modification is irrelevant here, it's just the result of combining JS/non-JS world.
In my experience, your best bet is using the same one that you have doing most (if not all) of your ajax calls. For instance, if you're using asp:UpdatePanel's, use the MS one - if you're using jQuery.ajax, use the jQuery history plugin. If you're doing a mix (which I've tried to avoid in my projects), I'd personally test with both and see which behaves better - if they both test fine, then it's a bit of preference. Some may argue the Microsoft one would have better support, but jQuery's history plugin may get more use and more mature.
http://msdn.microsoft.com/en-us/library/system.web.ui.updatepanel.aspx
http://docs.jquery.com/Ajax/jQuery.ajax#options

AJAX form editing and design

I want to build into mt ASP.NET application the ability for users to fill up forms, but the forms are not something I can hard-code into my (Enterprise) software.
So I need one screen that lets end users create the forms. Doing it the 2.0 way, I just love what PollDaddy did in their survey editor (great job guys!). How do I replicate that? (don't worry pollDaddy, my app has nothing to do with survey and I am not at all in your domain (-: )
Since you need to login to the site to play with it, I made a 20 sec video of how it looks like. Please see below and tell me how to best replicate this (as simple as using JQuery tip? use an entire open source project? Buy something?)
Requirements are:
support multiple types of elements (like multiple choice, free text, comboBox)
drag and drop
editing of order
click image for video or here
http://frame.revver.com/frame/120x90/1376799.jpg
The AJAX control toolkit is free and would work for some of this items shown in the video. However, JQuery has tons of plugins it is easy to extend and even easier to use. I would suggest starting there and extending it where needed.
Good Luck
You could build it yourself using an AJAX framework like ASP.NET AJAX or even telerik controls. You could use jQuery but ASP.NET AJAX is free and embeds well with ASP.NET pages. You could buy something, but I've yet to encounter packages as such.

Resources