What tools, preferably open source, are recommended for driving an automated test suite on a FLEX based web application? The same tool also having built in capabilities to drive Web Services would be nice.
Adobe distributes a test framework themselves: FlexUnit.
I heard of people using selenium as a free/open source testing tool. A quick google revealed a FLEX API for it. Not sure if it works or is still in development, but it may be worth a look.
http://sourceforge.net/projects/seleniumflexapi/
Are you looking to script code-level unit tests? If so, dpuint is the bomb: http://code.google.com/p/dpuint/ . This library makes it really easy to do automated testing on all sorts of asynchronous events, on either non-visual ActionScript objects or visual components. They also have a nice multi-page tutorial on the Google Code project page.
If you are looking for functional testing tools along the lines of automated record-and-playback simulating an end user using a Flex app, HP's QuickTest Pro is the Adobe-endorsed solution. It works great, but costs about $4,000 - $6,000 per seat.
Check out FlexMonkey. It does automated testing via FlexUnit tests.
Try looking at Melomel. It has Cucumber support baked right in and comes packaged with steps for most Halo and Spark components.
http://melomel.info
There's an automated test tool called RIATest that might fit the bill for you.
Unfortunately only for Windows, and not open source, but if it does the job it might be well worth the price ($399 at time of writing).
FunFX is an option for automating UI testing. I haven't used it extensively, but I've heard of some having success with it. Here is the article where I first learned about it.
I've been extensively using FunFX for several months now on a Flex 3 + Rails project. Not only is it open source, it's also written in Ruby, so integration with web services should be fairly easy. There are a few screencasts out there covering the basics.
The Flex code that your Flex app needs is contained in the SeleniumFlexAPI distribution .swc file, SeleniumFlexAPI.swc. Just include this file as a library when you compile your Flex app.
Sikuli is good tool which can be used to test flex/flash based web applications.
-It can automate anything on graphical user interface.
-It works on Windows, MAC OSX and Linux as well as iPhone and Android.
-Here is the Sikuli link
My preferred tool is Selenium Remote Control. There is a plug-in I discovered a few months ago:
http://code.google.com/p/flash-selenium/
This required 'hooks' to be written on the server side (ActionScript/Flex). Once they were added, I was able to do some browser testing using Selenium RC.
FunFX is great. We've used it extensively and have been very happy with it. The community is also active and very responsive, so that is a big plus for me.
The new version of the Selenium-Flex API (0.2.5) works great.
Related
I have to automate a Flex based rich internet application. We have tried out a few options like Ranorex and RIA Test. While the GUI based automation tools (both open source and commercial) do a decent job, the test scripts are hard to maintain and often result in flaky reports. PhantomJS has worked for me in previous projects very well but I am not sure if it works with Flex. Can anyone suggest if PhantomJS works at all with Flex?
I don't know anything about flex, but the tag excerpt suggests that it extensively uses Adobe Flash and Adobe Air. PhantomJS doesn't support Flash or other plugins (anymore). There are forks which enable flash, but they are a little behind with the versions.
Honestly i think this might have as much to do with how you build the automation tests that makes them hard to maintain. For instance with Ranorex instead of building 5 different recordings to test 5 different things build 15 tiny recordings that do only one thing, that can be pieced together in 5 different ways, then to maintain your scripts you just need to maintain those 15 recordings, and if all the scripts break at the same place the maintenance simply becomes ok why did that one tiny recording fail rather than ok let me maintain this suite of 5 recordings it's let me fix this one recording, and that will fix the larger recordings.
Try and use Testcomplete identifies most of the complex Flex controls not need for any compilation of helpers with your application libraries and if you have a team of dedicated automation engineers they should easily be able to create a reusable and maintainable GUI based automation pack around Testcomplete (vbscript)
Please suggest which tool should I use - QTP, RIATest, Selenium for Flex Application Functional Automation Testing. Please help me compare them with different advantages and disadvantages.
Any help appreciated.
The big picture is:
QTP - sophisticated, expensive, can automate almost anything, not just Flex applications, works on Windows.
RIATest - simple, inexpensive, designed specifically and only for Flex application, works on Windows and Mac.
Sorry, no experience with Selenium.
I don't have direct experience with it, but there is a Selenium Flex API available; since Selenium is free and open source, it's a great option if your budget is tight.
I do have experience with Selenium in general and can highly recommend it; it's relatively easy to learn and works on any platform with a multiplicity of browsers.
Recently I evaluated the above tools for our Orgnisation.
I say go with QTP. The configuring of qtp for flex is easy.
I wrote a hub about it.
http://hubpages.com/hub/Automating-Flex-Applications-with-QTP
Please ask questions if you have any.
Follow me n my hub
QTP does work with Flex apps...just have to have the right versions.
I was using Selenium to test Flex for a while, then Sun came out with a new plugin for Java in the browser with 1.6.14 I think it was, and I wasn't able to get it to work after that, though I admit its been quite a while since I tried.
I recently stumbled upon flex/openlaszlo (OL). Both seem very useful and I have a few questions about them:
What is needed to deploy flex apps? From what I understand, the flex sdk
is open source, but the other tools
(for development and deployment) are
proprietary.
What is needed to deploy openlaszlo? Is it completely open
source (from development to
deployment), or does it have
development/deployment "gotchas" like
flex?
Specifically, I'd like to use flex or openlaszlo to either augment or
replace an editable table I created
using js, ajax, html, and css. Is this
the type of thing flex/OL can/should
be used for? Are there any drawbacks
or pitfalls to using flex/OL for this
rather than straight js, ajax, html,
css?
Thanks.
Edit: Are there any licensing (use) restrictions on applications built on flex? i.e. applications built on flex can be for only non-commercial use, unless a commercial license is purchased?
The following link has some discussion about openlaszlo and flex, it may help you:
Use the best open source client-side framework for cloud computing
You do not need anything special to deploy Flex apps. Just put the compile app (a SWF file) on any web server. Flex apps can talk to any back-end.
Flex SDK is free, but Flex Builder is not. You can use the free and open source FlashDevelop to write flex apps - it doesn't have drag-n-drop features like flex builder, but it offers code hinting and stuff. I don't know about openlaszlo.
Below is the link providing details of tools/IDEs for developing OpenLazlo applications
http://wiki.openlaszlo.org/Development_Tools
Re: "What is needed to deploy openlaszlo? Is it completely open source (from development to deployment)"
OpenLaszlo is OpenSource, but the typical versions you install come with the main components pre-compiled into a SWF for the SWF run-time. However, you can download the full source code if you wish to look at it and/or compile the core yourself:
Last official released version (4.9.0): http://download.openlaszlo.org/4.9.0/openlaszlo-4.9.0-src.tar.gz
Nightly builds:
http://download.openlaszlo.org/nightly/trunk/ (you will see "source" as an option after you click the link of the version you want)
OpenLaszlo does not require anything else to be deployed but itself, except if your application is compiled to the SWF run-time then the user will need the Adobe Flash player installed in their browser to use it.
I'll answer your last question: the biggest drawback to using Flex is that it requires the client to have the Flash Player plugin installed in their browser. Not that big a deal for most people since Flash Player is over 98% of all computers. With the straight Javascript, AJAX, HTML, CSS approach it should work on all browsers, assuming you wrote it correctly.
What is the best way to test my webforms applications?
Looks like people are loving Watin, and selenium.
Just wondering, why would you call WatiN a unit testing tool? Last time I checked, it ran integration tests.
The best way would be to move all code that doesn't depend on HttpContext to a separate assembly and run unit tests as usual. The rest can be tested with Ivonna. She doesn't test the client behavior, that's where WatiN can be helpful; however, if you want to test your pages or controls in isolation, she's your only choice.
UPDATE: Given WatiN has been stagnant for over a year now, I would direct anyone that needs web ui tests towards selenium, it is in continuous use & development by many contributors, and is actively used by Google.
WatiN is the best that I've found. It integrates into Visual Studio unit testing or nunit & you can do pretty much anything you need in the browser (click links, submit forms, look for text/images, etc.)
See the following questions for similar answers:
What is the best way to do unit testing for ASP web pages (C#)?
Web Application Testing for .Net (watin Test Recorder)
How do you programmatically fill in a form and ‘POST’ a web page?
That's the biggest shortcoming of Webforms -- it's, for all practical reasons, untestable in terms of unit testing of testing controllers, etc.
That is one of the major advantages of the MVC framework.
I tend to favor the approach of separating the buisness logic out of the UI code.
Here's an article that describes a unit test friendly pattern (Model-View-Presenter)
http://www.unit-testing.net/CurrentArticle/How-To-Use-Model-View-Presenter-With-AspNet-WebForms.html
I would use a tool like WaitIn:
" WatiN is Web Application Testing in .NET, and this Test Recorder will generate chunks of source for you by recording your clicks in an embedded IE browser" (from Scott Hanselman's blog - which I found thanks to another post on StackOverflow
WaitIn website
I'd go with WATIR (Web Application Testing in Ruby) - http://wtr.rubyforge.org/. We (Acsys Interactive) have been using for about a year and the tool is great.
I developed a simple wrapper in .NET so that I can execute my WATIR scripts from Unit tests. The framework is incredible and you have entire Ruby power behind you. There's support for Firefox & Safari (FireWatir project).
It's very similar to WATIN (in fact I think WATIN was inspired by WATIR) but I find that WATIR community is much larger than WATIN one.
There're test recorders out there that you can use and tons of tutorials.
It's really your choice. If you feel like the tests need to be in .NET and you don't want to support any other language then your choice is WATIN. On the other hand, if you want to try a fun and quite powerful scripting language (that's what Ruby is) then go for WATIR.
Question to WATIN guys, does it support FireFox/Safari?
Here is a review of Watin,Watir and Selenium
http://adamesterline.com/2007/04/23/watin-watir-and-selenium-reviewed/
Apparently Selenium worked quite slow for the tester but if you'll notice, as one of the comments points out, that this is only the case due to its support of multiple browsers.
However there is a CTP (Community Technology Preview) release of WatiN which offers support for both Internet Explorer and FireFox automation.
I have had a great experience using Selenium. Web tests can be very fragile, but here is an article from my blog where I talk about how to make the tests less fragile.
http://www.unit-testing.net/CurrentArticle/How-To-Make-Web-Tests-Less-Fragile.html
Is it realistic to try and learn and code a Flex 3 application without purchasing FlexBuilder? Since the SDK and BlazeDS are open source, it seems technically possible to develop without Flex Builder, but how realistic is it.
I would like to test out Flex but don't want to get into a situation where I am dependent on the purchase of FlexBuilder (at least not until I am confident and competent enough with the technology to recommend purchase to my employer).
I am experimenting right now, so I'm taking a long time and the trial license on my Windows machine has expired. Also Linux is my primary development platform and there is only an alpha available for Linux.
Most of the documentation I've found seem to use Flex Builder.
Maybe I should use Laszlo...
IntelliJ IDEA works as a Flex IDE, if you happen to also be a Java developer. It's free if you contribute to open source projects.
Check out FlashDevelop for Windows. I like it better than Flex Builder.
I've been using Flex since version 2 and Flex3/BlazeDS since it came out of beta. I also have some experience with Lazzlo and the difference is day and night (Flex rocks!). I have not regretted once using Flex. Regarding FlexBuilder, it is worth every penny. While it is completely possible and reasonable to write Flex application without FlexBuilder, the productivity gains of using it will more than recoup the investment. Try the evaluation for 30 days and compare it to some of the other options suggested about (I'm going to try FlashDevelop).
Some things you get with FlexBuilder include:
Code completion
Visual editor
Debugger (it is fantastic!!)
Profiler (also very good)
Regarding Linux, the alpha version of FlexBuilder does not have a visual editor. Other than that, I understand it is reasonably feature complete, still free, and many of the Adobe employees I've talked with that use Linux are happy with it.
FlashDevelop is really easy to setup with the Flex SDK. Just download FlashDevelop, then download the Flex SDK. In FlashDevelop go to Tools > Program Options > AS3Context (under Plugins) > Set the "Flex SDK Location" to the root of the folder you extracted the SDK to and build away. FlashDevelop even has a basic MXML project that will get you going.
If you use ColdFusion for the backend, having FlexBuilder in Eclipse and CFEclipse can mean one less IDE to have to get familiar with.
I'm going to join the choir here and say FlashDevelop for an alternative. The only reasons you might want FlexBuilder are:
Flex charts
Step-through debugging.
Profiler (I haven't used it)
Visual style editor
However, the code-completion and general bloody-awesomeness of FlashDevelop's code-completion and syntax highlighting knocks the gimpy eclipse crap out of the water. So, pretty much what Todd said, except for the code-completion part. Flex Builder is very flakey in that department.
Short answer: Yes
I'm working on a team of developers and designers. We code our .MXML and .AS in FlashDevelop 3 and our designer creates .FLA with skins and widgets that get [Import()]ed in ActionScript.
I wrote a little more about this subject here:
Flash designer/coder collaboration best practices
I have been using FlashDevelop for along time (4/5 years), I am actively using it to develop Flex4.5 applications, it has built in support for code completion, it has a profiler and a debugger that work excellently. The IDE itself is responsive and require the .Net framework, in fact here, I'll list some stuff.
FlashDevelop Pros
Free IDE
Code completion feature
Very capable Debugger
Profiler
Documenting
Ability to build Air / Flex files
Templating
Plugins
FlashDevelop Cons
Lack of UI desing support
.Net support only (Won't work with Mono)
Everything else is pretty simple to get running with, the instructions are available at http://www.flashdevelop.org/
Absolutely. I've been a Flex developer since Flex 2 and until recently I've used my regular editor, TextMate, for coding and Ant for building. TextMate has some good extensions for ActionScript and Flex coding, but I think you could get that for any decent editor.
What's been missing from my setup is a usable debugger, the command line version is a pain to work with. Because of that I've been starting to use FlexBuilder on the side, using it in parallel with my regular setup.
Having a profiler doesn't hurt too.
I've been using FlexBuilder for awhile now and just started to switch to using Eclipse with Flex SDK. I work for a non-profit so the word FREE is huge.
Initially, it is fairly intimidating so if you have the money, you might want FlexBuilder.
There is a lot you need to know and do if you use the SDK. The learning and experience may pay off though... I am still undecided myself.
I second FlashDevelop. You don't get the visual design stuff for the MXML, but for the code (both MXML and AS) it's excellent.
I also use FlashDevelop when working on AS3 projects. For me, the ugliness (UI design) and sluggishness of Eclipse/Flex Builder is enough of a deterrent to stay away from Flex Builder.
In addition to the weaknesses of FlashDevelop pointed out previously, one of my biggest gripes is that it is not a true .NET only app and therefore will never work in mono and therefore can not be easily ported to the mac - which is my platform of choice for development web/javascript/AS3 development.
Amethyst is also a pretty good option to try. It is a plugin for MS Visual Studio, and takes advantage of a lot of the goodies there. It is significantly less sluggish than FlashBuilder, has a really good debugger, and a decent visual designer as well.
The personal version is free, but quite crippled. You have to buy the pro version after a 60 day free trial. However, it is (at time of writing) almost 1/3 the cost of Flash Builder.
As an added bonus you don't need to pay for Visual Studio since it works with the free (albeit hard to find) "shell version (integrated)" of Visual Studio. It won't work with any of the free Express editions, though.