We have a flex based UI whose functionality (eg: login, logout) needs to be tested periodically in IE 7. Manual testing takes a lot of time and hence we want to automate it.
EDIT: We dont have the source code of this app uder test so cannot use something like FlexUnit. Is there a way to automate this testing?
I have heard of FleXmonkey, but have read negative reviews of it and hence wary of trying it.
EDIT: source of negative review:
Automated testing of FLEX based applications
(Comment by Ryan H)
Please suggest if theres a way to go about it.
EDIT: An open-source solution is preferred.
Thank you.
You won't get very far in your development adventures if you avoid reviewing things just because you read negative reviews. ( Please provide a source for said negative reviews ). I've also read bad things about Flex and Flash Builder and the Flash Player. Yet, you still ended up using Flex?
That said, I strongly suggest checking out Flex Monkey.
RIATest is another option.
I believe you can also do this sort of testing with QTP. I couldn't find a specific source for QTP info, but there are a few links if you google.
http://www.learnqtp.com/does-quicktest-professional-support-flashadobe-flex-applications/
http://vishnuagrawal.blogspot.com/2009/04/flex-automation-testing-with-qtp.html
Without recompiling the source code, it is not possible to test Flex in all cases. Saucelabs IDE claims that it can test Flex without recompiling the code. You may check that.
I think FlexMonkey is pretty cool and worth judging on your own. Most of these tools use the same underlying automation framework provided by Adobe, and thus have many of the same pains and challenges. We've actually done a lot the last few months to make it easier to work with and have been working to provide more documentation on dealing with common challenges.
If you do have problem, you will find that our Forums are very active with questions and answers: http://www.gorillalogic.com/forum/7
-Jon Rose -
Gorilla Logic, Inc
Some Flex automation tools support testing of Flex applications without requiring to recompile it.
RIATest definitely supports it (called runtime loading) and if I am not wrong QTP does it as well. AFAIK there are no open source tools that support it.
Beware though that runtime loading is not for every application. If for example you are using custom html wrapper you cannot use it with runtime loading.
Related
I have been asked to check how to load test a flex application.
I've though about using Selenium, since it's precisely its job. I've also found that testing Flex with Selenium requires extensions to Selenium. I've seen that there some of these :
sfapi
flex-ui-selenium
First, it seems that both requires me to rebuild my Flex application to test it. is it true ? Isn't there any way to automatically add handles ?
Second, when doing so, will I be able to do all kind of tests ? Load tests ? Performance tests ? All these in a continous way ?
Thanks
Yes there is one another way, but that is too complex to care or to implement. It's called OCR or http://en.wikipedia.org/wiki/Optical_character_recognition
Also you could use image recognition.
Alas, of course you can't automate a flash or a flex you need to rebuild it to use ExternalInterfaces to make some methods publicly available. That is how it is. The flash is one big chunk of an object there are no tiny parts that you could catch with anything else.
So you have these options only, nothing else i'm afraid.
Adobe Genie can be used for testing flex apps without rebuilding the flex application, and its free.
However, only Actionscript 3 is supported. AIR, AS1 & AS2 is not supported. Currently MX and Spark components cannot be tested, the doc says it can be done, but doesn't say how.
Another option is Sikuli, this is a image based testing tool. Its strength is its image heuristics, and does not use coordinates for comparison, unlike most image based testing tools.
No you don't need to rebuild your application. There is an option to do this, but there are also options not too. The manuals clearly describe both approaches.
Check out also:
RiaTest
Flex Monkey
Neo4J
I am looking for recommendations for tools for automated testing of a web application with some flex components.
To provide some background we have a web application that was entirely developed in AJAX+HTML and we were somewhat successful in using Selenium for testing that application end to end. We recently added some flex components into the mix and it got complicated.
We tried using Selenium Flex but we are disappointed with what it can do. So now we are looking for some alternatives. Ideally the tool would be able to drive both the web and the flex parts simultaneously, but we can also settle for just testing the flex components on their own. We prefer open source but good commercial tool is also an option.
I've heard about Fluint and FlexMonkey but haven't tried them yet. Anybody has any experience with using those? Any advice for someone who is just starting? Any gotchas?
Are there some other tools that could be used?
Thanks.
i'v try FlexMonkey.
It was great, until he get lost in the recording of interaction with huge and complex Flex Component.
For me, and for now, it's great in the demo or with small app, but not ready for production.
BUT : seems to evolve quickly.
I've had much success using FunFX to automate a fairly complex Flex application. FunFX is built off FireWatir, which is another web automation framework similar to Selenium. So together that should cover the entire AJAX/HTML + Flex bits of your application.
Since both FunFX and FireWatir are Ruby-based, I also recommend something like Cucumber as your testing framework to glue everything together.
We've had some success using Fluint for unit testing and Quick Test Professional (with the Flex plug-in) for functional end-to-end testing. QTP is pretty heavy-weight, but once you've got it set up and have created a few tests, it works pretty well. It works with Flex as well as vanilla HTML/Javascript, so even if your app is a mix of the two, it should be able to handle it.
Fluint works great for unit testing, but given the amount of code required to test a component, I wouldn't advocate it for functional testing (it's too low-level). FlexMonkey is more of an automation framework for Flex Unit. You basically use it to record test cases, and it generates code using Flex Unit to perform the test. I've tinkered with it, but ultimately decided to use QTP.
As for tips to using Fluint: Learn how to use Sequences if you want to test a Flex component - it will make your life considerably easier.
RIATest. This thing rocks. It blew away Flex Monkey (at least in my evaluation)
Our app is fairly large and complicated with custom components and all. RIATest could recognize and manipulate almost everything. Others needed small tweaks since they were custom composite components.
RIA Test has the right balance between features, price and ease of use.
Down side. Paid commercial product.
Has anyone had any luck testing a Flex app without static Automation IDs attached to components? All of the elements in the apps are generated .....
We've investigated FlexMonkey but it appears to be incompatible with any app that utilizes the ExternalInterface. RIATest's scripting language leaves much to be desired...
Thanks-
Jonathan
Unfortunately I don't know much about this kind of stuff, but I went to a talk that presented these tools for TDD:
- Hudson
- Flex Unit 4
I guess there are tutorials online, don't know if it helps with ExternalInterface testing.
Is there anything that prevents you from generating appropriate automationNames for your generated components? This way you should be able to refer to them properly in automation tools.
Are there any other non-changing properties that your generated components have, like maybe 'id'? If so you can use these properties to address the components. This is definitely possible in RIATest.
FunFX is a Flex automation tool that allows you to access components via ID, "automation name", "automation value" or index. While using something like the component index might be less than ideal for robust tests, if that's all that's stable, it might be worth a try. And it's written in Ruby, so that should satisfy any "real programming language" related requirements. :)
We added an "automationPrefix" property to many of our custom controls (particularly those that are reused many times on a single screen), and wrote code to append the beginning of the automationName property on any child controls. Setting the automationName was the most important parting of enabling automation testing on our Flex apps. There are several ways you could modify the automationName to be unique without making it completely static at the level that most test automation packages need it. We are currently using QTP as the test automation tool of choice.
I've been working on a project called Axial that converts MSIL (compiled C# or VB.NET) to JavaScript. There are a few samples of working code, but some common situations don't work properly. (The current release doesn't work in production mode and the SVN code doesn't work in debug mode but is much cleaner.) I've heard from quite a few people that they hate writing JavaScript, so I know the project has some merit, but I'd guess that attitude is less prevalent among the SO community.
Assume the product works perfectly and smoothly, so your JavaScript works 100% of the time and a Visual Studio plug-in makes sure you're using the product correctly. Is this something you or your organization would use? What features other than straight code translation would interest you?
That sounds very much like a .NET-centric version of GWT. I guess the questions are:
Have I understood the purpose correctly?
Do you believe you can overcome any roadblocks that GWT users might stumble over?
Is there enough benefit in having a .NET version as well as GWT to make the duplication of effort worthwhile? (I'd personally just write the Java code and use GWT, but I know not all .NET developers know Java, and you may already have common library stuff you want to port.)
Do you have any useful ideas you could contribute to GWT? :)
(Disclaimer: I work for Google, but haven't used GWT myself.)
Been done (pretty much).
Some requirements:
Step-by-step debugging & Break Points
View variable values while debugging
IntelliSense (auto-completion). Ideally done with "smarts" so it only shows you the syntactically valid options to complete something. E.g., Textbox. would trigger members of that class (.Text, .BackColor), etc. And then Textbox.BackColor= would trigger a list of valid colors.
Drag'n Drop style GUI design.
I'm not concerned about cost (Ok, over $1K gets a little pricey looking :-)
Step by step debugging with variable view (and often with additional views like stack or global variables etc) is working in some IDEs. FlexBuilder, FDT3 Enterprise (only the enterprise version allows debugging), FlashDevelop (with extra plug-in). You can also try the (free!) Visual Studio plug-in Tofino (Ensemble), so you can debug inside Visual Studio.
But the best implementation for debugging I found in the Eclipse based IDEs. I'm not much impressed with FDT3 at all, so I recommend FlexBuilder for debugging (+profiling) and of course coding of AS and eventually (visual) design for MXML.
You can try FlashDevelop with the FDB plug-in. Many people prefer this free IDE for coding because of the nice code assist (IMHO the best of all known IDEs). Debugging is working with this plug-in but I wouldn't say that is the "most productive". But give it a try, its free!
There are only 2 I know:
Flex Builder
IntelliJ Idea
I was interested in the comments on debugging above. I am one of the developers of the Amethyst Flex/AIR IDE for Visual Studio. We are very keen to have the best Flex debugging available so I would appreciate any feedback to help us achieve this.
The current beta of Amethyst has breakpoints, watch variables, locals, call-stack, step-into/step-out/step-over plus drilldown expansion in the debugging windows or in hovering debug tips in the code editor. There's a small movie showing a few debugging features of Amethyst here:
http://www.sapphiresteel.com/Debugging-Adobe-Flex-Applications
Shortly we will add more features (e.g. tracepoints and conditional breakpoints). But if there are any features we've missed, now would be a good time to tell us ;-)
best wishes
Huw Collingbourne
SapphireSteel Software
I've used them all and I can say without a doubt IntelliJ is the most powerful of them all.
http://www.jetbrains.com/idea/features/flex_ide.html
It also is the best JS IDE, and unless you are using .NET on the back end it is probably the best IDE for your back end language. These guys really have it going on.
FlashDevelop from FlashDevelop.org is another free editor that appears to be gaining momentum. It's windows-only but seems to be pretty feature rich.
Netbeans also has a plugin called FlexBean. But by now isn't powerful.
How about SapphireSteel Amethyst Personal Edition? It's based on VS.
Quite a few prominent Flex+Flash developers swear by FDT
It's an Eclipse plugin, and it's probably the most expensive Flex development tool out there, but it's very good.
Flex Builder. And BONUS you can get it for FREE if you are a teacher or student.
From the discussion above, below is the list of IDEs for Flex with debugging support
Powerflasher FDT
Amethyst
Adobe Flash Builder
IntelliJ IDEA (Ultimate Edition)
Flash Develop