Has anyone had any luck testing a Flex app without static Automation IDs attached to components? All of the elements in the apps are generated .....
We've investigated FlexMonkey but it appears to be incompatible with any app that utilizes the ExternalInterface. RIATest's scripting language leaves much to be desired...
Thanks-
Jonathan
Unfortunately I don't know much about this kind of stuff, but I went to a talk that presented these tools for TDD:
- Hudson
- Flex Unit 4
I guess there are tutorials online, don't know if it helps with ExternalInterface testing.
Is there anything that prevents you from generating appropriate automationNames for your generated components? This way you should be able to refer to them properly in automation tools.
Are there any other non-changing properties that your generated components have, like maybe 'id'? If so you can use these properties to address the components. This is definitely possible in RIATest.
FunFX is a Flex automation tool that allows you to access components via ID, "automation name", "automation value" or index. While using something like the component index might be less than ideal for robust tests, if that's all that's stable, it might be worth a try. And it's written in Ruby, so that should satisfy any "real programming language" related requirements. :)
We added an "automationPrefix" property to many of our custom controls (particularly those that are reused many times on a single screen), and wrote code to append the beginning of the automationName property on any child controls. Setting the automationName was the most important parting of enabling automation testing on our Flex apps. There are several ways you could modify the automationName to be unique without making it completely static at the level that most test automation packages need it. We are currently using QTP as the test automation tool of choice.
Related
I'm building a Bridge game.
The main page, where you play the game, boast some 200 controls, like buttons that represents the 52 cards, 26 others for won / lost tricks, plenty of text fields... You name it, total some 200.
Actually its been already built and runs under Winforms model, but with the unavoidable question of implementing .NET 4.0 onto user's machine that must run Windows, and installing the software.
Hence I'm considering creating an ASP version (no PHP please, since I'm fluent in VB, and ignorant in C, Java, PHP and the likes).
Is that a "real" option or am I dreaming?
Will it run smoothly for users with a 2MB DSL connection?
Oh, by the way, no MVC please, I'm too old for that sort of sh...t :->
Can it be done? Sure.
The problem for you might be that to do it effectivly you will need to use HTML, CSS, Javascript and the like.
What most likely would be a good idea for you here would be to go the route of Silverlight. It will give you a more rich UI experience and will allow you to code in VB entirely.
I don't see using ASP.BET directly as a good option here. To make it look smooth, you would have to use javascript heavily. I thing you would be better off with Flash.
If you prefer to use pure HTML/Javascript/CSS you can create rich Web UI's by modifying the Document's Object Model (DOM). With little knowledge of Javascript and the usage of some library like: scriptaculus, prototype, sarissa, jquery, you can get rich interfaces. You should take a look at some of these libraries mainly scriptaculus because the set of animations it provides. Another javascript library made for game development is gameQuery.
With that many controls on an ASP.NET page your view state will be gigantic. If you plan on only serving up a couple of games at once then it shouldn't be a problem but you have to actually start thinking about bandwidth with that large of a viewstate. You would be better off with a DOM/jQuery type interface which displayed static images for the cards. All interaction can be handled client side and validated with a server side call.
This might be the perfect opportunity to learn javascript :). Remember each language you learn is another tool in your toolbox. The engineer with the most tools usually wins :).
We have a flex based UI whose functionality (eg: login, logout) needs to be tested periodically in IE 7. Manual testing takes a lot of time and hence we want to automate it.
EDIT: We dont have the source code of this app uder test so cannot use something like FlexUnit. Is there a way to automate this testing?
I have heard of FleXmonkey, but have read negative reviews of it and hence wary of trying it.
EDIT: source of negative review:
Automated testing of FLEX based applications
(Comment by Ryan H)
Please suggest if theres a way to go about it.
EDIT: An open-source solution is preferred.
Thank you.
You won't get very far in your development adventures if you avoid reviewing things just because you read negative reviews. ( Please provide a source for said negative reviews ). I've also read bad things about Flex and Flash Builder and the Flash Player. Yet, you still ended up using Flex?
That said, I strongly suggest checking out Flex Monkey.
RIATest is another option.
I believe you can also do this sort of testing with QTP. I couldn't find a specific source for QTP info, but there are a few links if you google.
http://www.learnqtp.com/does-quicktest-professional-support-flashadobe-flex-applications/
http://vishnuagrawal.blogspot.com/2009/04/flex-automation-testing-with-qtp.html
Without recompiling the source code, it is not possible to test Flex in all cases. Saucelabs IDE claims that it can test Flex without recompiling the code. You may check that.
I think FlexMonkey is pretty cool and worth judging on your own. Most of these tools use the same underlying automation framework provided by Adobe, and thus have many of the same pains and challenges. We've actually done a lot the last few months to make it easier to work with and have been working to provide more documentation on dealing with common challenges.
If you do have problem, you will find that our Forums are very active with questions and answers: http://www.gorillalogic.com/forum/7
-Jon Rose -
Gorilla Logic, Inc
Some Flex automation tools support testing of Flex applications without requiring to recompile it.
RIATest definitely supports it (called runtime loading) and if I am not wrong QTP does it as well. AFAIK there are no open source tools that support it.
Beware though that runtime loading is not for every application. If for example you are using custom html wrapper you cannot use it with runtime loading.
I have been asked to check how to load test a flex application.
I've though about using Selenium, since it's precisely its job. I've also found that testing Flex with Selenium requires extensions to Selenium. I've seen that there some of these :
sfapi
flex-ui-selenium
First, it seems that both requires me to rebuild my Flex application to test it. is it true ? Isn't there any way to automatically add handles ?
Second, when doing so, will I be able to do all kind of tests ? Load tests ? Performance tests ? All these in a continous way ?
Thanks
Yes there is one another way, but that is too complex to care or to implement. It's called OCR or http://en.wikipedia.org/wiki/Optical_character_recognition
Also you could use image recognition.
Alas, of course you can't automate a flash or a flex you need to rebuild it to use ExternalInterfaces to make some methods publicly available. That is how it is. The flash is one big chunk of an object there are no tiny parts that you could catch with anything else.
So you have these options only, nothing else i'm afraid.
Adobe Genie can be used for testing flex apps without rebuilding the flex application, and its free.
However, only Actionscript 3 is supported. AIR, AS1 & AS2 is not supported. Currently MX and Spark components cannot be tested, the doc says it can be done, but doesn't say how.
Another option is Sikuli, this is a image based testing tool. Its strength is its image heuristics, and does not use coordinates for comparison, unlike most image based testing tools.
No you don't need to rebuild your application. There is an option to do this, but there are also options not too. The manuals clearly describe both approaches.
Check out also:
RiaTest
Flex Monkey
Neo4J
I am looking for recommendations for tools for automated testing of a web application with some flex components.
To provide some background we have a web application that was entirely developed in AJAX+HTML and we were somewhat successful in using Selenium for testing that application end to end. We recently added some flex components into the mix and it got complicated.
We tried using Selenium Flex but we are disappointed with what it can do. So now we are looking for some alternatives. Ideally the tool would be able to drive both the web and the flex parts simultaneously, but we can also settle for just testing the flex components on their own. We prefer open source but good commercial tool is also an option.
I've heard about Fluint and FlexMonkey but haven't tried them yet. Anybody has any experience with using those? Any advice for someone who is just starting? Any gotchas?
Are there some other tools that could be used?
Thanks.
i'v try FlexMonkey.
It was great, until he get lost in the recording of interaction with huge and complex Flex Component.
For me, and for now, it's great in the demo or with small app, but not ready for production.
BUT : seems to evolve quickly.
I've had much success using FunFX to automate a fairly complex Flex application. FunFX is built off FireWatir, which is another web automation framework similar to Selenium. So together that should cover the entire AJAX/HTML + Flex bits of your application.
Since both FunFX and FireWatir are Ruby-based, I also recommend something like Cucumber as your testing framework to glue everything together.
We've had some success using Fluint for unit testing and Quick Test Professional (with the Flex plug-in) for functional end-to-end testing. QTP is pretty heavy-weight, but once you've got it set up and have created a few tests, it works pretty well. It works with Flex as well as vanilla HTML/Javascript, so even if your app is a mix of the two, it should be able to handle it.
Fluint works great for unit testing, but given the amount of code required to test a component, I wouldn't advocate it for functional testing (it's too low-level). FlexMonkey is more of an automation framework for Flex Unit. You basically use it to record test cases, and it generates code using Flex Unit to perform the test. I've tinkered with it, but ultimately decided to use QTP.
As for tips to using Fluint: Learn how to use Sequences if you want to test a Flex component - it will make your life considerably easier.
RIATest. This thing rocks. It blew away Flex Monkey (at least in my evaluation)
Our app is fairly large and complicated with custom components and all. RIATest could recognize and manipulate almost everything. Others needed small tweaks since they were custom composite components.
RIA Test has the right balance between features, price and ease of use.
Down side. Paid commercial product.
From what I understand a framework reduces complexity in areas that are common, like a log-in system. I use ASP.NET MVC at work and have done some work in Zend Framework but do not get how a framework helps with client side development. The reason ASP.NET MVC is used at work is for Unit Testing - does a Flex framework help with this too?
Please let me know why I should or should not use a framework with Flex?
The short answer is: it depends on the framework. :) My thoughts below:
Flex is itself a framework, and you can write reasonable applications without the need for any additional framework. Flash has a built-in event model that allows for bubbling events, so you can dispatch an event in a deeply-nested user interface component and have a listener higher up in the hierarchy that handles the event. The event handler can delegate to your model, which retrieves data from the server, and Flex's binding support can ensure that your views are appropriately updated from the model. I think it's important to understand that Flex applications can and should be written more or less according to that approach, and that any additional framework should help facilitate that approach, rather than providing its own way of doing things that ends up coupling you to the framework.
That being said, an extra framework that helps facilitate this approach can absolutely provide value. I would recommend Mate or Swiz because I think they achieve this goal. They don't try to reinvent the wheel or replace parts of the Flash / Flex APIs; instead they complement them. The dependency injection features make it much easier to provide data to your views, but without coupling them to any framework. There are a number of utilities available to make it easier to work with remote services. They also have a utils to facilitate testing or even persisting data in shared objects.
I have also worked with Cairngorm in the past and I would not recommend it. CG is notorious for requiring you to create a ton of classes that adhere to CG-specific APIs and requiring you to use many of their Singleton implementations which make your app brittle and hard to test in isolation. It is based on a number of J2EE patterns which fell out of favor in the Java community at least 5 years ago.
I've read a little about PureMVC and while I can't speak to its invasive nature, I think that reinventing the event model (called "notifications") is silly and couples you to their framework. Sure, you can say it "insulates" you from the Flash event model in case it changes, but I'd say the odds of PureMVC changing their notification model is far more likely than Adobe changing the event model. :)
If you've ever tried building a slightly large application, or one at all complex, things can quickly get out of control. I don't know how many projects I bailed on when I was first starting out because I didn't know patterns, or how to make parts of the system communicate without being tied to each other, or dependent on each other.
So, basically a framework is a collection of patterns put together. Theoretically, if you learn to follow the "rules" of a (tried and true) framework, your app will not get out of control to the point where you find yourself fixing one bug and causing two. I've been there and it's not fun.
I also found that by learning to use a framework you initially don't have to know so much about the patterns behind what you're doing. But before long you'll get a good handle on the patterns used and you'll be able to apply them in new situations or find a better pattern. So it's a good learning tool as well.
I'm sure people will have arguments against using frameworks -- this is just my experience. But if you become familiar with a few, you will likely find that one might be suitable for one project but not another.
As far as Flex frameworks, I personally like PureMVC. In all honesty the only other one I've given a good amount of time to is Cairngorm. But I like PureMVC because it feels right to me, but also it's generally not dependent on so much on the built-in Actionscript classes. For example, it uses it's own notification system. So if the notifications change in Flex, they'll still work in your PureMVC app. Also, the creator Cliff is super helpful in his forums, and he's really passionate about it. And the documentation is great.
I recommend coming up with a super basic app, and building it without any frameworks, then again with a couple others. You don't have to finish the app, but just get a feel for what's behind the frameworks.
You may find value in using a framework if:
You work in a team environment and want to be sure everyone uses the same approach
You have a large, sophisticated app and want to communicate effectively between parts of the app
You want to be able to hire additional programmers and have them be productive quickly
You want to ensure your application is built with a proven architecture
You want to leverage the knowledge of design patterns and solutions that are known to work well
You want to simplify a complex application
Here's a great article on Flex frameworks.
Flex Framework Comparison
And, I agree with the conclusion...Mate is a great Flex Framework.
Another interesting framework not mentioned in this article is Spicefactory's Parsley.
I use PureMVC (which is an MVC-framework for Flex) for my larger applications. Like 99miles stated, it depends on the application size.
If you just have to make a (relatively) small tool/application, you can get it going with just Flex. But if you want to create a large, complex application a Framework might come in handy.
The main reason I use PureMVC in Flex is because of the Model-View-Controller architecture. This separates the presentation layer from the business layer and works a lot with events and notifications. This gives you a lot of control in creating reusable classes/controls in Flex in a relatively simple way.
As I said before, for a small tool/app/website I wouldn't recommend using a framework, because in that case it might be to time consuming and complex. But for the larger applications, frameworks are quite handy.
More information on the MVC architecture can be found here.
Hope this helps :)