We are currently thinking of getting our apps automatically built and tested through a CI server.
My major concern is regarding UI tests. I've been playing with FlexMonkium/Selenium IDE in Firefox but it doesn't really play nice with my app. I'm also testing RIATest 3 but I would have liked the Firefox approach better because it's easy to use for non-technical users.
A bigger concern is that my app is really complex, using custom events, custom components, ExternalInterface calls.. in order to make it all work smoothly, should I give an automationName to every single component? How can I handle custom events? RPC responses time?
More generally, starting fresh, what is the best "way" to design your app to be Automation Compliant?
In my company, one Flex developer researched several tools for automating testing UI in Flex. His conclusion was that RIATest has the best support of non-trivial UI. Although, you can have problems even with it, if your UI is very complex.
As an alternative approach, you can reduce number of tests for UI, that is usually more expensive, and test on a lower level, that is cheaper. You will not have a headache with maintaining tests when UI changes, and so on.
If you can test behind UI, it will be better. Besides, as I understand, there is no an enough good tool for testing complex Flex UI.
Related
I have used Flex for about a year before deciding that I would rather develop Actionscript projects.
At the time, it seemed that the framework was too heavy for the kind of work I was dealing with, mainly small web applications , personal sites, portfolios this sort of thing. I also thought that Flex was like a odd hybrid , something targeting seasoned developers but at the same time , adding some function wizards that seemed to target beginners. It seemed overly complexed in some areas and way too basic in others.
On the other hand, the IDE was great , definitely no comparison with Flash CS IDE , so for me it made sense to stick to AS3 projects and use Flex , now FlashBuilder to write my code.
( I need to point out that I'm not a Flash designer, so working with Flash CS was never an option. )
It's been a while since I had a look at the Flex framework and I'm wondering about other Flex/Flash developers position on this issue.
Would you only consider Flex for enterprise level projects? What are the advantages of using one over the other? If you were a Flash developer and moved to Flex, what were your motivations? If you're creating both Flex & Actionscript projects , what are your choice criteria?
Edit:
Although I have received a great answer, I would have been interested to hear from Flex's users, what's your main practical motivation ( as opposed to philosophical :) ) for using Flex over pure Actionscript projects?
Preamble: my experience is primarily with AS3 projects built using a combination of the Flash IDE (FIDE) and Flash Builder 4 (FB4).
I generally prefer pure AS3 (PAS3) projects over Flex projects for the following reasons:
Size - Flex projects have a much larger minimum size than PAS3 projects. Not suitable for lightweight applications.
Performance - Flash is not known for its performance, and the layout computations required by a complex Flex application will hammer the end-user's machine. To them, things just end-up feeling slow, non-responsive, or "gunky". Unfortunately, this means that the applications where Flex might be most attractive (i.e. a very complex, adaptable, UI) are the exact places where it stumbles. In the end, you end up writing all this bizarre performance-enhancing optimization code that takes away most of the time you gained from using the built-in layout system.
Metaphor and Appearance - Flex aims to allow developers to provide end-users with a mature, flexible UI that has the same widgets and widget behaviors that they are used to from native applications. However, due to the performance problems echoed above, the UI never feels quite as nice or responsive as a native app. In addition, it's missing all of the OS-specific peculiarities that end-users are used to and will expect. I don't really understand the motivation to try to emulate native app development or behavior - you're never going to win that fight. Best to make something that stands by itself, which is what most native web applications are doing.
Flexibility - Dovetailing into the previous argument, Flash's main advantage is its ability to do things that traditional UI widget libraries can't do (at least not very easily). You can make some really, really novel UIs in Flash that just aren't practical to do in native apps without mucking about in OpenGL. Using Flex makes creating novel UI hard again (but it does make creating standard UI much easier, even if it is, in my opinion, sub-standard UI).
I'm curious if anyone has some good examples of Flex being used in any popular, public websites. Grooveshark is the only one that I know about (which is quite nice, but suffers from many of the problems I've outlined, especially on OS X where Flash performance is still poor).
However - it's a tradeoff. Always remember that your time is valuable. Your users might accept a slightly clunky, slightly confusing interface if it lets them do really cool things and that would mean that you could release it now as opposed to later. This brings is to the major downsides of PAS3 development:
Effort vs. Reward - You have to program all of your own UI. All of it. This can lead to some really, really bloated code where you have to define tons of event listeners for every button you want to create. I don't know how many times I've written various kinds of layout code specific to what I was working on. You can try to write your own abstract classes for these (which I have done), but at some point you're just going to end up re-implementing the Flex framework. Hardly worth your time.
Development - You can either use just Flash Builder 4, in which case you have to construct every graphical asset by hand in code (which takes forever), or you use the FIDE, in which case you can make lovely artwork but you're stuck with a stone-age code IDE and it takes forrrrrreeeeeeevvvvver to compile anything. Currently I use a hybrid setup where art generated in the FIDE is automatically imported into my FB4 project, but even that is not a perfect solution. They really need to be integrated better.
Another set of things to keep in mind: things that Flash sucks at.
Flash sucks at text. Do not try to re-implement a web browser inside Flash. Flash is actually quite good at displayed relatively small amounts of text that is unselectable (and, through the use of embedded fonts, is always pixel-perfect), but don't try to create large, expansive text documents inside your Flash project. First, performance will be terrible, and second, users will expect the text to behave the same way all other large text fields do in their native applications (most specifically, their web browser). Selecting text in Flash doesn't feel right because it doesn't feel how your OS does it.
Flash doesn't play nice with mouse and keyboard input - it constantly fights with the enclosing browser for focus. If your system needs either of those things, users need to click on it first. Don't fall into a trap where people will get confused because their inputs are going to the wrong place.
Flash is a performance hog - we've all heard this one, and it's not nearly as much of a problem as people like to think, but it does mean that you'll have to put a lot of thought into the performance of your system. Your UI should run at a stable 60FPS when being used and should not use much if any CPU when the user is not interacting with it. If your FPS dips below 60, then your UI will feel slow and gunky compared to native or HTML5 UI. Also make sure to watch for memory leaks.
In the end: user your head. Both approaches are just tools in your arsenal.
I am looking for recommendations for tools for automated testing of a web application with some flex components.
To provide some background we have a web application that was entirely developed in AJAX+HTML and we were somewhat successful in using Selenium for testing that application end to end. We recently added some flex components into the mix and it got complicated.
We tried using Selenium Flex but we are disappointed with what it can do. So now we are looking for some alternatives. Ideally the tool would be able to drive both the web and the flex parts simultaneously, but we can also settle for just testing the flex components on their own. We prefer open source but good commercial tool is also an option.
I've heard about Fluint and FlexMonkey but haven't tried them yet. Anybody has any experience with using those? Any advice for someone who is just starting? Any gotchas?
Are there some other tools that could be used?
Thanks.
i'v try FlexMonkey.
It was great, until he get lost in the recording of interaction with huge and complex Flex Component.
For me, and for now, it's great in the demo or with small app, but not ready for production.
BUT : seems to evolve quickly.
I've had much success using FunFX to automate a fairly complex Flex application. FunFX is built off FireWatir, which is another web automation framework similar to Selenium. So together that should cover the entire AJAX/HTML + Flex bits of your application.
Since both FunFX and FireWatir are Ruby-based, I also recommend something like Cucumber as your testing framework to glue everything together.
We've had some success using Fluint for unit testing and Quick Test Professional (with the Flex plug-in) for functional end-to-end testing. QTP is pretty heavy-weight, but once you've got it set up and have created a few tests, it works pretty well. It works with Flex as well as vanilla HTML/Javascript, so even if your app is a mix of the two, it should be able to handle it.
Fluint works great for unit testing, but given the amount of code required to test a component, I wouldn't advocate it for functional testing (it's too low-level). FlexMonkey is more of an automation framework for Flex Unit. You basically use it to record test cases, and it generates code using Flex Unit to perform the test. I've tinkered with it, but ultimately decided to use QTP.
As for tips to using Fluint: Learn how to use Sequences if you want to test a Flex component - it will make your life considerably easier.
RIATest. This thing rocks. It blew away Flex Monkey (at least in my evaluation)
Our app is fairly large and complicated with custom components and all. RIATest could recognize and manipulate almost everything. Others needed small tweaks since they were custom composite components.
RIA Test has the right balance between features, price and ease of use.
Down side. Paid commercial product.
Has anyone had any luck testing a Flex app without static Automation IDs attached to components? All of the elements in the apps are generated .....
We've investigated FlexMonkey but it appears to be incompatible with any app that utilizes the ExternalInterface. RIATest's scripting language leaves much to be desired...
Thanks-
Jonathan
Unfortunately I don't know much about this kind of stuff, but I went to a talk that presented these tools for TDD:
- Hudson
- Flex Unit 4
I guess there are tutorials online, don't know if it helps with ExternalInterface testing.
Is there anything that prevents you from generating appropriate automationNames for your generated components? This way you should be able to refer to them properly in automation tools.
Are there any other non-changing properties that your generated components have, like maybe 'id'? If so you can use these properties to address the components. This is definitely possible in RIATest.
FunFX is a Flex automation tool that allows you to access components via ID, "automation name", "automation value" or index. While using something like the component index might be less than ideal for robust tests, if that's all that's stable, it might be worth a try. And it's written in Ruby, so that should satisfy any "real programming language" related requirements. :)
We added an "automationPrefix" property to many of our custom controls (particularly those that are reused many times on a single screen), and wrote code to append the beginning of the automationName property on any child controls. Setting the automationName was the most important parting of enabling automation testing on our Flex apps. There are several ways you could modify the automationName to be unique without making it completely static at the level that most test automation packages need it. We are currently using QTP as the test automation tool of choice.
From what I understand a framework reduces complexity in areas that are common, like a log-in system. I use ASP.NET MVC at work and have done some work in Zend Framework but do not get how a framework helps with client side development. The reason ASP.NET MVC is used at work is for Unit Testing - does a Flex framework help with this too?
Please let me know why I should or should not use a framework with Flex?
The short answer is: it depends on the framework. :) My thoughts below:
Flex is itself a framework, and you can write reasonable applications without the need for any additional framework. Flash has a built-in event model that allows for bubbling events, so you can dispatch an event in a deeply-nested user interface component and have a listener higher up in the hierarchy that handles the event. The event handler can delegate to your model, which retrieves data from the server, and Flex's binding support can ensure that your views are appropriately updated from the model. I think it's important to understand that Flex applications can and should be written more or less according to that approach, and that any additional framework should help facilitate that approach, rather than providing its own way of doing things that ends up coupling you to the framework.
That being said, an extra framework that helps facilitate this approach can absolutely provide value. I would recommend Mate or Swiz because I think they achieve this goal. They don't try to reinvent the wheel or replace parts of the Flash / Flex APIs; instead they complement them. The dependency injection features make it much easier to provide data to your views, but without coupling them to any framework. There are a number of utilities available to make it easier to work with remote services. They also have a utils to facilitate testing or even persisting data in shared objects.
I have also worked with Cairngorm in the past and I would not recommend it. CG is notorious for requiring you to create a ton of classes that adhere to CG-specific APIs and requiring you to use many of their Singleton implementations which make your app brittle and hard to test in isolation. It is based on a number of J2EE patterns which fell out of favor in the Java community at least 5 years ago.
I've read a little about PureMVC and while I can't speak to its invasive nature, I think that reinventing the event model (called "notifications") is silly and couples you to their framework. Sure, you can say it "insulates" you from the Flash event model in case it changes, but I'd say the odds of PureMVC changing their notification model is far more likely than Adobe changing the event model. :)
If you've ever tried building a slightly large application, or one at all complex, things can quickly get out of control. I don't know how many projects I bailed on when I was first starting out because I didn't know patterns, or how to make parts of the system communicate without being tied to each other, or dependent on each other.
So, basically a framework is a collection of patterns put together. Theoretically, if you learn to follow the "rules" of a (tried and true) framework, your app will not get out of control to the point where you find yourself fixing one bug and causing two. I've been there and it's not fun.
I also found that by learning to use a framework you initially don't have to know so much about the patterns behind what you're doing. But before long you'll get a good handle on the patterns used and you'll be able to apply them in new situations or find a better pattern. So it's a good learning tool as well.
I'm sure people will have arguments against using frameworks -- this is just my experience. But if you become familiar with a few, you will likely find that one might be suitable for one project but not another.
As far as Flex frameworks, I personally like PureMVC. In all honesty the only other one I've given a good amount of time to is Cairngorm. But I like PureMVC because it feels right to me, but also it's generally not dependent on so much on the built-in Actionscript classes. For example, it uses it's own notification system. So if the notifications change in Flex, they'll still work in your PureMVC app. Also, the creator Cliff is super helpful in his forums, and he's really passionate about it. And the documentation is great.
I recommend coming up with a super basic app, and building it without any frameworks, then again with a couple others. You don't have to finish the app, but just get a feel for what's behind the frameworks.
You may find value in using a framework if:
You work in a team environment and want to be sure everyone uses the same approach
You have a large, sophisticated app and want to communicate effectively between parts of the app
You want to be able to hire additional programmers and have them be productive quickly
You want to ensure your application is built with a proven architecture
You want to leverage the knowledge of design patterns and solutions that are known to work well
You want to simplify a complex application
Here's a great article on Flex frameworks.
Flex Framework Comparison
And, I agree with the conclusion...Mate is a great Flex Framework.
Another interesting framework not mentioned in this article is Spicefactory's Parsley.
I use PureMVC (which is an MVC-framework for Flex) for my larger applications. Like 99miles stated, it depends on the application size.
If you just have to make a (relatively) small tool/application, you can get it going with just Flex. But if you want to create a large, complex application a Framework might come in handy.
The main reason I use PureMVC in Flex is because of the Model-View-Controller architecture. This separates the presentation layer from the business layer and works a lot with events and notifications. This gives you a lot of control in creating reusable classes/controls in Flex in a relatively simple way.
As I said before, for a small tool/app/website I wouldn't recommend using a framework, because in that case it might be to time consuming and complex. But for the larger applications, frameworks are quite handy.
More information on the MVC architecture can be found here.
Hope this helps :)
I'm in the process of designing a solution for the company I work at (but have just resigned from) which has a very complex application process in it. The process is close to 10 steps (including T&C's and preview) and also has some very tricky UI-level business rules (mostly driven by a legacy tie-in system).
Essentially the validation is driven by what has been selected on previous forms, and not just the one directly prior.
The solution has already been decided as an ASP.NET application but the more I design the solution the more it's looking like it's going to be very difficult to achieve in a stateless environment. It's going to either end up with query string parameters, hidden variables on the page, viewState or session to pass the information around.
Then I had it suggested that I look at Silverlight, to give it a more stateful environment and make it easier to handle the passing of the parameters around.
I've never done anything with Silverlight, other than watch presentations, so I've got no hands-on experience with it, but from everything I've read I think that it does have potential to solve some of the major problems which I can see coming out of trying to do this with standard ASP.NET.
So how do I go about:
Is Silverlight a viable option considering what I mentioned above?
Pitching Silverlight to a company without any Silverlight skills?
How do you deal with the "but it requires a browser plugin" argument?
From what you're saying your problem is about keeping state between multiple pages/forms. Sessions are good for that (not so much the query params or hidden variables as those can be tampered with, just having to think about that possibility when developing your app will slow you down.) I'd recommend that as simplest all round.
However, you can also put several forms on one page and submit them together, but have it look like different pages using javascript. It wouldn't be too hard to do with any decent javascript library, but I recommend jQuery (Microsoft is adopting, over their own, as the standard for ASP.NET)
Same as the silverlight problem if your company doesn't have many skills in javascript, but at least it doesn't require a plugin.
Don't get me wrong, I love silverlight, I love it so much that when it gets market share similar to flash, I'd drop HTML/css/javascript. It just doesn't seem to fit for you though.
I currently work for a company that builds its primary LOB app in Silverlight.
Silverlight may be a good choice for what you are trying to do, but consider:
Using Silverlight won't reduce the complexity of the application.
If you're already comfortable with ASP.Net, you can deal with complex workflow using existing idioms (Session state, the wizard control, or plain old CLR objects of your own that implement a decision tree/state machine model).
If your problem is really an issue of workflow design, you can integrate ASP.Net apps with something like Windows Workflow Foundation. However, "won't reduce the complexity of the application" probably applies to this, as well; in my limited experience, Windows Workflow Foundation hasn't made building workflows simpler, either.
What you gain by having state all wrapped up in local client objects will most likely be smaller than the cost of learning Silverlight and its idiosyncrasies.
That being said, there's nothing wrong with the idea of attacking this kind of problem in Silverlight. It's just not a silver bullet or anything; Silverlight has its own data access headaches, quirky UI code idiosyncrasies, and requires wrapping your head around yet another model for developing applications.
Silverlight has a Page class that enables you to easily support navigation where you can go back to the previous page as you do in your browser. I think Silverlight pages and the navigation framework is very suitable for the kind of application you have described.
Using Silverlight enables you to create a very rich client experience. Doing that is probably also possible in ASP.NET to some extent, but I find all the moving parts in a traditional web application (HTML, CSS, JavaScript etc.) and their interactions much more complex than the Silverlight model. If you don't have experience using Silverlight you could create a simple page navigation prototype without any real logic to get a feel for the experience. If you are satisfied you could then show it to management to demonstrate the power of Silverlight. If they like eye candy you could change the visual theme of your application in an attempt to impress them. This tactic may backfire, though, if management actually prefers battleship grey.