Appium or Xamarin UI test - experiences and drawbacks - automated-tests

Not sure if this is the right place to ask this kind of questions because it is also a bit personal favor but what are your experiences with Xamarin UITest and/or Appium. Until now I only had experience with Appium but for our new project we maybe want to use Xamarin UITest.

Wrong answer, with Xamari UI Test framewor you can automate Progressive Mobile Apps, Hybrid apps a Native Apps, on Android and IOS
Some samples here:
[https://www.zuehlke.com/blog/en/mobile-ui-testing/][1]

We have a React Native app that we are currently testing with Xamarin UI Test, and I've spent quite a while comparing these test frameworks. It looks like our team is going to switch to Appium now. Here is what I like about Appium:
Much better documentation, more resources online
More features (which allows you to automate more stuff)
Runs slightly faster on Android (30% faster maybe), not seeing almost any difference with iOS
Bigger, more active community, less chance of framework becoming deprecated
Based on our experience it's more reliable. With Xamarin we kept getting "device port 27753 in use" or smth like that which we couldn't resolve.
More "device farm" options to run tests on. This is very important for us. With Xamarin the only options I've found are BitBar (flakey, adds about 5 minutes overhead to test runs) and Xamarin Test Cloud (expensive). With Appium there are more options (SauceLabs seems to be working great, we are only starting to use it though)
Allows you to choose from a lot of programming languages (our team is most comfortable with TypeScript)
Client-server architecture which allows test code to run on-premises (might not be a benefit to many teams, but it is to us), so the test code is able to access our internal APIs thanks to this

Firstly, SO is usually not the right place for such questions as such questions usually get closed as off-topic and primarily opinion based. Luckily, xamarin.uitest is a very unpopular tag, hence, your question survived :)
For what Xamarin.UITest does, its actually not bad. The biggest issue to consider is that Xamarin.UITest requires your app to be written using Xamarin.Forms and for me, personally, this is the biggest drawback.
Other issues, that I've encountered:
Scrolling is terribly slow
Limited API (Only basic UI properties can be evaluated, see here)
Minimal differences betqeen Android and iOS which can become really annoying, depending on your use case
Other than that, Xamarin.UITest is a pretty solid UI-testframework and if your app is written using Xamarin.Forms, I would recommend you to use Xamarin.UITest.

Related

Project structure/methodology fo avoid/remedy xamarin forms.slow build times

Ive started my first a xamarin forms project. I have given up all simulators a while ago because of slow ...everything
The app performance itself is another thing, but Im talking about slow build and deploy times. This, in combination with that the app just crashes or gets a timeout when fetching the actual error is just gut-wrenching and I spend half my days waiting... ´nuff ranting right?
Are there specific project structure, for example so you can do most of your code through unit test for example? sadly the most trial and error I do has to do with gui-stuff.
I have trued Gorilla player, but it was harder that I thought to get it running, I have an open case at Gorilla-team..
Any tips to avoiding build->deploy to device is veeeee...extremely welcome.
Since I don't know your hardware setup I am going to propose different solutions. Maybe one of them might help.
Pick the platform that runs the fastest. When developing on a Windows machine, then probably the UWP project builds the fastest (since you cut out the Mac in the daisy build chain).
When using an Android simulator make sure to fully use hardware acceleration Making the Android emulator run faster
Visual studio for Mac beats its Windows counterpart hands down when it come to build times in my experience. I gave up on developing XF Apps on Windows since I was having countless build issues and some builds took forever. The situation improved dramatically when moving to a Mac.
Maybe you could also try using the Live Player which should give you immediate feedback when it comes to you UI https://learn.microsoft.com/en-us/xamarin/tools/live-player/
Adding unit tests might also speed up things considerably. NUnit is a pretty popular framework.
Another answer mentions that building to UWP is fastest, but I don't target UWP ever so I wouldn't know about that. I just do iOS and Android. Android is faster. We also have the question of simulator/emulator vs real devices. Of the full gamut of options out there to run the app, I've found that building to a physical Android device is the fastest way to build/run.
You could also check the project properties to make sure you have all optimizations set up for that, such as (for Android in particular):
Use Shared Runtime
Use Fast Deployment
Linking: None
DISABLE ProGuard
Aside from building all the way to a phone every time to test things, setting up your project for unit testing that incorporates mocks can also get you going faster. That would typically involve dependency injection and a unit testing framework. I have a Xamarin.Forms app that does just that on Github if you want to see how to do that.

Xamarin.Forms vs Xamarin Mvvmcross

We are planning to go with Xamarin for our next big project.
Seems like Xamarin.Forms is mature enough to use with production projects.
But I still want to take input from you experts - should we go with Xamarin.Forms or Xamarin with MvvmCross architecture?
The project is big and critical for our customer.
We experimented with Xamarin.Forms about 18 months ago, so this experience may be dated, but we found Xamarin.Forms to be unsuitable for production projects. Granted, it is very quick to get basic data capture apps running on multiple platforms, but we found that inevitably the UI capabilities were so limited that we ended up having to write custom renderers all over the place, which complicated the code no end.
In my opinion, Xamarin.Forms tries to solve the multiple platforms problem in the wrong way - by trying to provide common wrappers around the UI elements in each platform. This means you'll always be able to do less with Xamarin.Forms than you would be able to do with Xamarin out of the box, and you will always be fighting with lowest-common-denominator implementations of the most common elements, while having to write your own code for more advanced UI.
By contrast, the MvvmCross approach aims to consolidate as much as possible of your business logic into a single library while leaving you free to do whatever you want in the UI of each platform. You can get as much as 80-90% of your code into a shared library while having complete freedom to implement the UI you want for each platform. It's a cleaner way to solve the multi-platform problem, IMO.
Xamarin.Forms is a bit of a different technology than MvvmCross. Everything depends on the size of project, how complicated the UI is. I recommend reading this article - it should give you some ideas.

Enterprise Framework - UWP Vs. Web

Broad, sweeping question here...
Assume you have built an enterprise level framework with some rich client in the .Net (Microsoft) sphere, with a WCF back end. Now, imagine that that enterprise framework's UI technology is being deprecated in favour of UWP.
The choices for a front end replacement basically are: UWP, Web (HTML), or some other rich client technology.
How would you go about the decision making process?
I personally lean toward a rich client where the user base is a captive user base. I mean, where the users' IT department is happy to install the necessary runtime environment on the machines, etc. This is usually not a problem with Microsoft technologies, and this won't be a problem in about 10 years when organizations roll out Windows 10.
But, people are telling me these days that web has come a long way. People are telling me that JavaScript frameworks are becoming very sophisticated, and that low level JavaScript for basic data binding and the like is mostly unnecessary.
I have really been turned off by web solutions like ASP in the past, but I do understand that technology has moved forward, and I do understand that Microsoft have been working on ASP.Net v Next which might actually be good?
The question is not so much what would you opt for? But, what factors would you take in to account to decide which platform to go for?
Opnion based answer here...
In a decision to adopt a particular tech for any project lies in many factors. I can cite two majors for your particular scenario.
1 - Client adoption. It's easy for the customers to use/install it? They need to pay some sort of license? Can it run in all platforms/devices the customer already own?
2 - Market adoption. It's easy for you co-workers to adopt it? It's hard to find/hire experienced/hardened developers? We need to pay some kind of license? Can I trust it ill be a long lived technology?
The answer to your question can be HTML.
Not only it is already got a lot of momentum in market, it ill take years to change it even if today someone (big like MS or Google) put some new (better) stuff on the table.
Also if someone on MS marketing dep say next week Universal Windows Platform or WinRT must die it ill die (like Silverlight). So Im not adopting some new technology just because some big player told me to do it.
Yes web has come a long way indeed. You can do a lot of amazing things just with JS+HTML+CSS those days. Also the right usage/architecture of it ill allow you to put your app running in PCs, Tablets and Mobiles (at a minimum cost to port between devices) and capable in running in anything can access internet.
I suggest you to catch up and learn a lot about webservices, Json, JS libraries like JQuery, Sammy and some nice stuff like Knockout, SPA, Angular, Node, etc.
Edit, answer to comments
To not start a chatty comment I'll respond here. Yes your questions and comments brings interesting questions. To let it readable for posterity both of us can edit answer and question to organize it.
Silverlight. How not love it? In special after strugling with flash. It's a shame MS pulled the plug (die in hell MS CEOs). When MS let it to die I was planning a big web app SL was my first choice. Why I changed my mind? Well 2 years to develop that app and at the end how much browser ill get along supporting it? The SL community is great, the tool is great but browsers can just say, Hey tomorrow there's no guarantee it keep working.
.Net and MS platforms. I'm a .Net developer. I adopted it since beta, first to work with winforms (in a previous life I was a proud Delphi developer). After a while started to work with web. I also worked in classical ASP (bad times) and loved .Net ASP from start.
You can run .Net apps in almost any PC in the planet today. Not exactly true for all mobiles/gadgets. For browsers pure HTML+JS+CSS ill to better because it's lightweight (done right). Also we can move a lot of thing to client side and just let it hit the server only when necessary. .Net apps can do that, sure, but ill never be light as a tailored HTML+JS+CSS.
In fact I believe you can do anything with .Net and you can do amazing things if you got a few good developers in your team. But depending on the project it ill do better (and cheaper) in HTML or PHP or Ruby or Java, etc.
In fact at a previous shop, with both PHP and .Net teams we found (after 1 year study, metrics, lots of projects) small projects are better done in PHP, larger ones in .Net (if I remember a medium project can be 4k to 6k men/hours).
The point here is. You really must read a lot about HTML, CSS, JS, SPA, Angular, etc. Bringing to live a big and shinning web app is challenging today not because what we can do (we can do anything) but how we can do. DDD, MVC, MVVM. Testing framework, etc. Man Node is the future (the concept at least).
Web developing really changed in the last years and with it the clients and users expectations. Today no one ill wait for more than 2 secs for a page to load. Everybody wants usability to be at the top of the table from project scratch. You app must be responsive, etc. (not using Dilbertian management buzz words here for the sake of it. Just stating usability is that important today).
And don't forget everyone wants it to be beatifull (from a graphical designer point of view) even if it's a dull B2B supposed to be used only by cave mens.
Even if you stick to a classical .Net app learn about the (many) options, that can bring a new wider perspective.
I have decided to answer this question here because we've had a lot more time to investigate and look at different options. The original question turned out to be a bit of a furphy. Pure UWP and Web are not the only options. There is also a Xamarin Forms as an option which includes UWP, Android, and iOS. As a personal preference, I am leaning toward using Xamarin Forms as a client instead of any other development platform because it supports three OSs out of the box: Windows 10, iOS, and Android.
I believe the answer to the question is: you should only develop a web app if you need to. Does your user base consist of people who will mostly prefer a browser over apps? Are your potential users likely to want to avoid downloading an app? Is your app very simple, and you want people to be able to dive in very quickly? Are you able to get away without access to things like the camera, location and push notifications? If you can answer yes to these things, then I think you should go for HTML 5/JavaScript. If however, your user base is comfortable downloading apps, and you think that your app will require a UI more sophisticated than most browser apps, I'd recommend looking at Xamarin Forms as the preferred option. We've had very good success with Xamarin Forms so far, and the UWP version of our Xamarin Forms app has turned out just as good as our first stab at a UWP app.
Note: I should give Web Assembly (http://webassembly.org/) an honorable mention here. This technology is being considered in all the big tech organisations like Microsoft, Apple, and Google. One day, it may make deployment of native apps in a browser great again.

TideSDK and native code?

I've just discovered TideSDK and it seems to be a really great tool, but I have one requirement : I need to use some native code (for managing USB devices for example) and so I need communication between this native code and the web app, is such a thing possible with TideSDK?
Yes, working with native code in TideSDK is possible. Our SDK is modular and we have been reorganizing the code structurally to make it easier to do the sort of thing you want. At a modular level, you will be contending with support for multiple platforms typically.
A module should extend to all platforms that you are supporting. We expect to have documentation to help developers (familiar with native code) to better understand the SDK. This should include some module boilerplate to help you get started. At this point, we have yet to prepare this more detailed documentation. We have much to do and sometimes progress seems slow despite all the great efforts going into TideSDK.
TideSDK is a large and complex SDK but don't let this frighten you off. It is extensible and we will be shining light on this soon with module development guides. It would be cool to talk more on IRC about this with you so feel free to drop by at any time. Perhaps the functionality you are speaking of is of general use ie. to extend the APIs for everyone.
There are possibilities to work together with the core developers of TideSDK on modules and to contribute to this great open source project. Other possibilities also include sponsoring module development if this something that you need more immediately for a project. Hope this helps.

Is Flex ready for prime time?

I'm working on a project that currently has zero users but we would like to scale up to potentially hundreds. Currently we are running on a MySQL database with AMFPHP interacting with Flex. We used Flex because of its robust graphic features (important to this project) and because the initial developer (not me) already knew ActionScript. We are currently using AIR but might switch to web-based Flash at some point.
My questions are:
Is Flex a good tool for a project like this?
What are the main limitations of Flex that we might encounter?
What are other development platforms we might want to consider?
Thanks.
- Dave
Short answer, Yes. There are already many prime-time Apps using Flex as their UI development platform. If you go to the Adobe site they showcase quite a few.
Speaking personally, I chose Flex for two reasons, first was that, although you probably can do much of what Flex does in HTML or with an appropriate toolkit, Flex is designed for attractive and compelling user experience and has available all of Flash. Plus the development environment and available widgets make it easy and fun to program. I don't want to spark a religious war about HTML vs. Flex, so I'll leave that there - it works for me and my application and customers.
Second, and more important, was that it balances the processing load more towards the client which means my server architecture can be optimised just for serving the content and persisting the data. Most of my business logic has migrated across to the client. Having spent many years in classical architectures I think this is a huge step forward, but I can already her a chorus of disagreement about that too.
My word of caution about Flex comes from needing to adopt the right architecture for your client code. It is pretty easy to create a huge and badly performing app with Flex if you get that wrong. Make everything event driven and apparently asynchronous and you should be OK ('apparently' because the Flash player is single threaded). And that is downside 1, the single threaded Flash player sometimes causes issues.
Downside 2 is perhaps more serious and that is locked down desktops in corporate environments. Quite often your target audience won't have administrative rights to their computer and will have either the wrong flash player or none at all. This is particularly true in public sector organisations and the military, so if you are heading there I would test carefully the presence of Flash amongst your users.
Other than that I heartily recommend Flex. It's also a great thing to have on your CV!
HTH
Flex has no inherent scalability problems, however if you have a graphic intensive application, proper serving of these resources might be a problem, but that has little to do with Flex.
The only note-worthy and likely platform you won't be able to run on is the iPhone (no flash) and some older non-flash mobile devices (although most support Flash-lite nowadays)
As for alternatives, if you are Graphics heavy, and don't mind the iPhone, then Flex is good if not best cross platform solution besides using pure HTML technologies, the trick here is HTML alone can do 99% of what Flex can do, but if your App requires the missing 1% then you're out of Luck, also Flex will reduce crossplatform and most browser compatibility issues. So it might make your work more productive.
Silverlight 2 is an alternative to consider. WPF if you're looking for something with offline support.
Yes, the scale and type of project
fits.
Immaturity of frameworks and libraries you might depend on. Immaturity of IDE's.
Silverlight, JavaFX.
Flex + AIR is probably as good a tool as Visual Basic was; it may be a better tool for having a much more flexible programming language and having free development tools, but keep the limitations in mind....
The main limitation I've seen from working with it is documentation. There seems to be not enough documentation, not good enough documentation, and not enough high-visibility work on it in the community. (This is coming from years in .NET; I've been constantly upset with how little MSDN says about methods but generally able to deal with it by finding the most useful blog posts.)
Other possible development platforms would depend very heavily on the specifics of the project. Web-based platforms bog down in deep, stateful interactions with data sets (even with nice AJAX libraries), whereas maintaining client-side installations of any thick client program (say, Flex + AIR) might be overkill if it's just a few CRUD forms.

Resources