We are experimenting with test automation and have found that cucumber-jvm/espresso fits the best. Our next issue is finding a testing service that supports the framework. To run the tests we are currently running 'gradlew connectedCheck' at the command line but from what I have read, AWS Device Farm doesn't support cucumber-jvm/espresso. Any service should work, so long as it supports cucumber-jvm and has a variety of physical devices to test on.
I am hoping that someone out here has experience with this and knows where we should look.
Thanks
Is it necessary to use Cucumber along with Espresso in that case? You can always implement an entire test suite without the need of Cucumber. All you need is a good reporting tool to report you the passes and failures. There are plenty of solutions out there -- I would recommend using something like Spoon or Composer to run your tests in parallel on multiple devices or emulators or a combination of both. Both of these tools provide decent reporting. As far as a device farm is considered, AWS will support that.
Related
Ive started my first a xamarin forms project. I have given up all simulators a while ago because of slow ...everything
The app performance itself is another thing, but Im talking about slow build and deploy times. This, in combination with that the app just crashes or gets a timeout when fetching the actual error is just gut-wrenching and I spend half my days waiting... ´nuff ranting right?
Are there specific project structure, for example so you can do most of your code through unit test for example? sadly the most trial and error I do has to do with gui-stuff.
I have trued Gorilla player, but it was harder that I thought to get it running, I have an open case at Gorilla-team..
Any tips to avoiding build->deploy to device is veeeee...extremely welcome.
Since I don't know your hardware setup I am going to propose different solutions. Maybe one of them might help.
Pick the platform that runs the fastest. When developing on a Windows machine, then probably the UWP project builds the fastest (since you cut out the Mac in the daisy build chain).
When using an Android simulator make sure to fully use hardware acceleration Making the Android emulator run faster
Visual studio for Mac beats its Windows counterpart hands down when it come to build times in my experience. I gave up on developing XF Apps on Windows since I was having countless build issues and some builds took forever. The situation improved dramatically when moving to a Mac.
Maybe you could also try using the Live Player which should give you immediate feedback when it comes to you UI https://learn.microsoft.com/en-us/xamarin/tools/live-player/
Adding unit tests might also speed up things considerably. NUnit is a pretty popular framework.
Another answer mentions that building to UWP is fastest, but I don't target UWP ever so I wouldn't know about that. I just do iOS and Android. Android is faster. We also have the question of simulator/emulator vs real devices. Of the full gamut of options out there to run the app, I've found that building to a physical Android device is the fastest way to build/run.
You could also check the project properties to make sure you have all optimizations set up for that, such as (for Android in particular):
Use Shared Runtime
Use Fast Deployment
Linking: None
DISABLE ProGuard
Aside from building all the way to a phone every time to test things, setting up your project for unit testing that incorporates mocks can also get you going faster. That would typically involve dependency injection and a unit testing framework. I have a Xamarin.Forms app that does just that on Github if you want to see how to do that.
I am beginning to study the use of virtual machines with realtime applications, specifically network applications.
While I do understand the limitations and concerns, I'd like to get ideas as to how to get started on this task.
I am going to use a DPDK sample application over Linux, and probably use VMWare for starters. However, I do not know what my first steps with respect to setting up VMWare should be.
First I think it is better to use open source solution like QUEM/KVM for your virtualization platform. Many platform exist for run high performance network functions on virtualized platforms you can see OpenNetVM for example in order to get basic ideas.
I'm a test engineer for a software company and we've recently purchased Ranorex (an automated test application) and I'm looking at connecting it to something like QA Complete (test case management software)
Basically I want to run an automated test in Ranorex and for it to automatically mark a matching test case in QA Complete as passed/failed. I know this will obviously involve some coding at the Ranorex end using the commands from the QA Complete API but I can't find anything that even tells me where to start!
Ranorex seems to be very restrictive with 3rd party integration and offers very little help on the subject.
My manager wants a demo of the compatibility before we purchase QA Complete otherwise it won't be of any use to us.
I'm hoping someone on here has some experience with this sort of thing and can point me in the right direction :-)
Thanks!!
Here is the ranorex library documentation:
http://www.ranorex.com/Documentation/Ranorex/html/R_Project_RanorexApi.htm
I don't know QA Complete, but as far as help with ranorex goes, when ranorex runs a test suite it uses the Run method in the TestSuiteRunner class in the Core.Testing Namespace.
This method returns 0 when a test suite is successful, and -1 if it is a failure, I assume this can be used by QA Complete to indicate sucess or failure
Test Collab integrates well with Ranorex.
We've published an extensive guide to integrate Test Collab with Ranorex for test management.
(Disclaimer: I'm co-founder of Test Collab)
Some years ago I´ve tried to work with BDD and some tools of Acceptance tests, like Selenium/Web-driver, Fitnesse and JBehave.
I´d like to back to work with that in my current project, so I would like to know what the community is using of tools to perform that!
My project is based on Java.
Acceptance tests tools ?
BDD tools?
Should I consider a scripting language ?
The most successful teams are using conversations and not worrying too much about the tools!
Having said that, here are a few Java BDD tools that are in use in the community:
JBehave* (still)
Cucumber for the JVM (I don't think it's been as maven-ized as JBehave)
Fitnesse (though I recommend putting "Slim" behind it instead of "Fit")
Custom DSLs (it's not that hard).
Selenium is still the automation tool of choice for Java and the web.
*I helped write JBehave. One reason we got into it in the first place was because the acceptance tests we saw using scripts were such an astonishing mess. Meh. Also, you can't collaborate with the business or have conversations about scripts. I strongly recommend having conversations first**, then worrying about the tools!
** If you're working on your own, buy a rubber duck.
I strongly recommend spockframework + Geb. You need groovy support though. We have lot of tests running as part of CI every night. The reports are in junit format (being enhanced to be used by business users soon) and hence can be published to servers like Hudson or Sonar.
We are in search of an automated testing tool for our project. As we are in testing department we prefer a tool which would have less programming in it. Please suggest some tools for us .Till now we are testing our application manually.
Our project is being developed in Java.
Is there any freeware tool that I could use or is it better to go for a paid tool?
Thanks in Advance.
Less programming? You'll need something like JUnit to write unit tests if you want to do serious regression testing, but unit tests require you to write some code
Here's a big list of open-source testing tools, some of them may offer what you want: http://java-source.net/open-source/testing-tools/junit
For example, T2 claims to be a random testing tool. As one, it is fully automatic, but one must keep in mind that the code coverage of random testing is in general very limited. It should be used as a complement to other testing methods. T2 checks for internal errors, run time exceptions, method specifications, and class invariant.
Not sure if you mean a CI tool or not, but we use Hudson at Zappos and it works pretty well.
http://hudson-ci.org/
..and there's also CruiseControl: http://cruisecontrol.sourceforge.net/
If you're not talking about CI, maybe you mean QA testing - in which case you should take a look at something like Selenium (for web apps):
http://seleniumhq.org/
If you're doing GUI testing? I'm not really familiar with that area, but I've heard about WinRunner and Rational:
http://en.wikipedia.org/wiki/HP_WinRunner
http://www-01.ibm.com/software/rational/offerings/quality/
..though neither are really free tools. Something like AutoIT might help you move widgets around, but it lacks the reporting parts:
http://www.autoitscript.com/autoit3/index.shtml
There could be two answer to you question:
Besides Selenium, though it has ample of advantages, I am reading about another tool which uses same API which Selenium use. The only changes in API I have seen so far is it reduces the complexity of functions thus making it more easier and simpler for user who is learning.
The tool is called 'Helium' and it has 50% (and more) less complex functions and code as Selenium has.
The only problem with this tool is it is paid tool for learning purpose and for implementing not-so-big scale project you can use it. But yeah after some time its gonna cost you.
I have implemented some code on Helium. Please let me know , if you face any issue initially or you are thinking to implement it.
Other being, you can use Selenium Builder(http://khyatisehgal.wordpress.com/2014/05/26/selenium-builder-exporting-and-execution/) which is an advanced form of Selenium IDE. It imports your command in different languages and does work more effectively and efficiently as Selenium IDE does(http://khyatisehgal.wordpress.com/2014/05/25/selenium-builder/) . So you can import scripts in Eclipse IDE and just execute them as is.
Please let me know , if you have any doubt in any of the tool.