We're writing software for an embedded platform for which we also need automated on-device testing.
When a push/tagging is made to the repository a GitLab pipeline is started. The first step is to build the software.
The next step is to execute a custom written Python script (doing the testing) on a machine running locally in our office. This machine is physically connected to the embedded device via USB, hence the testing-part of the CI pipeline needs to be executes on this machine exclusively. The Python script needs to be executes multiple times with different parameters, but may (and should) run concurrently if possible.
Is this setup possible with GitLab runners, and if so, how do I configure it to only run this exact part of the pipeline locally?
Thanks in advance
Related
We are having some difficulties implementing the Azure DevOps pipeline as a team. We are conducting the testing on our agent-hosted Windows os in the background. A couple of our test scripts were successful, while others were unsuccessful for unknown reasons.
However, the identical test scripts are all run with a pass in locally 100% in versus code result. Unable to determine why the agent-hosted Windows operating system is failing
Ideally, all suggestions would be constructive.
All test scripts which execute need to pass in azure pipeline any suggestion would be helpful
I am trying to run my Automated test cases deployed on a virtual machine and trying to trigger it with the help of Octopus Deployment tool. I installed test agent and Octopus Tentacle on my machine. Octopus is triggering the DLL's for Automated test cases very well.But while Octopus trying to run the test cases it's giving me an Error as below:-
Microsoft.VisualStudio.TestTools.UITest.Extension.UITestException: To run tests that interact with the desktop, you must set up the test agent to run as an interactive process. For more information, see "How to: Set Up Your Test Agent to Run Tests That Interact with the Desktop" (http://go.microsoft.com/fwlink/?LinkId=255012)
Error 01:59:38
If you are running the tests as part of your team build, you must also set up the build agent to run as an interactive process. For more information, see "How to: Configure and Run Scheduled Tests After Building Your Application" (http://go.microsoft.com/fwlink/?LinkId=254735)
I setup my password in test agent and set it as intractive process but still i am facing the same issue.
I am triggering my DLL's as below through Octopus.
& "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" "C:\MyWebaPP\Automated_test\Automated_test.dll"
I tried each and every way i found.Please help me out in this.
Thanks in advance!!
We recently encountered the same problem.
During our research, we found this post on the Octopus support forum:
http://help.octopusdeploy.com/discussions/questions/5080-tentacle-running-interactive-tests
We also contacted Octopus Deploy by mail, and they essentially gave us the same response.
While we had no luck with the "scheduled task for test run" approach, we eventually managed to get it working by running the Octopus Tentacle as a process rather than a service.
The challenge here was making sure the Tentacle would start when our test machine started. We wanted this to happen automatically, so RDPing in and starting the process every time was out of the question (this also caused some additional problems for the UI test run...).
The final working solution was to schedule a task that would start the Octopus Tentacle as an interactive process whenever the machine booted (i.e. run Tentacle.exe directly), and then make sure we never RDP to this machine. Make sure the task has sufficient privileges, and that it "Runs whether the user is logged in or not". Also, remember to disable automatic startup of the Octopus Tentacle Service.
Edit: We had some trouble making this solution work across all our environments. It seems that for security reasons, newer versions of Windows are quite skeptical about allowing scheduled tasks to start interactive processes when there is no user logged on.
We did another search for possible solutions, and came across FireDaemon Pro (commercial software), which allows us to register interactive Windows services that run under a domain user. Not quite sure how it works, but they seem to be able to run a UI from a Windows service in session0 (the UI is also isolated). The Octopus Tentacle starts without complain, and the UI tests run the way we want them to.
I have written webdriver+TestNG scripts in windows using java. Now there is an upcoming requirement to run the same scripts in UNIX server without much of code modification. I have heard about running tests in headless browser, but I dont know about this much. I searched a lot but there is no clear and simple response to start looking into this.
Is it possible to just change the driver instance to htmlunitdriver and run the same in unix environment? How could I create the tests as a package and move to unix environment for running the scripts?
You can run your existing script by making few changes in driver instance creation. Suppose you are using Firefox driver for your test, then you can run your script in headless mode using virtual display Xvfb (Xvfb is an X server that can run on machines with no display hardware and no physical input devices).
Below are the steps to run your tests in headless mode:
Install Xvfb
Start the Xvfb by executing this command Xvfb :99 -screen 0 1024x768x24 & Xvfb Manual
Then create a Firefox driver instance that uses virtual display started in step 2 as below:
FirefoxBinary fb = new FirefoxBinary();
fb.setEnvironmentProperty("DISPLAY", ":99");
WebDriver driver = new FirefoxDriver(fb,null);
Now your script will run in headless mode. You may need to change few other things like path of your test data or any other references that uses windows file system (like C:\)
I'm trying to setup automated GUI tests in ESXi Virtual Machines using TestComplete. The problem, as I understand it, is that when no remote desktop connection is made to the ESXi virtual machine, then it is impossible for TestComplete to perform screen captures and therefore automate the GUI testing. As far as I understand it, this is due to the fact that Windows does not generate any user interface when nobody is viewing it.
I'm sure other have experienced this problem. How did you solve it ? Are you using a third party computer which automatically launch remote desktop connections prior to running the tests ?
Would it be possible to launch a remote desktop from a head-less virtual machine to another to fake somebody viewing ?
Any other smarter solutions I haven't thought about ?
You should be able to log in to Windows on the VM's console using the vSphere client, then close vSphere, and Windows will still believe the user is viewing the console. Simple as that. :)
So there shouldn't be a need to involve remote desktop in the mix.
As long as your tests then run as that logged-in Windows user, you should be fine.
This technique has always worked like a charm for me with certain Watir, Selenium, and MS UI Automation tests that depend on having an interactive desktop.
If you need to reboot the VM automatically before/during the test, instead of logging in manually in the vSphere client, you can make Windows log in as an arbitrary user automatically - check the "control userpasswords2" command, or you can use the Sysinternals app "Autologin":
http://technet.microsoft.com/en-us/sysinternals/bb963905
Only catch with this technique is that you need to be able to launch your tests while not viewing the console on the VM, but it sounds like you've already taken care of that?
If you need a solution for launching your tests remotely, I highly recommend using Jenkins or Hudson to kick off tests/collect results from the VM. Jenkins has changed my life in this regard.
You may consider using the Network Suites functionality of TestComplete:
http://smartbear.com/support/viewarticle/16849/
It can open Remote Desktop connections on its own, control tests on remote PCs, and pull the logs back to the "master" project. This feature is designed to be used for distributed tests, and looks like it's just what you need.
As for opening RDP to a head-less VM, it should not be a problem - it's up to Windows to "think" about this. You just open RDP and it works even if there is no monitor attached to the remote PC/VM.
I hope this helps,
Alex
You can always use VNC with checking the option "Do nothing" when disconnecting viewer. This way you'll trick windows to generate the image.
I am developing OpenCL code on a linux cluster through SSH -
are there any tools that would make this process easier, i.e.
something like NVIDIA Parallel Nsight for OpenCL ?
No there is no such tool, though you might try developing your code using ordinary computer and post production versions there..
If the computer where you perform development is also running Linux, you can easily mount a remote folder as local. In a Gnome environment, open Nautilus (the file manager), click File => Connect to server, chose SSH, fill the required parameters, and you have a remote folder as local.
You can then use any IDE you want to develop code, and maybe perform simple runs, tests and debugs if the OpenCL tools (compiler, debugger) you're using remotely are also installed locally. However, To compile and properly run the code on the cluster, you need to use the ssh client on the command line.