I am building a Qt GUI application via Jenkins. I added 3 build steps:
Building the test executable
Running the test executable
compiling a coverage report with gcovr
For some reason, the shell task for running the test executable stops after execution. Even a simple echo does not run after. The tests are written with Google Test and output xUnit XML files, which are analyzed after the build.
Some tests start the applications user interface, so I installed the jenkins xvnc plugin to get them to run.
The build tasks are as follows:
Build
cd $WORKSPACE/projectfiles/QMake
sh createbin.sh
Test
cd $WORKSPACE/bin
./Application --gtest_output=xml
Coverage Report
cd $WORKSPACE/projectfiles/QMake/out
gcovr -x -o coverage.xml
Now, an echo at the end of the first build task is correctly printed, but an echo at the end of the second is not. The third build task is therefore not even run, although the Google Test output is visible. I thought that maybe the problem is that some of the Google Tests fail, but why whould the script stop executing just because the tests fail?
Maybe someone can give me a hint on why the second task stops.
Edit
The console output looks like this:
Updating svn://repo/ to revision '2012-11-15T06:43:15.228 -0800'
At revision 2053
no change for svn://repo/ since the previous build
Starting xvnc
[VG5] $ vncserver :10
New 'ubuntu:10 (jenkins)' desktop is ubuntu:10
Starting applications specified in /var/lib/jenkins/.vnc/xstartup
Log file is /var/lib/jenkins/.vnc/ubuntu:10.log
[VG5] $ /bin/sh -xe /tmp/hudson7777833632767565513.sh
+ cd /var/lib/jenkins/workspace/projectfiles/QMake
+ sh createbin.sh
... Compiler output ...
+ echo Build Done
Build Done
[VG5] $ /bin/sh -xe /tmp/hudson4729703161621217344.sh
+ cd /var/lib/jenkins/workspace/VG5/bin
+ ./Application --gtest_output=xml
Xlib: extension "XInputExtension" missing on display ":10".
[==========] Running 29 tests from 8 test cases.
... Test output ...
3 FAILED TESTS
Build step 'Execute shell' marked build as failure
Terminating xvnc.
$ vncserver -kill :10
Killing Xvnc4 process ID 1953
Recording test results
Skipping Cobertura coverage report as build was not UNSTABLE or better ...
Finished: FAILURE
Generally, if one Build Step fails, the rest will not be executed.
Pay attention to this line from your log:
[VG5] $ /bin/sh -xe
The -x makes the shell print each command in console before execution.
The -e makes the shell exit with error if any of the commands failed.
A "fail" in this case, would be a return code of not 0 from any of the individual commands.
You can verify this by running this directly on the machine:
./Application --gtest_output=xml
echo $?
If the echo $? displays 0, it indicates successful completion of the previous command. If it displays anything else, it indicates an error code from the previous command (from ./Application), and Jenkins treats it as such.
Now, there are several things at play here. First is that your second Build Step (essentially a temporary shell script /tmp/hudson4729703161621217344.sh) is set to fail if one command fails (the default behaviour). When the Build Step fails, Jenkins will stop and fail the whole job.
You can fix this particular behaviour by adding set +e to the top of your second Build Step. This will not cause the script (Build Step) to fail due to individual command failure (it will display an error for the command, and continue).
However, the overall result of the script (Build Step) is the exit code of the last command. Since in your OP, you only have 2 commands in the script, and the last is failing, it will cause the whole script (Build Step) to be considered a failure, despite the +x that you've added. Note that if you add an echo as the 3rd command, this would actually work, since the last script command (echo) was successful, however this "workaround" is not what you need.
What you need is proper error handling added to your script. Consider this:
set +e
cd $WORKSPACE/bin && ./Application --gtest_output=xml
if ! [ $? -eq 0 ]; then
echo "Tests failed, however we are continuing"
else
echo "All tests passed"
fi
Three things are happening in the script:
First, we are telling shell not to exit on failure of individual commands
Then i've added basic error handling in the second line. The && means "execute ./Application if-and-only-if the previous cd was successful. You never know, maybe the bin folder is missing, or whatever else can happen. BTW, the && internally works on the same error code equals 0 principle
Lastly, there is now proper error handling for the result of ./Application. If the result is not 0, then we show that it had failed, else we show that it had passed. Note, this since the last command is not a (potentially) failing ./Application, but an echo from either of if-else possibilities, the overall result of the script (Built Step) will be a success (i.e 0), and the next Build Step will be executed.
BTW, you can as well put all 3 of your build steps into a single build step with proper error handling.
Yes... this answer may be a little longer than what's required, but i wanted you to understand how Jenkins and shell treat exit codes.
Related
I'm a big fan of Python's try/finally and the builtin trap command in various shells. I have a Make target to which I would like to apply the same sort of logic. Suppose I have this target and dependencies:
test : start-server run-test-group-1 run-test-group-2 stop-server
If tests fail during the run-test-* phases, the stop-server actions won't execute. Is there a way to guarantee the stop-server actions are executed, even if "-k" is not given? I realize I could place "-" before the relevant command(s) in in the run-test-* actions, but I think that would cause make to exit with a 0 status, causing the controlling process to think the tests succeeded. I still want the parent process to know the tests failed.
Use the shell trap mechanism instead and run make recursively, perhaps? Like this:
test:
trap EXIT ERR "$(MAKE) stop-server"; \
$(MAKE) start-server && \
$(MAKE) actual-test
actual-test: run-test-group-1 run-test-group-2
I'm trying to execute a command remotely through Robot Framework which is failing through Robot framework and giving me the wrong exit status of 13.
But if we run this manually exit status of TTman.sh is 112 which is actually pass(Not the standard return codes).
am I doing something wrong here?
You are not getting the remote code of the remote command, in fact the RC 13 you are getting from the run is most probably from the robotframework - on run completion its RC is the number of failed cases. I.e. 13 cases should have failed, when you observed this.
To get the return code of your command, a few changes in the case are needed; this is how the semi-last line should look like, with explanations below:
${rc}= Execute Command your_command_from_the_question &>/dev/null; echo $?
First, all the output of your command (stdout & stderr) is redirected to /dev/null - to not return it. Then the special var $? is printed - it holds the RC of the last executed command (and is available in most *sh variants, like bash).
Finally, that value is stored in the ${rc} robotframework variable, and you can do whatever checks you need on it, further in the case.
This approach has one drawback - as stderr is hidden, you will not be able to see any errors coming from running the command. But if it was not, then they would be interleaved with the RC, which would have required further processing of the {rc} var, to get the desired value. If you need it (the stderr output in case of failures), change accordingly.
P.S. don't add screenshots of a source in a question, it is much less usable than a text version.
I work with Rscript on a cluster. qmake (a specialized version of GNU-make for the cluster) is used to parallelize jobs on several nodes. But Rscript seems to need to write a Xauthority file and it creates an error when every nodes work in the same time. In this way, my makefile-bases pipeline stops after the first group of parallelized tasks and don't start the next group of tasks. But the results of the first group are ok.
I'm also invoking usr/bin/xvfb-run ( https://en.wikipedia.org/wiki/Xvfb) when runnning RScript.
I've already changed the ssh-config (FORWARD X11 yes) but the problem persists.
I also tried to change the name of Xauthority file for each job but it didn't work (option -f in Rscript).
Here is the error which appear at the beginning of the process
/usr/bin/xauth: error in locking authority file .Xauthority
Here is the error which appears before the process stops :
/usr/bin/xvfb-run: line 171: kill: (44402) - No such process
qmake: *** [Data/Median/median_B00FTXC.tmp] Error 1
I'm supposed to run some jbehave(automated) tests in bamboo. Once the tests run I'll generate some junit compatible xml files so that bamboo could understand the same. All the jbehave tests are ran as part of a script, because I need to run the jbehave tests in a separate display screen(remember these are automated browser tests). Example script is as follows.
Ex:
export DISPLAY=:0 && xvfb-run --server-args="-screen 0, 1024x768x24"
mvn clean integration-test -DskipTests -P integration-test -Dtest=*
I have one more junit parser task which points to the generated junit compatible xml files. So, once the bamboo build runs and even if all the tests pass, I get red build with the message "No failed tests found, a possible compilation error occurred."
Can somone please help me on this regard.
Your build script might be producing successful test reports, but one (or both, possibly) of your tasks is failing. That means that the failure is probably* occurring after your tests complete. Check your build logs for errors. You might also try logging in to your Bamboo server (as the bamboo user) and running the commands by hand.
I've seen this message in the past when our test task was crashing halfway through the test run, resulting in one malformed report that Bamboo ignored and a bunch of successful reports.
*Check the build log to make sure that your tests are indeed running. If mvn clean doesn't clean out the test report directory, Bamboo might just be parsing stale test reports.
EDIT: (in response to Kishore's links)
It looks like your task to kill Xvfb is what is causing the build to fail.
18-Jul-2012 09:50:18 Starting task 'Kill Xvfb' of type 'com.atlassian.bamboo.plugins.scripttask:task.builder.script'
18-Jul-2012 09:50:18
Beginning to execute external process for build 'Functional Tests - Application Release Test - Default Job'
... running command line:
/bin/sh
/tmp/FUNC-APPTEST-JOB1-91-ScriptBuildTask-4153769009554485085.sh
... in: /opt/bamboo-home/xml-data/build-dir/FUNC-APPTEST-JOB1
... using extra environment variables:
<..snip (no meaningful output)..>
18-Jul-2012 09:50:18 Failing task since return code was 1 while expected 0
18-Jul-2012 09:50:18 Finished task 'Kill Xvfb'
What does your "Kill Xvfb" script do? Are you trying something like pkill -f "[x]vfb"? pkill -f silently returns non-zero if it can't match the expression to any processes.
My solution was to make a 'script' task:
#!/bin/bash
/usr/local/bin/phpcs --report=checkstyle --report-file=build/logs/checkstyle.xml --standard=PSR2 ./lib | exit 0
Which always exits with status 0.
This is because PHP code sniffer return exit status 1 when only 1 coding violation (warning / error) is found which causes the built to fail.
Turns out to be a simple fix.
General bamboo behavior is to scan the entire log and see for any failure codes(1). For this specific configuration i had some 6 scripts out of which one of them was to kill the xvfb(frame buffer). For some reason server is not able to kill xvfb and that task was returning a failure code. Because of this, though all the tests passed, bamboo got one of this error codes from previous tasks and build was failing.
Current fix is to remove the task which kills xvfb and the build went green! \o/.
I have used R in the past to do very basic calls to the commmand line. The example can be found here.
This time around, I am looking to mimic this code which runs successfully from the command line in Windows:
> cd C:\Documents and Settings\BTIBERT\My Documents\My Dropbox\Eclipse\Projects\R\MLB\retrosheet\rawdata
> bgame -y 2010 2010bos.eva >2010bos.txt
This is the code I am trying to run inside of R. I have already set the working directory inside of R.
dir <- paste("cd", getwd(), sep=" ")
system(dir)
system("bgame -y 2010 2010bos.eva >2010bos.txt")
I am sure this is user error, but what am I doing wrong? It appears to work initially, but returns the following error. I very well could be doing something wrong, but I believe I am using the same commands.
Expanded game descriptor, version 109(185) of 05/08/2008.
Type 'bgame -h' for help.
Copyright (c) 2001 by DiamondWare.
[Processing file 2010bos.eva.]
>2010bos.txt: can't open.
Warning message:
running command 'bgame -y 2010 2010bos.eva >2010bos.txt' had status 2
Any help you can provide will be appreciated.
You need to issue all commands in one system() call:
system(paste("cd",getwd() "&& bgame -y 2010 2010bos.eva >2010bos.txt",sep=" "))
You should already be in your working directory, so I'm not sure the cd getwd() is necessary. And you may need quotes around your path because it contains spaces. The error may be resolved by putting spaces around >.
If I were in your shoes, I would try this:
system("bgame -y 2010 2010bos.eva > 2010bos.txt")
UPDATE:
And you should probably heed this advice in the "Differences between Unix and Windows" section of ?system that says you should use shell:
• The most important difference is that on a Unix-alike
‘system’ launches a shell which then runs ‘command’. On
Windows the command is run directly - use ‘shell’ for an
interface which runs ‘command’ _via_ a shell (by default the
Windows shell ‘cmd.exe’, which has many differences from the
POSIX shell).
This means that it cannot be assumed that redirection or
piping will work in ‘system’ (redirection sometimes does, but
we have seen cases where it stopped working after a Windows
security patch), and ‘system2’ (or ‘shell’) must be used on
Windows.
Has no-one else found that system("dir", intern = T) for example doesn't work, but that you need system("cmd.exe /c dir", intern = T)? Only the latter works for me. I found this at the discussion site here (William Dunlap's post, about a third of the way down).
Also, it doesn't work with the "cd" command, but you can use the setwd() function within R and then the command will be executed within that directory.
I created the following functions for convenience, for executing programmes and running commands:
#the subject is an input file that a programme might require
execute <- function(programme, subject.spec = "", intern = FALSE, wait = FALSE){
if(!identical(subject.spec, "")){subject.spec <- paste0(" ", subject.spec)} #put space before the subject if it exists
system(paste0("cmd.exe /c ", programme, subject.spec), intern = intern, wait = wait)
}
command <- function(command, intern = TRUE, wait = FALSE){
system(paste("cmd.exe /c", command), intern = T, wait = wait)
}
Does it break your code when you get error 1 or does execution continue?
Whenever executing system commands through another language it is useful to print the system call before you call it to see exactly what is happening, pull up the shell you are intending to use and check for the same error. As the command is executed correctly this could be a hickup in bgame or R.
If you look at http://astrostatistics.psu.edu/datasets/R/html/base/html/shell.html you can see the variable flag passed to the system call."flag the switch to run a command under the shell. If the shell is bash or tcsh the default is changed to "-c"."
Also "the shell to be used can be changed by setting the configure variable R_SHELL to a suitable value (a full path to a shell, e.g. /usr/local/bin/bash)."