There is a existing script which run daily. Is it possible to track return code of scripts after its completion to track that it was successfully finished or not in its previous run? I don't want to modify the existing script.
After the script was executed, its exit code is stored in the $? variable, which you could save to a file. For example
/path/to/script.sh
echo $? >> /path/to/script.log
You probably want to save the date too:
/path/to/script.sh
result=$?
echo $(date) $result >> /path/to/script.log
Related
I've written a script that checks for a specific file of format("OLO2OLO_$DATE.txt.zip") in the ftp server and then copies it to my local machine:
/usr/bin/ftp -n 93.179.136.9 << !EOF!
user $USR $PASSWD
cd "/0009/Codici Migrazione"
get $FILE
bye
!EOF!
echo "$FILE"
But I'm not getting the desired result from this.
This line triggers the error.
SOURCE_FOLDER="/0009/"Codici Migrazione""
It tries to execute the command Migrazione with the environment variable SOURCE_FOLDER set to /0009/Codici which doesn't exists.
What you probably wanted to do was:
SOURCE_FOLDER="/0009/Codici Migrazione"
I need to ferch the result of ngxtop command in a file. But when i m trying to run it from a shell script file it goes to a running state untill I manually kill or stop it...
I do this proccess with the next cli command group:
touch file.log && echo "please press CTRL+C to stop process and save ngxtop file.log" && ngxtop print request status http_referer >> file.log
When you run this code will save the output in file.log the you can show the informacion with next code:
cat file.log
This is an example when the process is complete and we load some sites in nginx while the command is executing screeshoot of ngxtop trace
I have developed a code that composed of two files:
An 'envelop bash file', which does a few things and writes to a log-file, and then at some point runs into a for loop in which within it it executes one job at a time using bsub.
And 'an internal bash file', which gets as input the name of the log-file (in addition to other input values that necessary for its execution), and executes process X (using the input values it received from the 'envelop file'.
Once process X is finished, the 'internal script' writes to the log-file that process X (with its specific serial number) has been completed.
Since the for-loop of the envelop file loops 10 times, there are at least 10 parallel processes that being executed and run in parallel, and they all being executed with bsub given the SAME log-file name. The idea is that they would all report to the same log-file once they completed their execution of Process X.
The general procedure works well, and in each case process X is being executed, and the log-file accumulates as required all the notifications regarding the completion of process X. However, in some incidences we see that the writing to the log-file get disturbed and output lines of two parallel runs are running into each other.
I would like to lock the log-file in such manner that would allow it to receive text only from one parallel run at a time. The idea is to avoid cases where the text becomes mixed due to two processes that write by chance to the log-file exactly at the same time.
Here is the part of my envelop file which call to the bsub (I reduced the content to the minimum necessary):
for ((i=1;i<=$batchesnumber; i++));
do
bsub -J $SerialName -q normal "bash FetchFasta.bash $genome_fa ${SerialFileName}".bed" $logfile"
done
Here is the part of my internal file that echo to the log-file:
(
echo "~~~~~~~~~~~~~~~~~~"
echo "^^^^^^^^^^^^^^^^^^"
echo -n "Completed running "; bedtools -version
echo "bedtools getfasta -s -fi $genome_fasta -bed $mySerialFile -fo ${mySerialFile%.*}".fa" "
echo "Run's completion time is: $timedate"
echo -e "~~~~~~~~~~~~~~~~~~\n"
) >> $logfile
I would appreciate any useful solution!
There's a couple of ways I can think of going about this:
Have each job write its output to a different file (use $LSB_JOBID inside each job to name the file). Then use another "cleanup" job to concatenate all of the ouptut into a single file. You can use job dependencies (bsub -w) to make sure the cleanup job runs after all the other jobs are done.
Implement a lock inside your "internal" job to make sure only one of them writes to a file at a time. This is a lot simpler than it might sound, one way to do it is to have each job try to create the same directory with mkdir before writing to the file and then delete the directory after its done. If they fail to create the directory it's because another one of the jobs got to it first and is currently writing to the file.
Here's a snippet illustrating #2 in bash:
# Try to get the lock every second
while ! mkdir lock &> /dev/null ; do
sleep 1
done
# Got the lock, write to the logfile
echo blahblahblah >> $logfile
# Release the lock
rmdir lock
I should mention an important caveat here though: if one of your jobs dies while it's "holding the lock" (say someone sends it a kill signal at the wrong time) then it'll never remove the directory and all the other jobs won't be able to create it, so they'll just keep sleeping forever.
I got a Korn Shell that calls a SQL/PLUS to check and spool data into a .CSV file in UNIX.
This KShell is working fine on Unix, it creates the file and Return 0.
Launching the Job from UC4 AppWorx i want him to Attach the Spooled File in UNIX in the Notification sended by the Job when he Finish.
I want this to work this way:
1º I Launch the Job
2º It checks the data, if data is founded then it creates a file in /tmp directory in UNIX with the .CSV extension.
3º When the job finishes he send me an email with Spool File (.CSV) in Unix.
Is there any way? How can i make this?
Thanks.
You'll need to create an chain in Appworx (the docs should be able to walk you through it). This chain will have 1 or more jobs.
First, you don't need a k script to call SQL/PLUS. You can invoke SQL/PLUS directly. Write the script as a .sql file (it can include sqlplus directives, sql, and PL/SQL as needed). Set the job as "program type" AWSQLP. Point it at the .sql script that you have made available to Appworx.
The sqlplus script can use logic to determine if it should create a file. If it should, it can write out files directly (though getting proper .csv files from it can be a pain).
Then, attach a notification to the job, and the notification object should be set to do an email attachment. You'll have to use the "pattern" type, and put in a full filepath to the csv. Substitution variables can be used if you want a new filename each invocation.
Depending on your version, some of these options can be moved around a bit (We just upgraded last year, UC4 no longer owns it). Click on the help menu and do to the documentation entry... it's not the best in the world, but far from the worst.
first of all thanks for answering.
I usually create a JOB lets say, SEM_CHECK_THINGS, with 1 prompt defined on UC4 that runs a Query in the Database to check if the table, test_table (Example) got data, to do this i use select decode(count(*),0, 'N', 'Y') from test_table;
This job also executes a simple KShell in unix:
KShell Content:
echo "Job Name: $1"
echo "Job Control Flag: $2"
jobName=$1
jobFlag=$2
echo "Job ${jobName} started ..." >> $logFile
date >> $logFile
if [[ ${jobFlag} == "Y" ]]; then
echo "Job ${jobName} executed successfully with data found." >> $logFile
echo "Job ${jobName} executed successfully with data found."
exit 1
else
echo "Job ${jobName} finished with no data found." >> $logFile
echo "Job ${jobName} finished with no data found."
exit 0
fi
I usually force "ABORT" by using Exit 1 if data is found to request another Job that will execute an .SQL that will spool the data from the test_table.
whenever sqlerror exit sql.sqlcode
whenever sqlerror exit 1
prompt this is a test
set echo off
set trimspool on
set trimout off
set linesize 1500
set feedback on
set newpage none
SET HEADING OFF
set und off
set pagesize 10000
alter session set nls_date_format = 'dd-MON-yyyy HH24:MI:SS';
spool &1
SELECT 'PREV_RESULTSET;LAST_RESULTSET;NR_COUNT' FROM DUAL
UNION ALL
SELECT PREV_RESULTSET||';'||LAST_RESULTSET||';'||COUNT(1) NR_COUNT
FROM SEM_REPORT_PEDIDOS
GROUP BY PREV_RESULTSET, LAST_RESULTSET;
spool off
exit 1
By using the Spool &1 and by "Hardcoding" the file testspool.csv in the "Other Output" option in the Notification of that Job i managed to do this, to receive an email with the content i need/want from that table.
But i want really want its to make this in one single job, make a validation, if data is found then Spool and attach .CSV file to the email notification sent by that job.
I'm writing a shell script that basically transfers data files through SFTP to a database server and then invokes a pl/sql procedure which loads the data from those files (external tables) into internal database tables.
I've been doing some research on effective exception handling in shell scripts and it appears the set -e option can be used to terminate a script with an error whenever any command in the script runs which returns a non-zero exitcode.
So, my plan is to have a script which contains all of the processing that needs to get done (SFTP, moving/deleting files, calling pl/sql procedure, etc...) and to include set -e at the top of the script. I also plan to redirect output to a log file in this script.
Then, I plan to have another script that calls the main processing script and then emails the log that gets produced with either a "Success" or "Failure" indicator in the subject of the email.
Are there any "gotchas" that any of you can foresee in this approach or does this seem reasonable?
Sounds reasonable.
One thing you may also do make it one command and less scripts:
someSFTPscipt &> somelogfile.txt; if [ $? -eq 0 ]; then echo "Success"; else echo "Failure"; fi
someSFTPscipt &> somelogfile.txt; redirects the output of the script to a logfile
if [ $? -eq 0 ]; then echo "Success"; else echo "Failure"; fi checks whether it succeeded (returned 0) or failed (any other non-zero value). Simply replace the echo with your mail commands.
Thanks for all of the feedback on this.
I ended up going with this "wrapper" shell that calls the main processing shell. A cron is going to launch this daily in my particular case.
Comments are certainly welcome if this can be improved.
#!/bin/sh
################################################################################
# Author : Zack Macomber #
# Date : 02/22/2012 #
# Description: Calls main_process.sh and emails results of the process. #
# Also appends to master log file #
################################################################################
# Flag any errors that occur during processing
set -e
# Set newly created files to "rw" for everyone
umask 111
#############
# VARIABLES #
#############
EMAIL_RECIPIENTS=my_email#some_domain.com
MAIN_DIR=/scripts/
#############
# FUNCTIONS #
#############
send_email()
{
uuencode results.log results.log | \
mailx -s "DATA_LOAD $1 - consult attached log for details" $EMAIL_RECIPIENTS
}
################
# MAIN PROCESS #
################
cd $MAIN_DIR
sh main_process.sh > results.log && send_email SUCCESS || send_email FAILURE
cat results.log >> pub_data_load.log
exit 0