When wildcards is used with file trigger in autosys to watch multiple files, is there any way or attribute to get the file name which triggered it.
You can define the file trigger job using a JIL script:
insert_job: file_trigger
job_type: FT
machine: machine_name
owner: user_name
watch_file: /path/to/file/*pattern*
watch_file_type: CREATE
Then define a CMD job to execute a command after success of the file_trigger job:
insert_job: cmd_job
job_type: CMD
command: /path/to/script/script.sh
machine: machine_name
owner: user_name
condition: success(file_trigger)
In script.sh, you can find the files that match the pattern you are looking for:
#!/bin/bash
files=(/path/to/file/*pattern*)
echo "The first matching file was ${files[0]}"
${files[0]} holds the file name of the first file that matches the pattern. The first file that matches the file trigger's wildcard is also the same file that matches the shell script's wildcard.
Related
I was solving a unix problem on Katacoda. I am solving it as instructed but it gives me an error.
The problem is to display number of lines and words in a given input file
Write a command/logic, which will read the content from a given input file and display the number of lines and number of words in the file.
Your default logged in directory: /home/scrapbook/tutorial/
Input file location: /home/scrapbook/tutorial/unix_countproject/
Input file Name: input.txt
Script file location: /home/scrapbook/tutorial/
Script file Name: script.sh
Instructions:
You can view content of input file (input.txt) in the folder at /home/scrapbook/tutorial/unix_countproject/.
Note that this folder and file already exists and are located in the path mentioned above.
You can open the script.sh file using vi editor, Write the logic inside the file to display the number of lines and number of words in the input file (input.txt), After writing the logic , save the file and quit from vi editor .
To test your command/logic, Run the shell script file (script.sh) at the terminal using the below command
sh /home/scrapbook/tutorial/script.sh
If any issues while running the above command, Modify the script.sh file and repeat the point#:3
Don't use any echo statement in the script.sh file, even in the commented line/code
On completion of the task, click on SUMMARY or CONTINUE to proceed to next assignment.
Example : Contents of sample Input file named as input.txt - starting from following line
Hello all
Welcome to all of you
Expected Output:
2 7
As instructed, I store my logic in the script.sh file and it executes successfully and gives me the expected output but it's not accepting my answer.
$ cd unix_countproject
$ cat input.txt
Hi all
Welcome to Unix module
$ vi script.sh
Inside script.sh : wc -l -w input.txt|cut -c-6
(exit vi)
$ chmod +x script.sh
$ ./script.sh
2 6
Then I click on complete but it's not accepting it.
I am defining a shell script in one of the stages in my Jenkinsfile. How can I access a variable that I define in my Jenkinsfile with the shell script?
In below scenario , I am writing the value of the shell variable to a file and reading into a groovy variable. Is there a way to pass data from shell to groovy without writing it to file system?
unstash 'sources'
sh'''
source venv/bin/activate
export AWS_ROLE_ARN=arn:aws:iam::<accountid>:role/<role name>
layer_arn="$(awssume aws lambda list-layer-versions --layer-name dependencies --region us-east-1 --query \"LayerVersions[0].LayerVersionArn\" | tr -d '\"')"
echo $layer_arn > layer_arn
'''
layer_arn = readFile('layer_arn').trim()
You can can shell command line, providing variable value.
sh "some stuff $my_var"
You can defined environment variable and use it within your shell
withEnv(["MY_VAR=${my_var}") {
sh 'some stuff'
}
Regards
I'm very new to AutoSys jobs and I have following commands stored in single jil file. let's call it, test.jil.
insert_job: job_A
command: echo 'mock'
description : mock job A
sendevent -E JOB_ON_ICE -J job_A
I'm trying to run jil < test.jil. it doesn't recognize sendevent. How can i get it working ?
In jil file we can write commands like insert_job,
delete_job,update_job but sendevent is different command which should be triggered by autosys agent.
So you can separately create executable file in which you can write that sendevent command and execute it through CLI.
Thanks.
Actually there is a change in one of the last service packs. For your JIL you can specify status:
insert_job: test_job2
command:dir
machine:localhost
status:on_ice
The valid parms are:
FAILURE, INACTIVE, ON_HOLD, ON_ICE, ON_NOEXEC, SUCCESS, or TERMINATED.
I am exporting data from hive to text file saved to the local file system using the query below:
INSERT OVERWRITE LOCAL DIRECTORY '/local/file/system/directory'
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
NULL DEFINED AS ''
SELECT * FROM staging_table WHERE date='2017-05-28';
The query generates the file as expected but I am having problem deleting the file because of the permission.
-rw-rw-r-- 1 hive hive 12345 May 31 13:03 000000_0
Is it possible to change the permission or owner of the file?
You will need Sudo or root access for doing this:
sudo chown -R NewOwnerName:NewGroupName /local/file/system/directory
If you are in linus just change the permission to 777 then that should be editable to any users
sudo chmod -R 777 directoryPath
or
sudo chmod 777 filePath
If you are root user you wouldn't need sudo
If you are in windows use icacls
C:\>icacls "D:\test" /grant John:(OI)(CI)F /T
According do MS documentation:
F = Full Control
CI = Container Inherit - This flag indicates that subordinate containers will inherit this ACE.
OI = Object Inherit - This flag indicates that subordinate files will inherit the ACE.
/T = Apply recursively to existing files and sub-folders. (OI and CI only apply to new files and sub-folders). Credit: comment by #AlexSpence
.
I have a shell script where I am calling the hana.scr script from within the main script. The hana.scr contains the below code.
chmod 777 /data/auto/SLT.out; rm -rf /data/auto/SLT.out; hdbsql -n plhesappr61 -i 00 -u USR -p $#^F#$GGG -o /data/auto/SLT.out "Select sum("ERPACC_RPPCLNT200"."VABD"."NETWR") FROM "ACC_CLNT"."VFKH" inner join "ACC_CLNT"."VNRO" on ("ACC_CLNT"."VNRO"."VBELN"="ACC_CLNT"."VFKH"."VBELN") where FKART in ('ZFP1','ZFP3') and FKDAT = (select ADD_DAYS (TO_DATE (current_date, 'YYYY-MM-DD'), -1) "add_days" from dummy) group by FKDAT";
When I run the main script manually, it calls this script fine and the SLT.out file is also generated.
But when I schedule it in cron, the main script executes just fine, except for this hana.scr which does not seem to execute because it does not does not even remove the old file as per the second command rm in the hana.scr.
The cron is the same user as the one I run the script manually with.
I read that if the cron does not get the same environment to run, these issues happen. I tried to import the UNIX profile of the user before executing as the hana.scr as well, but was not successful.
Below is the cron command which runs the main script which calls the hana.scr from within: Used absolute paths..
37 0,2,3,4,5,6 * * * /data/esb/auto/./main.sh R > /data/esb/auto/main.log
The hana.scr is executed in the below manner:
./hana.scr;
check6=$? ;
if [ $check6 = "1" ]
then
echo "***********HANA counts were not generated**********"
fi
After /data/esb/auto/./main.sh your current directory is not changed to /data/esb/auto/. I think you started main.sh from the commandline while your $PWD was the same as where hana.scr was.
Test it from the commandline with
cd /
/data/esb/auto/main.sh
How to fix?
The worst solution is changing the crontab line into
37 0,2,3,4,5,6 * * * cd /data/esb/auto; /data/esb/auto/main.sh R > /data/esb/auto/main.log
That is a workaround for the crontab but main.sh still fails when started from a different directory.
Slightly better is using the complete path in main.sh when you call hana.scr
myscriptdir=/data/esb/auto
..
${myscriptdir}/hana.scr
When you change the folders you need to edit the files and repair the settings.
You can try to use some config file with settings or let main.sh figure out what in which directory it is:
Getting the source directory of a Bash script from within