This is my pig script -
fs -cp -f gs://$codepath/db_password.sh file://$dataprochome/db_password.sh;
fs -cp -f gs://$codepath/jdbc_daily_load_tables.py file://$dataprochome/jdbc_daily_load_tables.py;
sh chmod +x $dataprochome/db_password.sh;
sh chmod +x $dataprochome/jdbc_daily_load_tables.py;
sh $dataprochome/db_password.sh $dataprochome $stg_gcs_bucket $se_stg_gcs_bucket $target_schema $target_table_stg_gm_add_attributes_orbit $target_table_orbit_delivery_partner_icc $kvenv;
All the input variable are defined properly still getting below error
2023-02-09 20:05:36,221 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. org.apache.pig.tools.parameters.ParseException: Encountered "<EOF>" at line 1, column 6.
Was expecting one of:
IDENTIFIER
OTHER
LITERAL
SHELLCMD
Details at logfile: /tmp/17114c5e-af3d-4a09-89c4-324250436a76/pig_1675973135671.log
2023-02-09 20:05:36,240 [main] INFO org.apache.pig.Main - Pig script completed in 712 milliseconds (712 ms)
As per comments in the answer provided by #OneCricketeer, the error was resolved by removing the spaces between the arguments passed to .sh file in the pig script.
That's a bash script. Not pig ... If you're running that in pig, then that perfectly explains why it's failing to parse that file.
You need to use sh rather than pig on shell scripts
Related
Script as -
/bin/bash
1 START_TIME=($date +%s)
echo "calling enviroment variable"
and when i tried to run tho above script i got
error as
calling enviroment variable
/bin/bash 22-1-2016 line 1 : command not found
You should start your script with a she-bang (source) therefore :
/bin/bash
should be
#!/bin/bash
I've configured Heroku's Scheduler to run a Symfony 2 command:
bash app/console myapp:send:confirmations --verbose
And set it to be run each 10 minutes.
But, in the logs I see those messages:
2015-09-10T13:01:25.313711+00:00 heroku[api]: Starting process with command `bash app/console myapp:send:confirmations` by scheduler#addons.heroku.com
2015-09-10T13:01:44.151426+00:00 heroku[scheduler.7629]: Starting process with command `bash app/console myapp:send:confirmations --verbose`
2015-09-10T13:01:44.811500+00:00 heroku[scheduler.7629]: State changed from starting to up
2015-09-10T13:01:45.565021+00:00 app[scheduler.7629]: app/console: line 2: ?php: No such file or directory
2015-09-10T13:01:45.565093+00:00 app[scheduler.7629]: app/console: line 19: unexpected EOF while looking for matching `''
2015-09-10T13:01:45.565096+00:00 app[scheduler.7629]: app/console: line 28: syntax error: unexpected end of file
2015-09-10T13:01:46.291606+00:00 heroku[scheduler.7629]: State changed from up to complete
2015-09-10T13:01:46.278800+00:00 heroku[scheduler.7629]: Process exited with status 2
Those are the three relevant that are confusing me:
2015-09-10T13:01:45.565021+00:00 app[scheduler.7629]: app/console: line 2: ?php: No such file or directory
2015-09-10T13:01:45.565093+00:00 app[scheduler.7629]: app/console: line 19: unexpected EOF while looking for matching `''
2015-09-10T13:01:45.565096+00:00 app[scheduler.7629]: app/console: line 28: syntax error: unexpected end of file
I'm a bit confused: the file app/console seems not exist, but then the script encounters an unexpected EOF (but the file doesn't exist o.O) and then an unexpected end of file (isn't this the same thing as the message immediatley before?
What am I doing wrong?
Use php instead of bash to launch the console:
php app/console myapp:send:confirmations --verbose
I have the same behaviour (crash) on Ubuntu 15.04:
$ bash app/console
app/console: line 2: ?php: No such file or directory
app/console: line 18: unexpected EOF while looking for matching `''
app/console: line 23: syntax error: unexpected end of file
$ php app/console -v
Symfony version 2.7.4 - app/prod
...
It seems that the shebang from the start of app/console is ignored and the PHP interpreter is not called:
#!/usr/bin/env php
<?php
....
Here are explanations from Aaron Copley:
It's not executable
Run the binary with absolute or relative path
So if you mark the file as executable and launch the script with relative path, the PHP interpreter will be called:
$ chmod +x app/console
$ ./app/console -v
Symfony version 2.7.4 - app/prod
I couldn't find existing posts related to my issue. On a Digital Ocean Droplet, mup setup went fine, but when I try to deploy, I get the following error. Any ideas? Thanks!
root#ts:~/ts-deploy# mup deploy
Meteor Up: Production Quality Meteor Deployments
Building Started: /root/TS/
Bundling Error: code=137, error:
-------------------STDOUT-------------------
Figuring out the best package versions to use. This may take a moment.
-------------------STDERR-------------------
bash: line 1: 31217 Killed meteor build --directory /tmp/dc37af3e-eca0-4a19-bf1a-d6d38bb8f517
Below are the logs. node -v indicates I am using 0.10.31. How do I check which script is exiting with the error? Any other ideas? Thanks!
error: Forever detected script exited with code: 1
error: Script restart attempt #106
Meteor requires Node v0.10.29 or later.
error: Forever detected script exited with code: 1
error: Script restart attempt #107
Meteor requires Node v0.10.29 or later.
error: Forever detected script exited with code: 1
error: Script restart attempt #108
stepping down to gid: meteoruser
stepping down to uid: meteoruser
After I went back to an old backup of the DO Droplet, and re-ran mup setup and mup deploy, I now get this in the command line output
Building Started: /root/TS
Bundling Error: code=134, error:
-------------------STDOUT-------------------
Figuring out the best package versions to use. This may take a moment.
-------------------STDERR-------------------
FATAL ERROR: JS Allocation failed - process out of memory
bash: line 1: 1724 Aborted (core dumped) meteor build --directory /tmp/bfdbcb45-9c61-435f-9875-3fb304358996
and this in the logs:
>> stepping down to gid: meteoruser
>> stepping down to uid: meteoruser
Exception while invoking method 'login' TypeError: Cannot read property '0' of undefined
at ServiceConfiguration.configurations.remove.service (app/server/accounts.js:7:26)
at Object.Accounts.insertUserDoc (packages/accounts-base/accounts_server.js:1024)
at Object.Accounts.updateOrCreateUserFromExternalService (packages/accounts-base/accounts_server.js:1189)
at Package (packages/accounts-oauth/oauth_server.js:45)
at packages/accounts-base/accounts_server.js:383
at tryLoginMethod (packages/accounts-base/accounts_server.js:186)
at runLoginHandlers (packages/accounts-base/accounts_server.js:380)
at Meteor.methods.login (packages/accounts-base/accounts_server.js:434)
at maybeAuditArgumentChecks (packages/ddp/livedata_server.js:1594)
at packages/ddp/livedata_server.js:648
Exception while invoking method 'login' TypeError: Cannot read property '0' of undefined
at ServiceConfiguration.configurations.remove.service (app/server/accounts.js:7:26)
at Object.Accounts.insertUserDoc (packages/accounts-base/accounts_server.js:1024)
at Object.Accounts.updateOrCreateUserFromExternalService (packages/accounts-base/accounts_server.js:1189)
at Package (packages/accounts-oauth/oauth_server.js:45)
at packages/accounts-base/accounts_server.js:383
at tryLoginMethod (packages/accounts-base/accounts_server.js:186)
at runLoginHandlers (packages/accounts-base/accounts_server.js:380)
at Meteor.methods.login (packages/accounts-base/accounts_server.js:434)
at maybeAuditArgumentChecks (packages/ddp/livedata_server.js:1594)
at packages/ddp/livedata_server.js:648
The memory issue stems from using DigitalOcean's $5 Droplet. To solve the problem, I added swap to the server, as explained in detail below.
Create and enable the swap file using the dd command :
sudo dd if=/dev/zero of=/swapfile bs=1024 count=256k
“of=/swapfile” designates the file’s name. In this case the name is swapfile.
Next prepare the swap file by creating a linux swap area:
sudo mkswap /swapfile
The results display:
Setting up swapspace version 1, size = 262140 KiB
no label, UUID=103c4545-5fc5-47f3-a8b3-dfbdb64fd7eb
Finish up by activating the swap file:
sudo swapon /swapfile
You will then be able to see the new swap file when you view the swap summary.
swapon -s
Filename Type Size Used Priority
/swapfile file 262140 0 -1
This file will last on the virtual private server until the machine reboots. You can ensure that the swap is permanent by adding it to the fstab file.
Open up the file:
sudo nano /etc/fstab
Paste in the following line:
/swapfile none swap sw 0 0
Swappiness in the file should be set to 10. Skipping this step may cause both poor performance, whereas setting it to 10 will cause swap to act as an emergency buffer, preventing out-of-memory crashes.
You can do this with the following commands:
echo 10 | sudo tee /proc/sys/vm/swappiness
echo vm.swappiness = 10 | sudo tee -a /etc/sysctl.conf
To prevent the file from being world-readable, you should set up the correct permissions on the swap file:
sudo chown root:root /swapfile
sudo chmod 0600 /swapfile
This only worked for me by increasing the swap space to 1gb:
Make all swap off
sudo swapoff -a
Resize the swapfile
sudo dd if=/dev/zero of=/swapfile bs=1M count=1024
Make swapfile usable
sudo mkswap /swapfile
Make swapon again
sudo swapon /swapfile
I'm trying to figure out how to install Pear on my Mac (10.6.6).
Not understanding what they're telling me at pear.php.net, I got some code from http://clickontyler.com/blog/2008/01/how-to-install-pear-in-mac-os-x-leopard/
First, I entered curl http://pear.php.net/go-pear > go-pear.php in my terminal.
It resulted in this output
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 88004 100 88004 0 0 47537 0 0:00:01 0:00:01 --:--:-- 59744
What does that all mean? Am I on the right track?
Next, I entered sudo php -q go-pear.php
and it gave me the long output below. In short I have no idea where I am in the installation process. However, I'm pretty sure that I'm not where I'm supposed to be at following the tutorial at http://clickontyler.com/blog/2008/01/how-to-install-pear-in-mac-os-x-leopard/
because the tutorial tells me to select all the default choices, and I don't see any options to select.
The next line of code is asking me to modify the php.ini files and it requires a password so I'm worried about doing it...Can anyone tell me if I'm on the right track?
sudo cp /etc/php.ini.default /etc/php.ini
Usage: php [options] [-f] <file> [--] [args...]
php [options] -r <code> [--] [args...]
php [options] [-B <begin_code>] -R <code> [-E <end_code>] [--] [args...]
php [options] [-B <begin_code>] -F <file> [-E <end_code>] [--] [args...]
php [options] -- [args...]
php [options] -a
-a Run interactively
-c <path>|<file> Look for php.ini file in this directory
-n No php.ini file will be used
-d foo[=bar] Define INI entry foo with value 'bar'
-e Generate extended information for debugger/profiler
-f <file> Parse and execute <file>.
-h This help
-i PHP information
-l Syntax check only (lint)
-m Show compiled in modules
-r <code> Run PHP <code> without using script tags <?..?>
-B <begin_code> Run PHP <begin_code> before processing input lines
-R <code> Run PHP <code> for every input line
-F <file> Parse and execute <file> for every input line
-E <end_code> Run PHP <end_code> after processing all input lines
-H Hide any passed arguments from external tools.
-s Output HTML syntax highlighted source.
-v Version number
-w Output source with stripped comments and whitespace.
-z <file> Load Zend extension <file>.
args... Arguments passed to script. Use -- args when first argument
starts with - or script is read from stdin
--ini Show configuration file names
--rf <name> Show information about function <name>.
--rc <name> Show information about class <name>.
--re <name> Show information about extension <name>.
--ri <name> Show configuration for extension <name>.
php does not have an argument -q. Its also mentioned in go-pear.php (http://pear.php.net/go-pear) itself, but I dont know, what it wants to tell me. However, try
sudo php go-pear.php
and then follow the instructions.
Update:
-q was used, to start the interpreter in "quiet" mode. It seems, that this option does not exists anymore, because php always starts "quiet", but it should not cause an error, anyway. Now make sure you are in the same directory as the file go-pear.php before you call php go-pear.php.
The first part shows that you successfully downloaded the file to go-pear.php.
The second part is showing that -q isn't a valid option. The third part is asking for the root password, since you're doing 'sudo'.
I used this, though I wasn't installing on Mac:
Getting and installing the PEAR package manager
I have a script that looks like this
#!/bin/bash
function something() {
echo "hello world!!"
}
something | tee logfile
I have set the execute permission on this file and when I try running the file like this
$./script.sh
it runs perfectly fine, but when I run it on the command line like this
$sh script.sh
It throws up an error. Why does this happen and what are the ways in which I can fix this.
Running it as ./script.sh will make the kernel read the first line (the shebang), and then invoke bash to interpret the script. Running it as sh script.sh uses whatever shell your system defaults sh to (on Ubuntu this is Dash, which is sh-compatible, but doesn't support some of the extra features of Bash).
You can fix it by invoking it as bash script.sh, or if it's your machine you can change /bin/sh to be bash and not whatever it is currently (usually just by symlinking it - rm /bin/sh && ln -s /bin/bash /bin/sh). Or you can just use ./script.sh instead if that's already working ;)
If your shell is indeed dash and you want to modify the script to be compatible, https://wiki.ubuntu.com/DashAsBinSh has a helpful guide to the differences. In your sample it looks like you'd just have to remove the function keyword.
if your script is at your present working directory and you issue ./script.sh, the kernel will read the shebang (first line) and execute the shell interpreter that is defined. you can also call your script.sh by specifying the path of the interpreter eg
/bin/bash myscript.sh
/bin/sh myscript.sh
/bin/ksh myscript.sh etc
By the way, you can also put your shebang like this (if you don't want to specify full path)
#!/usr/bin/env sh
sh script.sh forces the script to be executed within the sh - shell.
while simply starting it from command line uses the shell-environemnt you're in.
Please post the error message for further answers.
Random though on what the error may be:
path specified in first line /bin/bash is wrong -- maybe bash is not installed?