Hadoop 2.6 start-dfs.sh errors on Centos 6.7 - centos6

I use this tutorial to install Hadoop 2.6 on Centos 6.7 with Java 1.8.0_72 and everything goes well before execution of start-dfs.sh from Hadoop-home/sbin/srart-dfs.sh. Below is the output:
[hadoop#10 sbin]$ start-dfs.sh
16/02/26 21:47:40 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [localhost]
localhost: /etc/bashrc: line 65: id: command not found
localhost: /etc/bashrc: line 65: id: command not found
localhost: /usr/bin/env: bash: No such file or directory
localhost: /etc/bashrc: line 65: id: command not found
localhost: /etc/bashrc: line 65: id: command not found
localhost: /usr/bin/env: bash: No such file or directory
Starting secondary namenodes [0.0.0.0]
0.0.0.0: /etc/bashrc: line 65: id: command not found
0.0.0.0: /etc/bashrc: line 65: id: command not found
0.0.0.0: /usr/bin/env: bash: No such file or directory
16/02/26 21:47:46 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
It seems there is something wrong with /etv/bashrc # line 65. But, I checked and there is nothing I modified.
I run CentOS 6.7 final releas using Parallel VM manager of my Mac which is a 64-bit machine.
Thanks in advance

Edit your core-site.xml and add this part:
<property>
<name>hadoop.tmp.dir</name>
<value>/app/hadoop/tmp</value>
<description>A base for other temporary directories.</description>
</property>
Then create the folder accordingly, example command:
mkdir -p /app/hadoop/tmp
chown yourHadoopUsername:yourHadoopGroupName /app/hadoop/tmp
chmod 777 /app/hadoop/tmp
Format your datanode:
hdfs namenode -format
Start your hadoop:
start-dfs.sh
start-yarn.sh

Related

Command failed when running ghost start

Fresh install, trying to run ghost start I get the following error:
Debug Information:
OS: Raspbian, v8.0
Node Version: v6.13.0
Ghost-CLI Version: 1.5.2
Environment: production
Command: 'ghost start'
An error occurred.
Message: 'Command failed: /bin/sh -c systemctl is-active ghost_blog-dev
unknown
'
Stack: Error: Command failed: /bin/sh -c systemctl is-active ghost_blog-dev
unknown
at makeError (/usr/lib/node_modules/ghost-cli/node_modules/execa/index.js:169:9)
at module.exports.sync (/usr/lib/node_modules/ghost-cli/node_modules/execa/index.js:338:15)
at handleShell (/usr/lib/node_modules/ghost-cli/node_modules/execa/index.js:116:9)
at Function.module.exports.shellSync (/usr/lib/node_modules/ghost-cli/node_modules/execa/index.js:361:43)
at SystemdProcessManager.isRunning (/usr/lib/node_modules/ghost-cli/extensions/systemd/systemd.js:88:19)
at Instance.running (/usr/lib/node_modules/ghost-cli/lib/instance.js:120:34)
at StartCommand.run (/usr/lib/node_modules/ghost-cli/lib/commands/start.js:28:22)
at precheck.then (/usr/lib/node_modules/ghost-cli/lib/command.js:159:52)
at process._tickCallback (internal/process/next_tick.js:109:7)
at Module.runMain (module.js:613:11)
at run (bootstrap_node.js:387:7)
at startup (bootstrap_node.js:153:9)
at bootstrap_node.js:500:3
Code: 3
If I manually run the command that is says failed it seems to execute without error though I am not sure what it does. I assume it has something to do with checking with nginx if ghost has actually started or not.
Any suggestions would be very helpful! Thank you!
I faced with the same problem. You should add your service file to etc directory also as symbolic link using the following command:
sudo ln -sf /var/www/html/your-blog/system/files/ghost_blog-yourblog.service /etc/systemd/system/ghost_blog-yourblog.service
After adding this, you should execute the following commands:
sudo systemctl stop ghost_blog-yourblog.service
sudo systemctl start ghost_blog-yourblog.service
Then, I hope you'll see 'active' result for is-active command.

KnpSnappyBundle - wkhtmltopdf.exe.bat: Permission denied

In dev environment (windows) using xampp my pdf files are generated successfully, but in prod (linux):
string(545) "The exit status code '126' says something went wrong:
stderr: "sh: /home/trainingexperience/beta.trainingexperience.org/app/../bin/wkhtmltopdf.exe.bat: Permission denied"
stdout: ""
command: /home/trainingexperience/beta.trainingexperience.org/app/../bin/wkhtmltopdf.exe.bat --lowquality --encoding 'utf-8' '/home/trainingexperience/beta.trainingexperience.org/app/cache/prod/snappy/knp_snappy5936b3644fdf62.71499624.html' '/home/trainingexperience/beta.trainingexperience.org/app/../web/uploads/company/payments/test_cXrfA4L99zsiKi6Cfl5a.pdf'."
My config:
knp_snappy:
pdf:
binary: %kernel.root_dir%/../bin/wkhtmltopdf.exe.bat
temporary_folder: %kernel.cache_dir%/snappy
Could it be because of Linux? What is the problem?
UPDATE
I tried:
knp_snappy:
pdf:
binary: %kernel.root_dir%/../bin/wkhtmltopdf
temporary_folder: %kernel.cache_dir%/snappy
I got:
No such file or directory
I think could be the problem (composer.json): "wemersonjanuario/wkhtmltopdf-windows": "0.12.2.3". Is there a version to that could work on windows and linux at the same time?
You should define a shell executable for your production environement and your developement environment.
for linux
binary: /usr/local/bin/wkhtmltopdf
for windows
binary: "C:\\Program Files\\wkhtmltopdf\\bin\\wkhtmltopdf.exe\\"
as you can see, You can't sh (execute) a .bat file on linux.

How to run a Symfony command from Heroku's Scheduler

I've configured Heroku's Scheduler to run a Symfony 2 command:
bash app/console myapp:send:confirmations --verbose
And set it to be run each 10 minutes.
But, in the logs I see those messages:
2015-09-10T13:01:25.313711+00:00 heroku[api]: Starting process with command `bash app/console myapp:send:confirmations` by scheduler#addons.heroku.com
2015-09-10T13:01:44.151426+00:00 heroku[scheduler.7629]: Starting process with command `bash app/console myapp:send:confirmations --verbose`
2015-09-10T13:01:44.811500+00:00 heroku[scheduler.7629]: State changed from starting to up
2015-09-10T13:01:45.565021+00:00 app[scheduler.7629]: app/console: line 2: ?php: No such file or directory
2015-09-10T13:01:45.565093+00:00 app[scheduler.7629]: app/console: line 19: unexpected EOF while looking for matching `''
2015-09-10T13:01:45.565096+00:00 app[scheduler.7629]: app/console: line 28: syntax error: unexpected end of file
2015-09-10T13:01:46.291606+00:00 heroku[scheduler.7629]: State changed from up to complete
2015-09-10T13:01:46.278800+00:00 heroku[scheduler.7629]: Process exited with status 2
Those are the three relevant that are confusing me:
2015-09-10T13:01:45.565021+00:00 app[scheduler.7629]: app/console: line 2: ?php: No such file or directory
2015-09-10T13:01:45.565093+00:00 app[scheduler.7629]: app/console: line 19: unexpected EOF while looking for matching `''
2015-09-10T13:01:45.565096+00:00 app[scheduler.7629]: app/console: line 28: syntax error: unexpected end of file
I'm a bit confused: the file app/console seems not exist, but then the script encounters an unexpected EOF (but the file doesn't exist o.O) and then an unexpected end of file (isn't this the same thing as the message immediatley before?
What am I doing wrong?
Use php instead of bash to launch the console:
php app/console myapp:send:confirmations --verbose
I have the same behaviour (crash) on Ubuntu 15.04:
$ bash app/console
app/console: line 2: ?php: No such file or directory
app/console: line 18: unexpected EOF while looking for matching `''
app/console: line 23: syntax error: unexpected end of file
$ php app/console -v
Symfony version 2.7.4 - app/prod
...
It seems that the shebang from the start of app/console is ignored and the PHP interpreter is not called:
#!/usr/bin/env php
<?php
....
Here are explanations from Aaron Copley:
It's not executable
Run the binary with absolute or relative path
So if you mark the file as executable and launch the script with relative path, the PHP interpreter will be called:
$ chmod +x app/console
$ ./app/console -v
Symfony version 2.7.4 - app/prod

Memory issue with meteor up (mup) on Digital Ocean

I couldn't find existing posts related to my issue. On a Digital Ocean Droplet, mup setup went fine, but when I try to deploy, I get the following error. Any ideas? Thanks!
root#ts:~/ts-deploy# mup deploy
Meteor Up: Production Quality Meteor Deployments
Building Started: /root/TS/
Bundling Error: code=137, error:
-------------------STDOUT-------------------
Figuring out the best package versions to use. This may take a moment.
-------------------STDERR-------------------
bash: line 1: 31217 Killed meteor build --directory /tmp/dc37af3e-eca0-4a19-bf1a-d6d38bb8f517
Below are the logs. node -v indicates I am using 0.10.31. How do I check which script is exiting with the error? Any other ideas? Thanks!
error: Forever detected script exited with code: 1
error: Script restart attempt #106
Meteor requires Node v0.10.29 or later.
error: Forever detected script exited with code: 1
error: Script restart attempt #107
Meteor requires Node v0.10.29 or later.
error: Forever detected script exited with code: 1
error: Script restart attempt #108
stepping down to gid: meteoruser
stepping down to uid: meteoruser
After I went back to an old backup of the DO Droplet, and re-ran mup setup and mup deploy, I now get this in the command line output
Building Started: /root/TS
Bundling Error: code=134, error:
-------------------STDOUT-------------------
Figuring out the best package versions to use. This may take a moment.
-------------------STDERR-------------------
FATAL ERROR: JS Allocation failed - process out of memory
bash: line 1: 1724 Aborted (core dumped) meteor build --directory /tmp/bfdbcb45-9c61-435f-9875-3fb304358996
and this in the logs:
>> stepping down to gid: meteoruser
>> stepping down to uid: meteoruser
Exception while invoking method 'login' TypeError: Cannot read property '0' of undefined
at ServiceConfiguration.configurations.remove.service (app/server/accounts.js:7:26)
at Object.Accounts.insertUserDoc (packages/accounts-base/accounts_server.js:1024)
at Object.Accounts.updateOrCreateUserFromExternalService (packages/accounts-base/accounts_server.js:1189)
at Package (packages/accounts-oauth/oauth_server.js:45)
at packages/accounts-base/accounts_server.js:383
at tryLoginMethod (packages/accounts-base/accounts_server.js:186)
at runLoginHandlers (packages/accounts-base/accounts_server.js:380)
at Meteor.methods.login (packages/accounts-base/accounts_server.js:434)
at maybeAuditArgumentChecks (packages/ddp/livedata_server.js:1594)
at packages/ddp/livedata_server.js:648
Exception while invoking method 'login' TypeError: Cannot read property '0' of undefined
at ServiceConfiguration.configurations.remove.service (app/server/accounts.js:7:26)
at Object.Accounts.insertUserDoc (packages/accounts-base/accounts_server.js:1024)
at Object.Accounts.updateOrCreateUserFromExternalService (packages/accounts-base/accounts_server.js:1189)
at Package (packages/accounts-oauth/oauth_server.js:45)
at packages/accounts-base/accounts_server.js:383
at tryLoginMethod (packages/accounts-base/accounts_server.js:186)
at runLoginHandlers (packages/accounts-base/accounts_server.js:380)
at Meteor.methods.login (packages/accounts-base/accounts_server.js:434)
at maybeAuditArgumentChecks (packages/ddp/livedata_server.js:1594)
at packages/ddp/livedata_server.js:648
The memory issue stems from using DigitalOcean's $5 Droplet. To solve the problem, I added swap to the server, as explained in detail below.
Create and enable the swap file using the dd command :
sudo dd if=/dev/zero of=/swapfile bs=1024 count=256k
“of=/swapfile” designates the file’s name. In this case the name is swapfile.
Next prepare the swap file by creating a linux swap area:
sudo mkswap /swapfile
The results display:
Setting up swapspace version 1, size = 262140 KiB
no label, UUID=103c4545-5fc5-47f3-a8b3-dfbdb64fd7eb
Finish up by activating the swap file:
sudo swapon /swapfile
You will then be able to see the new swap file when you view the swap summary.
swapon -s
Filename Type Size Used Priority
/swapfile file 262140 0 -1
This file will last on the virtual private server until the machine reboots. You can ensure that the swap is permanent by adding it to the fstab file.
Open up the file:
sudo nano /etc/fstab
Paste in the following line:
/swapfile none swap sw 0 0
Swappiness in the file should be set to 10. Skipping this step may cause both poor performance, whereas setting it to 10 will cause swap to act as an emergency buffer, preventing out-of-memory crashes.
You can do this with the following commands:
echo 10 | sudo tee /proc/sys/vm/swappiness
echo vm.swappiness = 10 | sudo tee -a /etc/sysctl.conf
To prevent the file from being world-readable, you should set up the correct permissions on the swap file:
sudo chown root:root /swapfile
sudo chmod 0600 /swapfile
This only worked for me by increasing the swap space to 1gb:
Make all swap off
sudo swapoff -a
Resize the swapfile
sudo dd if=/dev/zero of=/swapfile bs=1M count=1024
Make swapfile usable
sudo mkswap /swapfile
Make swapon again
sudo swapon /swapfile

Installing supervisord with erpnext

I am a newbie to centos commands and scripting.
Scenario:
I have installed ERPNEXT and is working okay on the server but I have to do it manually by doing ./lib/wnfy.py --serve
What I want to do is to start it automatically using supervisord. I have gunicorn and nginx installed. Upon trying to start supervisord here's the error:
[root#vps3 etc]# chkconfig supervisord on [root#vps3 etc]# service
supervisord start /etc/init.d/supervisord: line 11:
./etc/rc.d/init.d/functionsprog=supervisord: No such file or directory
Starting : daemon --pidfile [ -f ]/etc/init.d/supervisord: line 14:
success: command not found /etc/init.d/supervisord: line 14: failure:
command not found
I am not yet familiar with how things work. Please advise. Thanks in advance.
Have you installed supervisor? Follow,
Install supervisor
Initscripts (use jkoppe ones).
Also for production setup, use gunicorn. Supervisor config for it is,
[program:gunicorn]
command=gunicorn -b 127.0.0.1:8000 -w 2 -t 120 lib.webnotes.app:application
directory=/path/to/erpnext
user=erpnext
process_name=%(program_name)s
autostart=True
autorestart=True
redirect_stderr=True

Resources