When trying to start riak server, it failed to start, by running
riak console
it shows
Erlang R16B02_basho6 (erts-5.10.3) [source-bcd8abb] [64-bit] [smp:4:4]
[async-threads:64] [kernel-poll:true] [frame-pointer]
[os_mon] memory supervisor port (memsup): Erlang has closed [os_mon]
cpu supervisor port (cpu_sup): Erlang has closed {"Kernel pid
terminated",application_controller,"{application_start_failure,riak_api,{{shutdown,{failed_to_start_child,\"pb://\"lab-vault.aspirapps.com\":8087\",{bad_return_value,{error,einval}}}},{riak_api_app,start,[normal,[]]}}}"}
here is the riak.config (partial)
##
## Acceptable values:
## - an IP/port pair, e.g. 127.0.0.1:10011
listener.http.internal = lab-vault.aspirapps.com:8098
## listener.protobuf.<name> is an IP address and TCP port that the Riak
## Protocol Buffers interface will bind.
##
## Default: 127.0.0.1:8087
##
## Acceptable values:
## - an IP/port pair, e.g. 127.0.0.1:10011
listener.protobuf.internal = lab-vault.aspirapps.com:8087
## The maximum length to which the queue of pending connections
## may grow. If set, it must be an integer > 0. If you anticipate a
## huge number of connections being initialized *simultaneously*, set
## this number higher.
##
## Default: 128
##
## Acceptable values:
## - an integer
## protobuf.backlog = 128
## listener.https.<name> is an IP address and TCP port that the Riak
## HTTPS interface will bind.
##
## Acceptable values:
## - an IP/port pair, e.g. 127.0.0.1:10011
## listener.https.internal = lab-vault.aspirapps.com:8088
tried to replace url names with IP but no luck, it appears same error message.
How do I get riak up and running? Thanks.
Just get it worked.
Thanks Joe and Vempo for help to indicate possible solutions.
I have to use IP address to bind the listening IP and Port.
The additional step after revise the riak.conf file are removing previous generated configuration files
rm -f /var/lib/riak/ring/*
rm -f /var/lib/riak/generated.configs/*
thanks you all.
Related
I have been struggling to get a Multi Node H2O cluster up and running using AWS EC2 instances.
I have followed the advice from this thread, but still struggle with the nodes not seeing each other. The EC2 instances all use the same AMI that I have pre-built, so the same h2o.jar file is on all of them,
I have also tried the following troubleshooting advice:
Name cluster -name
Rather use -network flag
Open port 54321 on security group as 0.0.0.0
Here are my steps:
1) Start AWS EC2 in same availability zone and get private IPs and network cidr (172.31.0.0/20). Put ip addresses into flatfile.txt
172.31.8.210:54321
172.31.9.207:54321
172.31.13.136:54321
2) Copy the flatfile.txt to all servers to which I want to connect as nodes and start H2O
# cluster_run
library(h2oEnsemble)
library(ssh)
ips <- gsub("(.*):.*", "\\1", readLines("flatfile.txt"))
start_cluster <- function(ip){
# Copy flatfile across
session <- ssh_connect(paste0("ubuntu#", ip), keyfile = "mykey.pem")
scp_upload(session, "flatfile.txt")
# Ensure no h2o instance is already running
out <- ssh_exec_wait(session, "sudo pkill java")
# Start H2O cluster
cmd <- gsub("\\s+", " ", paste0("ssh -i mykey.pem -o 'StrictHostKeyChecking no' ubuntu#", ip,
" 'java -Xmx20g
-jar /home/rstudio/R/x86_64-pc-linux-gnu-library/3.5/h2o/java/h2o.jar
-name mycluster
-network 172.31.0.0/20
-flatfile flatfile.txt
-port 54321 &'"))
system(cmd, wait = FALSE)
}
start_cluster(ips[3])
start_cluster(ips[2])
start_cluster(ips[1])
3) Once this has been done, I now want to connect R to my new Multi Node cluster
h2o.init(startH2O = F)
h2o.shutdown(prompt = FALSE)
This is where I see that the nodes aren't being picked up:
I have also seen that when I start the H2O cluster on the different nodes, it isnt picking up the other machines within the network:
You need to add port 54321+1 (so 54322) to the security group, as well.
The internal communication goes through 54322.
(I would also specify /16 for -network because it’s easier for other people to understand. For example, even if you are sure /20 is technically correct for your network setup, I can’t easily be sure. :-)
Depending on the actual network setup, you probably don’t need -network flag at all. Your instances probably only have one interface.
I am trying to configure a Zookeeper cLuster with three node on 3 different machine
this is the zoo.cfg file I'm using on the three nodes
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
#dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
dataDir=/home/user/HA/zookeeper_data
dataLogDir=/home/user/HA/zookeeper_log
Server.1=Server1:2888:3888
Server.2=Server2:2888:3888
Server.3=Server3:2888:3888
when I start zk Server on three nodes with
zkServer.sh start
the three server work all in standalone mode
what is wrong with this configuration ? and is there another way to start the cluster ?
thnak you
After one hour of search I figured out that the problem was in the config file "zoo.cfg"
exactly in the list of nodes and ips :
Wrong configuration
Server.1=Server1:2888:3888
Server.2=Server2:2888:3888
Server.3=Server3:2888:3888
Correct configuration
server.1=Server1:2888:3888
server.2=Server2:2888:3888
server.3=Server3:2888:3888
the issue is that the first S in "Server..." was in uppercase .
somtimes the solution is in the details !
thank you
I'm having a hard time configuring an Openstack environment based on the All-In-One Single Machine installer for bridged networking in my LAN.
My objective is to SSH into the instances created in Openstack from my LAN.
The server is an Ubuntu 16.04 LTS with minimal installation and OpenSSH. The network configuration of the server is:
auto enp3s0
iface enp3s0 inet static
address 10.4.4.1
netmask 255.255.255.0
gateway 10.4.4.254
broadcast 10.4.4.255
network 10.4.4.0
dns-nameservers 10.4.1.12 10.4.1.10
Basically my network details are the following:
LAN 10.4.4.0
MASK 255.255.255.0
Gateway/DHCP Server 10.4.4.254
The local.conf file I've used for deploying the devstack is the following:
# Sample ``local.conf`` for user-configurable variables in ``stack.sh``
# NOTE: Copy this file to the root DevStack directory for it to work properly.
# ``local.conf`` is a user-maintained settings file that is sourced from ``stackrc``.
# This gives it the ability to override any variables set in ``stackrc``.
# Also, most of the settings in ``stack.sh`` are written to only be set if no
# value has already been set; this lets ``local.conf`` effectively override the
# default values.
# This is a collection of some of the settings we have found to be useful
# in our DevStack development environments. Additional settings are described
# in https://docs.openstack.org/devstack/latest/configuration.html#local-conf
# These should be considered as samples and are unsupported DevStack code.
# The ``localrc`` section replaces the old ``localrc`` configuration file.
# Note that if ``localrc`` is present it will be used in favor of this section.
[[local|localrc]]
# Minimal Contents
# ----------------
# While ``stack.sh`` is happy to run without ``localrc``, devlife is better when
# there are a few minimal variables set:
# If the ``*_PASSWORD`` variables are not set here you will be prompted to enter
# values for them by ``stack.sh``and they will be added to ``local.conf``.
FLOATING_RANGE=10.4.4.192/27
FIXED_RANGE=192.168.0.0/24
FIXED_NETWORK_SIZE=256
FLAT_INTERFACE=enp3s0
ADMIN_PASSWORD=nomoresecret
DATABASE_PASSWORD=stackdb
RABBIT_PASSWORD=stackqueue
SERVICE_PASSWORD=$ADMIN_PASSWORD
# ``HOST_IP`` and ``HOST_IPV6`` should be set manually for best results if
# the NIC configuration of the host is unusual, i.e. ``eth1`` has the default
# route but ``eth0`` is the public interface. They are auto-detected in
# ``stack.sh`` but often is indeterminate on later runs due to the IP moving
# from an Ethernet interface to a bridge on the host. Setting it here also
# makes it available for ``openrc`` to include when setting ``OS_AUTH_URL``.
# Neither is set by default.
HOST_IP=10.4.4.1
#HOST_IPV6=2001:db8::7
# Logging
# -------
# By default ``stack.sh`` output only goes to the terminal where it runs. It can
# be configured to additionally log to a file by setting ``LOGFILE`` to the full
# path of the destination log file. A timestamp will be appended to the given name.
LOGFILE=$DEST/logs/stack.sh.log
# Old log files are automatically removed after 7 days to keep things neat. Change
# the number of days by setting ``LOGDAYS``.
LOGDAYS=2
# Nova logs will be colorized if ``SYSLOG`` is not set; turn this off by setting
# ``LOG_COLOR`` false.
#LOG_COLOR=False
# Using milestone-proposed branches
# ---------------------------------
# Uncomment these to grab the milestone-proposed branches from the
# repos:
#CINDER_BRANCH=milestone-proposed
#GLANCE_BRANCH=milestone-proposed
#HORIZON_BRANCH=milestone-proposed
#KEYSTONE_BRANCH=milestone-proposed
#KEYSTONECLIENT_BRANCH=milestone-proposed
#NOVA_BRANCH=milestone-proposed
#NOVACLIENT_BRANCH=milestone-proposed
#NEUTRON_BRANCH=milestone-proposed
#SWIFT_BRANCH=milestone-proposed
# Using git versions of clients
# -----------------------------
# By default clients are installed from pip. See LIBS_FROM_GIT in
# stackrc for details on getting clients from specific branches or
# revisions. e.g.
# LIBS_FROM_GIT="python-ironicclient"
# IRONICCLIENT_BRANCH=refs/changes/44/2.../1
# Swift
# -----
# Swift is now used as the back-end for the S3-like object store. Setting the
# hash value is required and you will be prompted for it if Swift is enabled
# so just set it to something already:
SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5
# For development purposes the default of 3 replicas is usually not required.
# Set this to 1 to save some resources:
SWIFT_REPLICAS=1
# The data for Swift is stored by default in (``$DEST/data/swift``),
# or (``$DATA_DIR/swift``) if ``DATA_DIR`` has been set, and can be
# moved by setting ``SWIFT_DATA_DIR``. The directory will be created
# if it does not exist.
SWIFT_DATA_DIR=$DEST/data
At the end of the deployment I'm able to ping from the instance to my LAN and do nslookup on google.com for example, but I can't do it backwards, ping/ssh/telnet the instance in Openstack.
The security group permits all traffic, all ICMP ingress/egress, SSH from everywhere.
I've tried to telnet on my local computer from the Openstack instance and it's showing the IP of the Openstack host, not the host. So I'm missing something in the network topology.
netstat -ant | grep 1716
tcp6 0 0 :::1716 :::* LISTEN
tcp6 0 0 10.4.3.34:1716 10.4.4.1:42992 ESTABLISHED
Is there any type of network deployment I'm missing?
Any advice would be much appreciated!
If you are trying to access your instances from the "outside", you will need to create a floating IP pool and assign a floating IP to one of your instances.
I need to ingest data from remote server using flume to hdfs::
I have used source as syslogtcp. My flume.conf file is as:
Agent.sources = syslog
Agent.channels = MemChannel
Agent.sinks = HDFS
Agent.sources.syslog.type = syslogtcp
Agent.sources.syslog.channels = MemChannel
Agent.sources.syslog.port = 5140
Agent.sources.syslog.host = localhost
Agent.sinks.HDFS.channel = MemChannel
Agent.sinks.HDFS.type = hdfs
Agent.sinks.HDFS.hdfs.path = hdfs://192.168.111.130:8022/user/cloudera/Twitter/apple_data/%y/%m/%d/
Agent.sinks.HDFS.hdfs.fileType = DataStream
Agent.sinks.HDFS.hdfs.writeFormat = Text
Agent.sinks.HDFS.hdfs.batchSize = 1000
Agent.sinks.HDFS.hdfs.rollSize = 0
Agent.sinks.HDFS.hdfs.rollCount = 10000
Agent.channels.MemChannel.type = memory
Agent.channels.MemChannel.capacity = 10000
Agent.channels.MemChannel.transactionCapacity = 100
I have a file in /etc/rsyslog.d/B2B.conf
# rsyslog v5 configuration file
# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html
#### MODULES ####
#$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
#$ModLoad imklog # provides kernel logging support (previously done by rklogd)
$ModLoad imfile
$InputFileName /home/cloudera/Desktop/my_logs/my_log.txt
$InputFileTag tag1:
$InputFileStateFile stat-file1
$InputFileFacility local7
$InputRunFileMonitor
$InputFilePollingInterval 10
#$ModLoad immark # provides --MARK-- message capability
# Provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514
# Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514
#### GLOBAL DIRECTIVES ####
# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
#### RULES ####
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
$WorkDirectory /var/lib/rsyslog # where to place spool files
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList # run asynchronously
$ActionResumeRetryCount -1 # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
*.* ##192.168.111.130:514
# ### end of the forwarding rule ###
Now when I run the java application to create log, flume and rsyslog:
It halts at
Shutting down system logger:
Starting system logger:
I am facing a strange problem on my Ubuntu Karmic system.
When I call getaddrinfo() with AI_PASSIVE and AF_UNSPEC, for an empty host and the UDP 12000 port to get a bindable address, I only get back one IPv4 result (0.0.0.0:12000 for instance).
If I change my call and specify AF_INET6 instead of AF_UNSPEC, then getaddrinfo() returns "Name or service not known".
Shouldn't I get [::]:12000 as a result ?
The same thing happens if I set the host to ::1.
When I call getaddrinfo() without AI_PASSIVE (to get a "connectable" address) for the host "localhost" and the UDP 12000 port, I first get [::1]:12000 then 127.0.0.1:12000.
So apparently, my system is IPv6 ready (I can ping to both IPv4 and IPv6 addresses, as well as DNS resolution). But how is it that I can't get an IPv6 address to bind to with getaddrinfo() ?
Do you guys have any idea about what could be wrong ?
My OS is Ubuntu Karmic, fresh install without any networking tweaking.
Thank you.
P.S: If you have no idea but still want to help me, you can get this sample program or type:
wget http://people.apache.org/~jorton/gai.c
And give me the result of:
$ ./gai -ap null 12000
My result is:
$ ./gai -ap null 12000
getaddrinfo(NULL, "12000", {.family=AF_UNSPEC, .hints=0|AI_ADDRCONFIG|AI_PASSIVE}) = 0:
family= 2, proto= 6 inet4: addr=0.0.0.0, port=12000
There you can see that I only have one IPv4 result.
This happens on new systems that use eglibc: debian-glibc.
Apparently, there is a bug that requires you to set at least one valid IPv6 address to one of your network interfaces (the loopback doesn't count).
After I did this:
$ sudo ip -6 addr add 2001:660:4701:1001::1 dev eth0
I have:
$ ./gai -ap null 12000
getaddrinfo(NULL, "12000", {.family=AF_UNSPEC, .hints=0|AI_ADDRCONFIG|AI_PASSIVE}) = 0:
family= 2, proto= 6 inet4: addr=0.0.0.0, port=12000
family=10, proto= 6 inet6: addr=::, port=12000, flowinfo=0
I hope this can help someone.