IP Address:
Web Application -> 192.168.x.209
HAProxy -> 192.168.x.211
Logstash Shipper -> 192.168.x.210
With the below configuration, HAProxy is not able to receive logs from Logstash application and thus logstash shipper is not able to receive the logs.
Following are the configurations I did:
WebApplication - logback.xml
<appender name="stash"
class="net.logstash.logback.appender.LogstashAccessTcpSocketAppender">
<destination>192.168.x.211:5001</destination>
<encoder class="net.logstash.logback.encoder.LogstashEncoder" />
</appender>
HAProxy Config
listen logstash
bind :5001
mode tcp
balance leastconn
option tcplog
server logstash01 192.168.x.212:5001 check
server logstash02 192.168.x.210:5001 check
Logstash Shipper - config
input{
tcp{
type => "log1"
port => 5001
codec => "json"
}
tcp{
type => "log2"
port => 5002
codec => "json"
}
output {
stdout{
codec => rubydebug
}}
Is there anything wrong with the configuration? help required.
Found the issue.
This was because firewall rules were active. After shutting it down, things worked as expected.
service iptables stop
setenv 0
Related
I'm trying to run Mercure on my Raspbian.
First :
I tried with mercure-legacy_0.13.0_Linux_armv6.tar.gz using the following command to run mercure
JWT_KEY='example'; ADDR='localhost:3000'; DEMO='1'; ALLOW_ANO NYMOUS='1'; CORS_ALLOWED_ORIGINS='*'; PUBLISH_ALLOWED_ORIGINS='*'; PUBLISHER_JWT_KEY='example' ./mercure run
It returns :
"msg":"Unexpected error","error":"listen tcp :80: bind: permission denied"
Second : I tried with mercure_0.13.0_Linux_armv6.tar.gz using the following command to run Mercure
MERCURE_PUBLISHER_JWT_KEY='!ChangeMe!' MERCURE_SUBSCRIBER_JWT _KEY='!ChangeMe!' ./mercure run
Caddy file :
{
{$GLOBAL_OPTIONS}
}
{
auto_https off
}
{$SERVER_NAME:localhost}
log
route {
encode zstd gzip
mercure {
# Transport to use (default to Bolt)
transport_url {$MERCURE_TRANSPORT_URL:bolt://mercure.db}
# Publisher JWT key
publisher_jwt {env.MERCURE_PUBLISHER_JWT_KEY} {env.MERCURE_PUBLISHER_JWT_ALG}
# Subscriber JWT key
subscriber_jwt {env.MERCURE_SUBSCRIBER_JWT_KEY} {env.MERCURE_SUBSCRIBER_JWT_ALG}
# Extra directives
{$MERCURE_EXTRA_DIRECTIVES}
}
respond /healthz 200
respond "Not Found" 404
}
It returns :
run: loading initial config: loading new config: http app module: start: tcp: listening on :443: listen tcp :443: bind: permission denied
Can anyone provide a solution : I intend to host my symfony project on a web server using apache2 on the same Raspberrry
I don't know this specific application, but your error message:
listen tcp :80: bind: permission denied
could be related with restriction for ports 80 and 443 (second message) - non-root user cannot use ports lower than 1024 on standard Linux configuration. Try to use different port or (if you don't care about security - i.e. local hobby project) run app as root.
Keep in mind that you can run Nginx as reverse proxy, so you can run your app on any high port (like 3000) on standard user.
it's a rights issue with your user.
Try with sudo, it should work.
I have an udp input in fluentd :
<source> # mylog
#type udp
port 41234
tag json
<parse>
#type none
</parse>
</source>
I'd like to pass it through a tcp ouput,because I go through haproxys and then in a logstash input. Historic reason.
I got it worked with #type forward for output, So I assume it forward udp, according to the doc.
But then in logstash side I got some extra weird characters around the payload. logstash conf :
input {
tcp {
host => "127.0.0.1"
port => 41234
codec => "json"
}
}
How would you do that ? Is that even possible?
I've been using elasticsearch, metricbeat and elastalert to watch my server. I have nginx intalled on it that is been used as a reverse proxy and I need to send an to it if nginx drop or return some error, I have already some alerts configured but how can I make a rule to send alert to nginx when it drop or return some error.
Thank a lot
Metricbeat is just for data about the system resources usage. What you need is installing filebeat and activating the nginx module. Then you can use the rule type any of elastalert and filter by fileset.module: nginx and fileset.name: error:
name: your rule name
index: filebeat-*
type: any
filter:
- term:
fileset.module: "nginx"
- term:
fileset.name: "error"
alert:
- "slack"
... # your slack config stuff
realert:
minutes: 1
I am setting up simple tcp connection routing using HAProxy acl's. The idea is to route connections depending on request content having two flavors: read and write requests.
For testing I made a simple tcp client/server setup using perl. Strangely enough about 10-40% of the ACL's fail and are sent to the default backend.
The ACL's should find the substring 'read' or 'write' and route accordingly, but this is not allways the case.
Sending a read/write request using nc (netcat) has the same effect.
I tested this configuration with mode=http and everything works as expected.
I also tested with reg, sub and bin, to no avail.
The example server setup is as follows:
HAProxy instance, listens on port 8000
Client (creates tcp connection to proxy and sends user input (read/write string) to server through port 8000, after which it closes the connection)
Server1 (write server), listens on port 8001
Server2 (read server), listens on port 8002
Server3 (default server), listens on port 8003
My HAProxy configuration file looks is:
global
log /dev/log local0 debug
#daemon
maxconn 32
defaults
log global
balance roundrobin
mode tcp
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
frontend tcp-in
bind *:8000
tcp-request inspect-delay 3s
acl read req.payload(0,4) -m sub read
acl write req.payload(0,5) -m sub write
use_backend read_servers if read
use_backend write_server if write
default_backend testFault
backend write_server
server server1 127.0.0.1:8001 maxconn 32
backend read_servers
server server2 127.0.0.1:8002 maxconn 32
backend testFault
server server3 127.0.0.1:8003 maxconn 32
The client code (in perl):
use IO::Socket::INET;
# auto-flush on socket
#$| = 1;
print "connecting to the server\n";
while(<STDIN>){
# create a connecting socket
my $socket = new IO::Socket::INET (
PeerHost => 'localhost',
PeerPort => '8000',
Proto => 'tcp',
);
die "cannot connect to the server $!\n" unless $socket;
# data to send to a server
$req = $_;
chomp $req;
$size = $socket->send($req);
print "sent data of length $size\n";
# notify server that request has been sent
shutdown($socket, 1);
# receive a response of up to 1024 characters from server
$response = "";
$socket->recv($response, 1024);
print "received response: $response\n";
$socket->close();
}
The server (perl code):
use IO::Socket::INET;
if(!$ARGV[0]){
die("Usage; specify a port..");
}
# auto-flush on socket
$| = 1;
# creating a listening socket
my $socket = new IO::Socket::INET (
LocalHost => '0.0.0.0',
LocalPort => $ARGV[0],
Proto => 'tcp',
Listen => 5,
Reuse => 0
);
die "cannot create socket $!\n" unless $socket;
print "server waiting for client connection on port $ARGV[0]\n";
while(1){
# waiting for a new client connection
my $client_socket = $socket->accept();
# get information about a newly connected client
my $client_address = $client_socket->peerhost();
my $client_port = $client_socket->peerport();
print "connection from $client_address:$client_port\n";
# read up to 1024 characters from the connected client
my $data = "";
$client_socket->recv($data, 1024);
print "received data: $data\n";
# write response data to the connected client
$data = "ok";
$client_socket->send($data);
# notify client that response has been sent
shutdown($client_socket, 1);
$client_socket->close();
print "Connection closed..\n\n";
}
$socket->close();
Binary data in haproxy is tricky. Probably some bug, but the following worked for me on haproxy 1.7.9.
I am trying to build a thrift proxy server which can route to appropriate backend based on the user_id in the payload.
frontend thriftrouter
bind *:10090
mode tcp
option tcplog
log global
log-format "%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq captured_user:%[capture.req.hdr(0)] req.len:%[capture.req.hdr(1)]"
tcp-request inspect-delay 100ms
tcp-request content capture req.payload(52,10) len 10
tcp-request content capture req.len len 10
tcp-request content accept if WAIT_END
acl acl_thrift_call req.payload(2,2) -m bin 0001 # Thrift CALL method
acl acl_magic_field_id req.payload(30,2) -m bin 270f # Magic field number 9999
# Define access control list for each user
acl acl_user_u1 req.payload(52,10) -m sub |user1|
acl acl_user_u2 req.payload(52,10) -m sub |user2|
# Route based on the user. No default backend so that one always has to set it
use_backend backend_1 if acl_user_u1 acl_magic_field_id acl_thrift_call
use_backend backend_2 if acl_user_u2 acl_magic_field_id acl_thrift_call
When matching binary data in acl, make sure you're looking at the right number of bytes, for substring to work properly. Or use the hex conversion method and match on hex bytes.
Dont I feel silly. Re-reading the HAProxy documentation I found the following directive (fetch method) that fixes the issue:
tcp-request content accept if WAIT_END
That solved the unexpected behaviour.
I have box A and it has a consumer on it that listens on a Rabbit MQ server
I have box B that will publish a message to the listener
So as long as all of this in on box A and I start Rabbit MQ server w/ defaults it works fine.
The defaults are host=127.0.0.1 on port 5672, but
when I telnet box.a.ip.addy 5672 from box B I get:
Trying box.a.ip.addy...
telnet: connect to address box.a.ip.addy: No route to host
telnet: Unable to connect to remote host: No route to host
telnet on port 22 is fine, I can ssh into Box A from Box B
So I assume I need to change the ip that the RabbitMQ server uses
I found this: http://www.rabbitmq.com/configure.html and I now have a config file in the location the documentation said to use, with the name rabbitmq.config and it contains:
[
{rabbit, [{tcp_listeners, {"box.a.ip.addy", 5672}}]}
].
So I stopped the server, and started RabbitMQ server again. It failed. Here are the errors from the error logs. It's a little over my head. (in fact most of this is)
=ERROR REPORT==== 23-Aug-2011::14:49:36 ===
FAILED
Reason: {{case_clause,{{"box.a.ip.addy",5672}}},
[{rabbit_networking,'-boot_tcp/0-lc$^0/1-0-',1},
{rabbit_networking,boot_tcp,0},
{rabbit_networking,boot,0},
{rabbit,'-run_boot_step/1-lc$^1/1-1-',1},
{rabbit,run_boot_step,1},
{rabbit,'-start/2-lc$^0/1-0-',1},
{rabbit,start,2},
{application_master,start_it_old,4}]}
=INFO REPORT==== 23-Aug-2011::14:49:37 ===
application: rabbit
exited: {bad_return,{{rabbit,start,[normal,[]]},
{'EXIT',{rabbit,failure_during_boot}}}}
type: permanent
and here is some more from the start up log:
Erlang has closed
Error: {node_start_failed,normal}
^M
Crash dump was written to: erl_crash.dump^M
Kernel pid terminated (application_controller) ({application_start_failure,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot}}}}})^M
Please help
did you try adding?
RABBITMQ_NODE_IP_ADDRESS=box.a.ip.addy
to the /etc/rabbitmq/rabbitmq.conf file?
Per http://www.rabbitmq.com/configure.html#customise-general-unix-environment
Also per this documentation it states that the default is to bind to all interfaces. Perhaps there is a configuration setting or environment variable already set in your system to restrict the server to localhost overriding anything else you do.
UPDATE: After reading again I realize that the telnet should have returned "Connection Refused" not "No route to host." I would also check to see if you are having a firewall related issue.
You need to open up the tcp port on your firewall
Using Linux, Find the iptables config file:
eric#dev ~$ find / -name "iptables" 2>/dev/null
/etc/sysconfig/iptables
Edit the file:
sudo vi /etc/sysconfig/iptables
Fix the file by adding a port:
# Generated by iptables-save v1.4.7 on Thu Jan 16 16:43:13 2014
*filter
-A INPUT -p tcp -m tcp --dport 15672 -j ACCEPT
COMMIT