I am using Apache Kafka. I use KafkaProducer to produce data and KafkaConsumer to consume data. My config data is:
Properties props = new Properties();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
props.put(ProducerConfig.CLIENT_ID_CONFIG, "DemoProducer");
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.IntegerSerializer");
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer");
The consumer config is the same. How do I view the whole TCP packet details which are being sent and received by the producers and consumers?
After you installed Wireshark, you must go to Edit->Prefrences or press Ctrl + Shift + P. In the new window open protocols tree and find Kafka in the list. After selecting Kafka you must set the Kafka broker port numbers(It's by default 9092). And Also check Show length for string and bytes fields in the protocol tree checkbox as shown in blew image:
and click ok to dismiss the window. And then you can view all Kafka APIs calls:
You can use tshark with a Kafka dissector already available.
sudo apt-get update
sudo apt-get install tshark
sudo tshark -V -i lo -o 'kafka.tcp.port:9092' -d tcp.port=9092,kafka -f 'dst port 9092'
More details here Using the kafka dissector in wireshark/tshark 1.12
Related
I am developing an UEFI App that will need to perform a GET request through http.
As a start up point, I want to make sure my setup is working properly so that the http requests can actually go through.
To that end, I spent the last few days trying to make the http command work in the EFI Shell launched inside QEMU.
I can get the ping command to work properly, but calling:
http httpbin.org/get
Always returns:
Unable to open http protocol on `eth0` - Unsupported
Unable to download the file `/get` on `eth0` - Unsupported
This is my startup.nsh script to configure the EFI Shell's interface:
connect
ifconfig -r eth0
ifconfig -s eth0 dhcp
ifconfig -l eth0
These were my different attempts at invoking Qemu properly:
-netdev user,id=mynet0,hostfwd=tcp::8080-:80 -device e1000,netdev=mynet0 \
-netdev user,id=user.0 -device e1000,netdev=user.0 \
-nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:32 \
And following this guide I tried to setup a tap, albeit without luck, I'd launch qemu with the following configuration:
-netdev tap,id=mynet0,ifname=tap0,script=no,downscript=no -device e1000,netdev=mynet0,mac=52:55:00:d1:55:01 \
Do you have any clue what step am I missing?
Where do you believe I could be failing in making eth0 supported?
Is the tap crucial?
Are you able to make this setup work on your side?
Update:
Very good suggestion #MiSimon, I hadn't realized that the HttpDxe driver wasn't being built with the OvmfPkg.
I have now added its INF to OvmfPkgX64.dsc and OvmfPkgX64.fdf.
Although, running drivers is displaying a duplicate entry:
0000000A D - - 1 - HttpDxe HttpDxe
0000000A ? - - - - HttpDxe HttpDxe
With respect to calling the http command, the error has progressed to:
Downloading 'http://httpbin.org/get'
Unable to download the file '/get' on 'eth0' - Unsupported
The debug log shows:
HttpNotify: Event - 0, EventStatus - Unsupported
Error: Could not retrieve the host address from DNS server.
The tool requires nearly all network drivers to be loaded.
Make sure your image contains the following drivers:
SnpDxe
MnpDxe
ArpDxe
Ip4Dxe/Ip6Dxe
Dhcp4Dxe/Dhcp6Dxe
Udp4Dxe/Udp6Dxe
DnsDxe
TcpDxe
HttpDxe
HttpUtilitiesDxe
All of them are can be found in EDK2 inside the NetworkPkg.
I have a virtual machine that is supposed to be the host, which can receive and send data. The first picture is the error that I'm getting on my main machine (from which I'm trying to send data from). The second picture is the mosquitto log on my virtual machine. Also I'm using the default config, which as far as I know can't cause these problems, at least from what I have seen from other examples. I have very little understanding on how all of this works, so any help is appreciated.
What I have tried on the host machine:
Disabling Windows defender
Adding firewall rules for "mosquitto.exe"
Installing mosquitto on a linux machine
Starting with the release of Mosquitto version 2.0.0 (you are running v2.0.2) the default config will only bind to localhost as a move to a more secure default posture.
If you want to be able to access the broker from other machines you will need to explicitly edit the config files to either add a new listener that binds to the external IP address (or 0.0.0.0) or add a bind entry for the default listener.
By default it will also only allow anonymous connections (without username/password) from localhost, to allow anonymous from remote add:
allow_anonymous true
More details can be found in the 2.0 release notes here
You have to run with
mosquitto -c mosquitto.conf
mosquitto.conf, which exists in the folder same with execution file exists (C:\Program Files\mosquitto etc.), have to include following line.
listener 1883 ip_address_of_the_machine(192.168.1.1 etc.)
By default, the Mosquitto broker will only accept connections from clients on the local machine (the server hosting the broker).
Therefore, a custom configuration needs to be used with your instance of Mosquitto in order to accept connections from remote clients.
On your Windows machine, run a text editor as administrator and paste the following text:
listener 1883
allow_anonymous true
This creates a listener on port 1883 and allows anonymous connections. By default the number of connections is infinite. Save the file to "C:\Program Files\Mosquitto" using a file name with the ".conf" extension such as "your_conf_file.conf".
Open a terminal window and navigate to the mosquitto directory. Run the following command:
mosquitto -v -c your_conf_file.conf
where
-c : specify the broker config file.
-v : verbose mode - enable all logging types. This overrides
any logging options given in the config file.
I found I had to add, not only bind_address ip_address but also had to set allow_anonymous true before devices could connect successfully to MQTT. Of course I understand that a better option would be to set user and password on each device. But that's a next step after everything actually works in the minimum configuration.
For those who use mosquitto with homebrew on Mac.
Adding these two lines to /opt/homebrew/Cellar/mosquitto/2.0.15/etc/mosquitto/mosquitto.conf fixed my issue.
allow_anonymous true
listener 1883
you can run it with the included 'no-auth' config file like so:
mosquitto -c /mosquitto-no-auth.conf
I had the same problem while running it inside docker container (generated with docker-compose).
In docker-compose.yml file this is done with:
command: mosquitto -c /mosquitto-no-auth.conf
I am getting node down error on rabbitmq, this is happening sometimes.
Able to see the below error when I execute: sudo rabbitmqctl status or sudo rabbitmqctl list_queues
Error: unable to connect to node : nodedown
connected to epmd (port 4369) on host-name
epmd reports node 'rabbit' running on port 25672
can't establish TCP connection, reason: timeout
suggestion: blocked by firewall?
version: {rabbit,"RabbitMQ","3.6.9"}
os: Ubuntu 16.04
I have checked hostname which is ok with me, not changed since the installation
Also able to telnet localhost 25672
What could be the reason behind this error and possible solution?
And one more question, I am checking node status using below API
curl -s GET http://edx:edx#127.0.0.1:15672/api/healthchecks/node/
Is above API ok or not to check the health status of the node? Please suggest if there is anything else. I have set up one shell script which will call this API and if status is not ok then it will restart rabbitmq-server service. Script is executed from cron every minute.
Looks like your rabbitmq node is... down. rabbitmqctl needs a running node to perform these commands.
If you're using systemd, you can check the service status:
service rabbitmq-server status
Or just try to restart the node:
rabbitmqctl start_app
Telnet on port 25672 tells you the rabbitmqctl is running, but RabbitMQ itself does not run on that port (by default, it's listening on 5672).
I am writing amqp 1.0 client (using rabbitMQ.Client in .NET) for a broker who provided me the following information:
amqps://brokerRemoteHostName:5671
certificate_openssl.p12
password for certificate as a string "mypassword"
queue name
I developed the following code in Visual Studio which is supposed to work (based on long searches on the web):
var cf = new ConnectionFactory();
cf.Uri = new Uri("amqps://brokerRemoteHostName:5671");
cf.Ssl.Enabled = true;
cf.Ssl.ServerName = "brokerRemoteHostName";
cf.Ssl.CertPath = #"C:\Users\mahmoud\Documents\certificate_openssl.p12";
cf.Ssl.CertPassphrase = "myPassword";
var connection = cf.CreateConnection();
However, the output shows an exception:
RabbitMQ.Client.Exceptions.BrokerUnreachableException:
None of the specified endpoints were reachable ---> System.IO.IOException:
connection.start was never received
likely due to a network timeout) as seen in the image.
Where line 50 corresponds to the line where we create the connection.
I appreciate your kind assistance on the error above.
If you're connecting to a docker container, you need to add the 5672 port in addition to 15672 port when creating the container. For those using ssl, the port would be 5671 instead of 5672.
Example: docker run -d --hostname my-rabbit --name rabbitmq --net customnet -p customport:15672 -p 5672:5672 rabbitmq:3-management.
You would connect from client by calling this: ConnectionFactory factory = new ConnectionFactory() { HostName = "localhost" };.
Feel free to pass in username and password if those were changed.
Official RabbitMq docker image https://hub.docker.com/_/rabbitmq starts RabbitMq broker on port 5672, but .NET RabbitMq library expects to see broker on port 5673 which for sure differs from what we have in fact in docker. The solution is just to remap 5672 to expected 5673 port
docker run -d --hostname my-rabbit --name ds-rabbit -p 8080:15672 -p 5673:5672 rabbitmq:3-management
I have a process using https. I found its PID using ps and used the command lsof -Pan -p PID -i to get the port number it is running on.
I need iftop to see the data transfer. The filter I am using now is
iftop -f "port http 57787".
I don't think this is giving me the right output.
Can someone help me the right filter to use with iftop so that I know the traffic going through only this port?
I can see 2 problems here:
1/ Is that a typo? The correct option for filtering is -f (small "f"). -F (capital "F") option is for net/mask.
2/ Though not explicitly stated by iftop documentation, the syntax for filtering seems to be the pcap one from the few examples given (and using ldd I can see that yes, the iftop binary is linked with libpcap). So a filter with http is simply not valid. To see the doc for pcap filtering syntax, have a look at pcap-filter (7) - packet filter syntax man page. In your example, a filter such as "tcp port 57787" would be OK. pcap does not do layer 5 and above protocol dissection such as http (pcap filters are handled by BPF in the kernel, so above layer 4 you're on your own, because that's none of the kernel business).
All in all, these looks like iperf bugs. It should refuse your "-F" option, and even with "-f" instead exit with an error code because pcap will refuse the filter expression. No big deal, iftop is a modest program. See edit bellow.
EDIT:
I just checked iftop version 1.0pre4 source code, and there is no such obvious bug from a look at set_filter_code() and its caller packet_init() in iftop.c. It correctly exit with error, but...
Error 2, use the "-f" option, but your incorrect filter syntax:
jbm#sumo:~$ sudo iftop -f "port http 57787"
interface: eth0
IP address is: 192.168.1.67
MAC address is: 8c:89:a5:57:10:3c
set_filter_code: syntax error
That's OK.
Error 1, the "-F" instead of "-f", there is a problem:
jbm#sumo:~$ sudo iftop -F "port http 57787"
(everything seems more or less OK, but then quit the program)
Could not parse net/mask: port http 57787
interface: eth0
IP address is: 192.168.1.67
MAC address is: 8c:89:a5:57:10:3c
Oops! "Could not parse net/mask: port http 57787"! That's a bug: it should exit right away.