I have a client which sends data via UDP-broadcast. (To let's say 127.0.0.255:12345)
Now I want to have multiple servers listening to this data. To do so on a local machine, they need to share the port 12345 for listening.
My question is, if that is possible, if there are any disadvantages and if there could be problems with this approach.
There is one alternative which unfortunately brings with a lot of overhead:
Implement some kind of registration-process. On startup, each server tells the client its port. The client then sends the messages to each port (having to send the data multiple times, some kind of handshaking needs to be implemented...)
Do you know any better alternative?
If that matters:
I'm using C++ with Boost::Asio. The software should be portable (mainly Linux and Windows).
You will have to bind the socket in both processes with the SO_REUSEPORT option. If you don't specify this option in the first process, binding in the second will fail. Likewise, if you specify this option in the first but not the second, binding in the second will fail. This option effectively specifies both a request ("I want to bind to this port even if it's already bound by another process") and a permission ("other processes may bind to this port too").
See section 4.12 of this document for more information.
This answer is referenced to the answer of cdhowie, who linked a document which states that SO_REUSEPORT would have the effect I'm trying to achieve.
I've researched how and if this option is implemented and focused mainly on Boost::Asio and Linux.
Boost::Asio does only set this option if the OS is equal to BSD or MacOSX. The code for that is contained in the file boost/asio/detail/reactive_socket_service.hpp (Boost Version 1.40, in newer versions, the code has been moved into other files).
I've wondered why Asio does not define this option for platforms like Linux and Windows.
There are several references discussing that this is not implemented in Linux:
https://web.archive.org/web/20120315052906/http://kerneltrap.org/mailarchive/linux-netdev/2008/8/7/2851754
http://kerneltrap.org/mailarchive/linux-kernel/2010/6/23/4586155
There also is a patch which should add this functionality to the kernel:
https://web-beta.archive.org/web/20110807043058/http://kerneltrap.org/mailarchive/linux-netdev/2010/4/19/6274993
I don't know if this option is existing for Windows, but by defining portable as an attribute for software which runs on Linux too, this means, that SO_REUSEPORT is OS specific and there is no portable solution for my question.
In one of the discussions I've linked it is recommended for UDP to implement a master-listener which then provides the incoming data to multiple slave-listeners.
I will mark this answer as accepted (though feeling kind of bad by accepting my own answer), because it points out why the approach of using SO_REUSEPORT will fail when trying to use it with portable software.
Several sources explain that you should use SO_REUSEADDR on windows. But none mention that it is possible to receive UDP message with and without binding the socket.
The code below binds the socket to a local listen_endpoint, that is essential, because without that you can and will still receive your UDP messages, but by default your will have exclusive ownership of the port.
However if you set reuse_address(true) on the socket (or on the acceptor when using TCP), and bind the socket afterwards, it will enable multiple applications, or multiple instances of your own application to do it again, and everyone will receive all messages.
// Create the socket so that multiple may be bound to the same address.
boost::asio::ip::udp::endpoint listen_endpoint(
listen_address, multicast_port);
// == important part ==
socket_.open(listen_endpoint.protocol());
socket_.set_option(boost::asio::ip::udp::socket::reuse_address(true));
socket_.bind(listen_endpoint);
// == important part ==
boost::array<char, 2000> recvBuffer;
socket_.async_receive_from(boost::asio::buffer(recvBuffer), m_remote_endpoint,
boost::bind(&SocketReader::ReceiveUDPMessage, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred)
Related
My questions concerns gRPC clients using the C core, specifically C++
I've been debugging one of our servers, and I've noticed a certain two-client flow worked when the clients were launched from separate processes (two separate console windows) but not from within an automated test case (which runs within a single process). The flow in question involves two "clients" (channels, basically) which are alive at the same time and concurrently issuing requests to the same server
Digging further, I discovered that in the working flow the server received requests from two different ip:port combinations: 127.0.0.1:xxxx and 127.0.0.1:yyyy. In the failing scenario however, both requests come from the same ip:port.
I create a separate channel for every client, so this behavior confused me. I have a couple of questions
Does gRPC share ports between channels in the same process like this? If not, then I have to imagine there's a bug in my code
If yes to (1), is there any way to avoid this port reuse?
I do see the "grpc.so_reuseport" option in the channel's metadata, and note that it is enabled by default. This seems more related to servers than clients (though perhaps I'm making an arbitrary distinction), but I'll disable it and try things out
EDIT: The so_reuseport option doesn't do anything, but I am on Windows so I should have expected that anyways :/ I also found a related question without any answers here
EDIT 2: The discussion on this question seems promising. Will try it out and report back
This is mentioned/explained in the question I link in edit 2, but the answer is to use
auto metadata = grpc::ChannelArguments();
metadata.SetInt(GRPC_ARG_USE_LOCAL_SUBCHANNEL_POOL, 1);
and pass it to grpc::CreateCustomChannel
I've got a proprietary BMS language that is sending it's info over a specific UDP port on the network. The existing interface is not very well made or maintained, and functions poorly.
I have access to the stack for the code, and don't mind creating some interpretation functionality
My question is what is the best way that I should be receiving these raw packets in my program to be interpreted? I'm not finding any good documentation on how to do this, and I wanted to try and do it in a reasonably appropriate way.
Do I basically need to make my program constantly sniff a specific port? and will this be cumbersome to the network or program to be doing this?
You tagged this BACnet. Why don't you try Wireshark, with a capture filter "udp port 47808" and see if wireshark exposes the packets in a way that makes sense to you. (or have you done this). If it is bacnet, then normal UDP sockets, bound to port 47808 is the way to go. Note, that 47808-47823 are the most common BACnet "default" ports. Use cports or something to see exactly what port(s) your application is bound to.
You could use a packet-capture library - but that has security connotations, so instead you can probably (for most part) get away with using a .NET 'UdpClient'.
But! The real challenge is the breaking-down & interpretation of the BACnet packets, which is the hard part.
There is (now!/finally) a NuGet package for BACnet - not that I've used it, but that might be one of the best choices for your case.
But I also suggest you experiment with the (advanced & free) VTS (Visual Test Tool) too.
You could also try using the BACnet stack that YABE uses too.
I'm writing a bit of desktop software which has two components. Component B queries component A. Creating a web service seems like an ideal way to do IPC in principle. The data model fits, there are ready-made client and server libraries, a well known way to encode and decode parameters etc.
But setting up an HTTP server on a network socket doesn't seem right for a local application. For example what port do I choose? I don't really want people to be able to scan and talk to the app from outside etc.
So I was thinking that I might be able to do HTTP over a domain socket. Does that make any sense? Is there any precedence for it? Is there an equivalent protocol that I could use for IPC which has the same properties as HTTP (requests for specified resources (URIs), encoded parameters, response)?
Looking for C libraries (and possibly Go and ObjC for bonus points).
Binding to the loopback interface only (127.0.0.1) solves your "external visibility" problem, only processes on the local machine will be able to connect.
It does not solve your port allocation problem though, the port number you choose might be taken by the time your app starts. Then your server can't bind and your client connects to the other process bound to your port.
Old, less hip, but CORBA implementations tend to have the problems you have not thought of yet figured out already.
How (GA) Global array library (an implementation of ARMCI) is used for communication between two process located on different remote machines.
Is that something similar to TCP socket programming where one process wait for data and the other transfers it ?
I try to see the documentation that ga_put() and ga_get() are two operation that used for inter-process communication. till now I only able to come up with a program running on the same machine that use shared-Memory architecture (I have used ga_put() and ga_get() to put data in Global array and to get it respectively ).
Now, I want use this program for communicating data (basically performaning one-sided communication) between two remote processes. Obiviously putting the program that I am running on single machine on the remote side will work out. It needs some way to tell which machine should we access and get the right data. And here is where I need your help. how can I do this? (what is its equivalent of TCP/IP listen, accept and connect ... on GA ? )
Or is that the case that GA also uses TCP/IP socket underneath ?
can some one please explain to me? and sample code of two remote processes communicating is also appreciable.
thanks,
I am answering my question after all. May be it will help some one looking for the same issue.
GA Library is implemented to work with MPI. So we have something like:
MPI_Init(..)
GA_Initialize()
MA_Init(..)
// .... do sothing here
GA_Terminate()
MPI_Finalize()
The answer to my question is:
MPI has the following primitives to be able to support client-server commuication:
//in the server side
MPI_Open_port()
MPI_Comm_accept()
//do MPI_Send() or MPI_Recv()
MPI_Close_port()
//client Side
MPI_Comm_connect()
//do MPI_Recv() or MPI_Send()
depending on the hardware support and the MPI implementation used, MPI might use sockets, or other mechanisms (e.g SAN (System area network)).
In general, most MPI implementations use sockets for TCP based communication.
So, yes GA also uses sockets underneath (of course depending on the MPI implementation used)
cheers,
Just a technical question -
Can two or more SNMP agents be run on the same port (on the same machine)?
My first instinct would be no since host:port identifies an instance of an application but I'm not sure.
Thank you!
Technically, if the OS supports it, the SO_REUSEADDR SO_REUSEPORT options may be set on a socket to allow other processes to bind to the same address/port and thus allow multiple processes to receive messages on the same address/port. But both processes would have to set the option, and I doubt any agent implementations do that because it would not make sense to do so--it would just cause headaches having both agents potentially responding to a single request. Managers won't be equipped to handle it.
However, you can instead run an SNMP proxy in the primary address/port, configured to forward requests to one of multiple agents based on query, security, or (with SNMPv3) context/engine ID parameters, and forward responses back.
Also, using AgentX, you have an SNMP master agent running on the primary address/port, and one or more SNMP sub-agents connected to the master agent. The master agent dispatches requests to the sub-agents as appropriate, merging the results into a single response, so that to the outside world it appears as a single agent. Each sub-agent typically handles a different branch of OID space (one sub-agent implementing certain module(s), another sub-agent implementing other module(s)).
But taking two agents intended to own the address/port exclusively, and forcing them to share through the REUSE options, while it may be possible, would not be wise.
You can run multiple agents on the same host and with the same port if they have differents ip address (can use a netsh script for that).
Personnaly I use the nsoftware ddl : SecureSNMP V8 edition .NET to do this.
You can look at this post : Multiple SNMP Agents with nsoftware dll
No, two agents cannot both run on the same port as seperate applications for the reasons you assumed (except with a brittle packet sniffing hack, which we'll not go into).
However, 2 agents can be accessed through the same port if there is some mechanism that handles the actual port and distributes requests based on MIB. For example the Windows SNMP service does this, allowing any number of SNMP agents to be added as "extensions" through the registry (HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SNMP\Parameters\ExtensionAgents) by writing them as DLLs and using the snmp.h headers in the platform SDK.
You are correct: ports can't be shared.
If both the agents were designed by you, then the answer can be different.
Consider the HTTP and FTP cases, we can use host names to distinguise multiple sites on the same port, then why can't we do it for SNMP?
We can create a dispatcher who monitors port 161 for incoming traffic. Then use multiple real agents to handle those traffic behind. We can feel free to design how to distinguise them. Personally I prefer the FTP virtual host name manner and use | to distinguise agents.
Maybe I can create a demo for #SNMP Suite in the future.
But if you need to work with existing agents on the same server, then such flexibility is lost.