I have an application (say, TcpApp) sending pure TCP messages (i.e., no SOAP, no envelope ... just a raw string or even bytes). I need to connect ESB to listen those messages over a specific port (say, 3333), and make some mediation (for now, do nothing but logging is enough). I think it would be a good idea to make an ActiveMQ queue from TcpApp and then to make a proxy service from JMS in the ESB (instead of connect directly the ESB to the TcpApp).
I read several samples and answers, but always the contect is XML, and TCP is only the transport. What sometime happens is that applications send no special formats over TCP (sometime called telegrams).
I tried to change the content type, but still the ESB refuses to read the TCP port.
<parameter name="transport.tcp.contentType">text/plain</parameter>
May be I'm still confuse with the architecture of the solution, but I think a Broker, or an ESB like WSO2, should work is this case as a mediator from this TcpApp. I prefer to disscus the solution before to get the real config to make it work.
All comments, welcomed!
In WSO2 EI 6.1.1, I have found that I can successfully process plain text TCP messages if I also specify a recordDelimiter and recordDelimiterType. Example from a working proxy (with the line feed character as delimiter):
<parameter name="transport.tcp.responseClient">true</parameter>
<parameter name="transport.tcp.inputType">binary</parameter>
<parameter name="transport.tcp.recordDelimiter">0x0A</parameter>
<parameter name="transport.tcp.contentType">text/plain</parameter>
<parameter name="transport.tcp.port">50001</parameter>
<parameter name="transport.tcp.recordDelimiterType">byte</parameter>
The message body in the input sequence looks like this:
<text xmlns="http://ws.apache.org/commons/ns/payload">this_is_the_message</text>
You need to use the correct message formatters and builders to process anything. Use following formatters in the axis2.xml file.
<messageFormatter contentType="application/binary" class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>
<messageBuilder contentType="application/binary" class="org.wso2.carbon.relay.BinaryRelayBuilder"/>
Just change the content type whatever you like and use the same in the proxy service config as well. Actually I have a blog post on this as well [1] :)
[1] - http://soatechflicks.blogspot.com/2017/05/processing-binary-data-from-tcp.html
Related
thank you for your time and help on this. I am having relentless issue's with getting Cisco ASA ingested to the CommonSecurityLog data table. I think it's stems from how I'm receiving the messages via syslog and my understanding of the architecture of the omsagent and how it differentiates between CEF and Syslog. Currently, we don't have anything writing to any syslog facilities. I am writing my cisco asa messages to a custom file that is generated everyday. It it sending on TCP/1470 because cisco asa does not support TCP/514. The logs are flowing to the machine successfully, so I don't have a conf syntax issue. Although I can't seem to find anything helpful to get this in to Sentinel now that it is sitting on my syslog server outside of creating a custom log that won't have field mapping. Below is what my syslog-ng.conf looks like for the related source. I also ran the validate connectivity script within the data connector page to make sure everything was okay with the agent connecting to the workspace.
source s_cisco {
tcp(port(1470));
};
destination d_cisco_asa { file("/opt/syslog-ng/cisco_asa/$HOST/$YEAR-$MONTH-$DAY-$SOURCEIP-cisco_asa.log");};
filter f_cisco_asa {
host(x.x.x.x);
};
log { source(s_cisco); filter(f_cisco_asa); destination(d_cisco_asa); };
To collect logs by Linux agent we need send them to the agent over port 25226
#syslog config
destination security_oms { udp("127.0.0.1" port(25226)); };
and then create security events configuration file
#oms config
#/etc/opt/microsoft/omsagent/<workspace id>/conf/omsagent.d/
<source>
type syslog
port 25226
bind 127.0.0.1
protocol_type tcp
tag oms.security
format /(?<time>(?:\w+ +){2,3}(?:\d+:){2}\d+|\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.[\w\-\:\+]{3,12}):?\s*(?:(?<host>[^: ]+) ?:?)?\s*(?<ident>.*CEF.+?(?=0\|)|%ASA[0-9\-]{8,10})\s*:?(?<message>0\|.*|.*)/
<parse>
message_format auto
</parse>
</source>
<filter oms.security.**>
type filter_syslog_security
</filter>
More about that you could find on oms github page OMS security events configuration
I'm using ForcebindIP to point an app at a specific network adapter, like this:
forcebindip -i 192.168.0.5 MyCSharpApp.exe
This works fine and the app isn't aware (or doesn't access) any of the other network adapters on the PC.
Is it possible to restrict ForceBindIP to outbound traffic only leaving the app to receive data from any local network adapter? Or even to specify a network adapter for outbound and another for inbound traffic?
I can't find an extra startup parameter for ForceBindIP that does this.
I'd appreciate any help with this.
If I get your problem correctly, you want to bind your application to listen for packets on all available interfaces but return packets to only through one given interface. I also assume it's a server application and you don't have neiter source code nor control over its behaviour.
Disclosure: I do not know how ForceBindIP works internally, I'm basing my understanding of it on this passage from the website:
it will then inject a DLL (BindIP.dll) which loads WS2_32.DLL into memory and intercepts the bind(), connect(), sendto(), WSAConnect() and WSASendTo() functions, redirecting them to code in the DLL which verifies which interface they will be bound to and if not the one specified, (re)binds the socket
Problems to overcome
I don't believe your desired configuration is possible with just one application level DLL injector. I'll list a few issues that ForceBindIP will have to overcome to make it work:
to listen to a socket, application has to bind() it to a unique protocol-address-port combination first. An application can bind itself to either a specific address or a wildcard (i.e. listen on all interfaces). Apparently, one can bind to wildcard and specific address simultaneously as outlined in this SO question. This however will be two different sockets from the application standpoint. Therefore your application will have to know how to handle this sort of traffic.
When accepting client connection, accept() will create a new socket and parameters on that are managed by Windows, I don't believe there's an API to intercept binding here - by this time the connection is considered established.
Now imagine, we somehow got a magic socket. We can receive packets on one interface and send to another. The client (and all routing equipment on the way) will have to be aware that two packets originating from two different source IP addresses are actually part of the same connection and be able to assemble the TCP session (or correctly merge UDP streams).
You can have multiple gefault gateways with different priorities and rules (which is a whole different topic to explore) but as far as I'm aware that's not going to solve your particular issue: majority of routing protocols assume links are symmetric and expect packets to keep within same interface. There are special cases like asymmetric routing and network interface teaming but they have to be implemented on per-interface level.
One potential solution
One way to achieve what you're after (I don't know enough about your environment to claim it will work), will be to create a virtual interface, set it into yet another IP network, bind your application to it, then use firewall (to, say, allow multicast backets into the "virtual" network) and routing from that network to required default gateway with metric set to 1. I also suspect just any Windows will not be that flexible, so you might need like a Server Edition.
I am sorry this didn't turn out to be the ready-to-fly solution, I however am hoping this gives you more context to the problem you are facing and points you into other directions to explore.
You can use Set-NetAdapterAdvancedProperty command in Powershell to set the flow control of your specified adapter
To get the names and properties of all the network adapter :-
Get-NetAdapterAdvancedProperty -Name "*"
Suppose you want the network adapter named "Ethernet 2" to be only used to receive data from internet then type :-
Set-NetAdapterAdvancedProperty -Name "Ethernet 2" -DisplayName "Flow Control" -DisplayValue "Rx Enabled"
You can find more in :
https://learn.microsoft.com/en-us/powershell/module/netadapter/set-netadapteradvancedproperty?view=win10-ps
Microsoft winsock example has a usage in their example for limiting a socket to only send or receive mode. It might help.
https://learn.microsoft.com/en-us/windows/win32/winsock/complete-client-code
Outbount and Inbount limits are not imposed while binding. But latter or when connection is established.
Line of code pertaining to this in client code is toward the end.
// shutdown the connection since no more data will be sent
iResult = shutdown(ConnectSocket, SD_SEND);
I am new in the field of IPTV. I want to get channels from a certain server, i have the protocol the receiver can connect to this server through.
which means i have its host name and port.
and i have xml files which have the following inside them:
<root>
<category>
<category name="Arabic قنوات عربية">
<movie name="MBC 1" link="http://xtreamip.dynns.com:9000/live/iippdd/5sdgFSDdsg/1480.ts"`
picture="http://178.62.220.69/pic/mbc1.jpg"/>
</category>
</category>
</root>
i am not sure if these files are the same as .M3U files, the ones used to play channels in VLC player.
i would like to ask how am i supposed to get channels in order to be able to retransmit them to many other people using set top boxes?
In general you need to understand if the content owner allows transmission without any cost/fee, first.
Common sense usually applies here - if it is a commercial program or channel the chances are very high that you can't legally just retransmit it. In fact you may find the content is protected with DRM, anyway.
Assuming you have a channel which you are allowed to retransmit, then if your STB's are IP enabled you could simply use the original URL for the movie/channel if that works in the target region.
If your target STB's are cable or satellite then you need a much more involved solution with a proper cable/satalite headend and infrastructure.
I have been studying WSO2 ESB for this particular case:
We got some remote devices that monitor various types of data (temperature, wind, warnings, alarms, panic, Etc.) this devices send data packages to a server by UDP and TCP mostly in binary format (start bit, protocol, values, time, stop bit).
I know that WSO2 ESB can support TCP and UDP Transport by an axis2 server, however all the examples I have found need the data to be in SOAP format (or XML like).
Is there any way to config the Axis2 server to receive the raw packages?
Thanks in advance.
AFAIK this can be achieved by configuring message builders and formatters in the axis2.xml.
Apache Axis2, which is the base for Apache Synpase's SOAP processing, helps users to add their custom message formats through Builders and Formatters.
A Builder accepts a binary data stream and creates an XML message, and a Formatter accepts a XML message and converts it to bytes.
Add the following message builders and formatters to the corresponding sections in conf/axis2.xml (for both Apache Synapse and WSO2 Enterprise Service Bus).
<messageBuilder contentType="text/html"
class="org.wso2.carbon.relay.BinaryRelayBuilder"/>
<messageFormatter contentType="text/html"
class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>
The above example shows how to enable Binary Relay for text/html content type.
You need to repeat the above pair of configurations for each content type you want to be handled at the byte level.
For more information on Setting up Binary-Relay please refer this documentation.
Hope this information will help you.
I am using mochiweb for a server that may also get a TCP connction to which the client sends a simple string (without a newline, the string is not http). Mochiweb uses HTTP sockets and therefore fails to detect this (i dont even get http_error that i can easily get in mochiweb). How can I solve this? Ideally I wish to change mochiweb code to do setopt({packet, http_or_raw}) but this kind of thing does not exist. How would you recommend handling this? my current idea was to modify mochiweb and use erlang:decode_packet, is there a better approach?
EDIT:
More info.
Our server is a websocket service. We wish to allow
people without a ws supporting browser to use it so we use a
flash object to do websocket when the browser can't. The flash object needs to get a flash policy file. Flash
forces the file to be in one of two places:
- port 843 (flash hard coded)
- the port of the ws service
The flash protocol is NOT HTTP based.
Amazon ELB does not allow port forwarding for
most ports below 1024, so we
implemented the flash server in the same port via a patch to
mochiweb (https://github.com/nivertech/mochiweb/tree/ori_flash_170811).
Any advice?
mochiweb isn't designed to handle this use case, if it doesn't look
like HTTP then the connection is closed and it gets discarded. You
would have to go around mochiweb_http for this purpose. I'd suggest
using an alternate port, or making it look like HTTP.
If I really wanted to do what you say you want to do, I would copy
mochiweb_http.erl to some other name (e.g. sometimes_not_http.erl) and
make the appropriate changes to loop/2 and request/2… then instead of
adding mochiweb_http to your supervisor you'd add sometimes_not_http.
It is not necessary or recommended to make modifications to mochiweb
in-place.