I'm trying to read the data from HTTP stream with TCP inside Linux kernel. I'm able to get most data from skb_buff here. However, if the server pushes data to the client without requests, the data won't be copied to user space so that I cannot find it any more.
Using Wireshark, I can find the additional data as a single packet normally. Therefore I think these data should go into kernel somewhere, even if they are not requested by the user space. Is it possible to find all the data when they are retrieved from Network Interface like Wireshark did? If so, where should I find them?
Thanks! Any ideas are appreciable.
EDIT: It should be different from another similar question. I even couldn't get the skb instance containing the data I need because the client didn't request it. Therefore such data won't be copied into user space. Thanks for pointing that question to me but I still need to find the correct skb instance first. I suspect I should catch the data somewhere when the data is retrieved from network interface.
Related
I'm supposed to duplicate a binary TCP Stream.
So I set up a NiFi 1.9.0 server, put in a ListenTCP processor and a PutTCP processor, configured the proper IPs and Ports and connected them.
So far so good, the packets were received by the ListenTCP processor and also forwareded by the PutTCP processor.
But NiFi seems to mess around with the data somehow, the sent packets aren't exactly the same as received. I expected NiFi to just forward everything 1:1 but something is happening and I cannot find out what.
I've been playing around with the Character Set, Max Batch Size and Batching Message Delemiter settings on the ListenTCP processor and also with the Outgoing Message Delemiter and Character Set on the PutTCP processor.
I also messed around with a MergeContent processor but didn't get it to work properly.
Here you can see the difference between received (red) and sent data (captured using tcpflow).
Link to picture
Another problem is that I don't really know the data I'm processing, it says in the documentation:
These log files are in the machine-readable binary format that is described by the XML file called ebm.xml.
and
The streamed events are in the TCP-based binary format.
I do have access to ebm.xml file, but not sure how I can make use of it.
Anyone an idea how I can get NiFi to simply forward everything?
I'm new to NiFi, so I might have missed some possibilites...
The ListenTCP processor reads data from the stream using a new-line character as a logical message separator. For example, if the stream had:
<chunk1><new-line><chunk2><new-line><chunk3><new-line>
It would result in reading chunk1, chunk2, and chunk3 into an internal queue.
When it writes them back out it uses the outgoing message delimiter. So the outgoing flow file would be:
<chunk1><outgoing-delim><chunk2><outgoing-delim><chunk3><outgoing-delim>
Unfortunately it is more geared towards receiving textual data such as logs which are typically line-delimited. The chunks should be passing through unaltered as byte[], but typically binary data wouldn't have these logical new-line boundaries, so I'm not sure how well it works for that.
Is there a way to preserve metadata or any equivalent data while sending and receiving mbuf to and from kni interface in dpdk??
Thanks in advance.
If you are asking for an out of the box solution, then no there is no such functionality. Only length and frame content get copied to/from the host kernel. For more information please see kni_net_tx/rx() functions in kni_net.c:
http://dpdk.org/browse/dpdk/tree/lib/librte_eal/linuxapp/kni/kni_net.c#n202
But sure, you can add whatever you want to each mbuf passing from/to kernel. The only requirement is that you have to change both user-space (i.e. DPDK lib/librte_kni) and kernel-space (i.e. lib/librte_eal/linuxapp/kni) to support the metadata you need.
I just wanted to write a program to show the server-list of teknomw3. I'm not familiar with wireshark or packet-sniffing. I just tried using wireshark and got information.
Below is one of information sent by the master-server to mw3,
I can clearly see the data sent by the master-server, that contains the data of a dedicated-server.
here's the packet info mw3 sent, before getting the server list,
Is there a way to create my own program to get the server list? I know C#, Java. but not an expert.
From where should I start?
Closest resemblance to my question was posted here.
However, I am still having troubles understanding how a TCP data stream creates "messages" if you will. Arn't messages things that happen after xx amount of time? A TCP stream is a constant flow of data.
For example, a game server running at 30hz. If messages are sent out at 30 times a second, they must be using something internally to do that, what is it?
I was wondering, is it possible to trace a packet that you send out of your own computer? The idea here would be to build something to protect your data. The packet sent out containing your password and other vital information is open to rerouting by a hacker. I want to know if it is possible (and if so, how I might go about approaching this) to trace the intermediate and/or final destinations of a certain packet, and then have them sent back to my computer for verification.
I would appreciate any help you guys could give on this matter.
There is no mechanism to do what you want. The packet itself might reach its "destination" just fine, only to be further re-directed elsewhere. Consider it like mailing an envelope -- whoever receives it is free to photocopy the contents a thousand times and send the copies to newspapers, tabloids, and telephone poles all over the world.
Same story with data -- once it leaves your computer, you have to trust the remote endpoint to not do anything harmful with it.
TLS and openPGP can go a long way to preventing third-parties from reading or modifying your data while it is in transit, but they cannot make sure the remote peer only handles your data with care.
No, not with ISO OSI, and there are other approaches to protect youre data