So the problem comes down to -
Redirecting the output of my $ python script.py to a TCP Port using nc.
Capturing the data remotely listening to the Port.
Now when I remotely listen this port number's stream using nc, the data is being redirected properly.
When I add to Data Inputs to listen to the port in Splunk, I don't get any data(there is not syslog or logs from tornado).
Also, there is nothing added to the data sources when I hit search tab in Splunk (all visual).
When I use a file and redirect the port output to this file using nc and then use that file as a source for Splunk it works perfectly.
Now as per point 6, I would have get my results by redundantly storing my log in a temp file and Splunk which is not desired. And as per point 4, my data is not showing up directly when Splunk listens to my TCP port.
Please help.
So, the problem was - When the port number is being listened by nc itself on the splunk computer, it wasn't coming as a data source in splunk itself. Which certainly makes sense as the data would be lost to nc before it reaches splunk.
This shows a loophole in splunk which could be corrected. They do point out busy ports but they should also point our multiple listeners at that port.
My question's closed.
Related
I write a simple server application. In that application, I created a server socket and put it into the listen state with listen call.
After that, I did not write any code to accept the incoming connection request. I simply waited for the termination with pause call.
I want to figure out practically that how many bytes are buffered in the server side if the connection is not accepted. Then I want to validate the number with the theory of the TCP.
To do that,
First, I started my server application.
Then I used "dd" and "netcat" to send the data from client to server. Here is the command:
$> dd if=/dev/zero count=1 bs=100000 | nc 127.0.0.1 45001
Then I opened wireshark and wait for the zero-window message.
From the last properly acknowledged tcp frame. the client side can successfully send 64559 byte data to the server.
Then I execute the above dd-netcat command to create another client and send data again.
In this case, I got the following wireshark output:
From the last successfully acknowledged tcp frame, I understand that the client application can successfully sent 72677 bytes to the server.
So, it seems that the size of the related buffer can change in runtime. Or, I misinterpret the output of the wireshark.
How can I understand the size of the related receive buffer? What is the correct name to refer that receive buffer in terminology? How can I show the default size of the related receive buffer?
Note that the port number of the tcp server is "45001".
Thank you!
I am running the following command with the find-SCU tool from the OFFIS DICOM toolkit (dcmtk):
movescu -k 0010,0020="PAT004" ip_adress 104 -aec serverAET -aet myAET --study -ll debug -od data
And I keep getting the error.
The association seem to have worked well but the actual c-move seems to fail at the moment of the transfer
message of the error
The screenshot tells that you can successfully establish the connection, but the server aborts after receiving the request.
You missed to specify the mandatory key QueryRetrieveLevel (0008,0052).
add
-k 0008,0052="PATIENT"
to your command, and it should work.
However, moving means, that the server (MOVE-SCP) is prompted to transfer the images matched by the request to a destination application entity. This must be specified by providing the AET of that system:
-aem <AET of the destination>
This frequently fails due to one of these reasons:
the move destination AE title is resolved to an IP-address and port. This is achieved through the C-MOVE-SCP's configuration.
A Storage SCP has to listen for images transferred in the scope of the C-MOVE, its IP, AET and port have to match the MOVE-SCP's configuration for the Move destination AE title.
I want to write an application layer sniffer (SMTP/ftp/http).
Based on my searchs, first (and perhaps hardest!) step is to reassemble the tcp stream of the sniffed connections.
Indeed, what I need is something like the "follow TCP stream" option of wireshark, but I need a tool which do it on live interface and automatically. As I know, Tshark can extract TCP streams data from the saved pcap files automatically (link) but not from live interfaces. Can Tshark do it on live interfaces???
As I know, TCPflow can do exactly what I want, however, it can not handle IP defragmentation and SSL connections (I want to analyse the SSL content in the case I have the server private key).
Finally, I also try bro network monitor. Although it provides the list of TCP connections (conn.log), I was not able to get TCP connections contents.
Any suggestion about mentioned tools or any other useful tool is welcome.
Thanks in advance, Dan.
perl Net::Inspect library might help you. It also comes with a tcpudpflow which can write tcp and udp flows into separate files, similar to tcpflow. It works on pcap files or can do live captures. The library handles IP fragmenting. It also comes with a httpflow tool to extract HTTP requests and responses (including decompression, chunked encoding..). It does not currently handle SSL.
As the author of this library I don't think that extracting TCP flows is the hardest part, the HTTP parser (exluding decompression, including chunked mode) is nearly twice as big than IP and TCP combined.
This example works for reassembling application data of a single protocol:
tshark -Y "tcp.dstport == 80" -T fields -d tcp.port==80,echo -e echo.data
It captures live http data, reassembles it, and outputs it as raw hex.
I can add a small script to parse the hex into ascii if you like.
I want to analyse the SSL content in the case I have the server private key
TL;DR: This can't be done with a capturing tool alone.
Why not: Because each SSL session generates a new secret conversation key, and you can't decrypt the session without this key. Having the private server key is not enough. The idea behind this is that if someone captures your SSL traffic, saves it, and then a year later he "finds" the private server key, then he still won't be able to decrypt your traffic.
I currently have a Java Applet running on my web page that communicates to a display pole via COM1. However since the Java update I can no longer run self-signed Java Applets and I figure it would just be easier to send an AJAX request back to the server and have the server send a response to a TCP port on the computer...the computer would need a TCP > COM virtual adapter. How do I install a virtual adapter to go from a TCP port to COM1?
I've looked into com0com and that is just confusing as hell to me, and I don't see how to connect any ports to COM1. I've tried tcp2com but it doesn't seem to install the service in Windows 7 x64. I've tried com2tcp and the interface seems like it WOULD work (I haven't tested), but I don't want an app running on the desktop...it needs to be a service that runs in the background.
So to summarize how it would work:
Web page on comp1 sends AJAX request to server
Server sends text response to comp1 on port 999
comp1 has virtual COM port listening on port 999, sends data to COM1
pole displays data
EDIT: I'm using Win 7 x64 and tcp2com doesn't work as a service. I tried using srvany but I get an error stating that the application started then stopped. If I use powershell and pass the tcp2com as an argument, it doesn't quit but it also doesn't run. So I nixed the whole 'service' deal and put the command: powershell -windowstyle hidden "tcp2com --test tcp/999 com1" and it works...sort of. The characters that get sent are all effed. I can write "echo WTF > COM1" on another computer which has COM2TCP (different vendor) and it'll come up as a single block on the POS display pole. However if I use COM2TCP on both the server and client machines, everything works fine...but that's only a trial version and it costs several hundred dollars! On another note, is there a way to send the raw text over IP without having to use another Virtual COM > IP adapter on another computer? Sort of like how curl works but different...?
After somewhat of an exhaustive search, I came across a program called 'piracom'. It's a very simple app that lets you specify port settings for the express purpose of connecting a serial port to an listening port over the network. So this is IP > Serial. For Serial > IP I used HW-VSP3-Single as even on the piracom website it said it's compatible! I've tested and it works!
I just put a shortcut to piracom in the startup folder of my user account; the app runs off of a .ini that it updates every time you make a change...so if you run the server and hide it, on the next reboot of the pc it'll start up running and hidden with all prior settings. Easy.
Now it's a matter of installing HW-VSP3 on the server and making a method on the Rails app which will write to the virtual COM port. The only issue I can see right now is that writing echo \14Test This! > COM3 actually prints the \14...if I do that in my Java applet, it sends the "go to beginning" signal.
Addendum 1: The \14 problem was fixed by using the serialport gem for RoR. I created a method in a controller that returned head :no_content and then send data to the COM port. Calls to this method were made via jQuery's $.Ajax, using "HEAD" HTTP method. Apparently though I had to add the GET verb in Rails routes because the HEAD option isn't supported for some gimpy reason.
Addendum 2: Some garbage data was being sent to the display pole at the end of the string...turns out I needed to turn off the "NVT" option in HW-VSP3. Also keep in mind that firewalls need to be modified to allow communication.
when I type wget http://yahoo.com:80 on unix shell. Can some one explain me what exactly happens from entering the command to reaching the yahoo server. Thank you very much in advance.
RFC provide you with all the details you need and are not tied to a tool or OS.
Wget uses in your case HTTP, which bases on TCP, which in turn uses IP, then it depends on what you use, most of the time you will encounter Ethernet frames.
In order to understand what happens, I urge you to install Wireshark and have a look at the dissected frames, you will get an overview of what data belongs to which network layer. That is the most easy way to visualize and learn what happens. Beside this if you really like (irony) funny documents (/irony) have a look at the corresponding RFCs HTTP: 2616 for example, for the others have a look at the external links at the bottom of the wikipedia articles.
The program uses DNS to resolve the host name to an IP. The classic API call is gethostbyname although newer programs should use getaddrinfo to be IPv6 compatible.
Since you specify the port, the program can skip looking up the default port for http. But if you hadn't, it would try a getservbyname to look up the default port (then again, wget may just embed port 80).
The program uses the network API to connect to the remote host. This is done with socket and connect
The program writes an http request to the connection with a call to write
The program reads the http response with one or more calls to read.