This is a broad question:
How do File descriptor, connection, socket, ip all relate to each other?
Is a file descriptor just a text file? And so when I make a connection to a server, does it mean the server and my computer basically create a file descriptor and we communicate through writing into the file?
Why servers have a limit of file descriptors? When a server opens up too many connections, it could fail with an error: too many files open. Is it because with each connection the server opens up a new text file?
An IP address is a "number" that identifies a machine in a network. So you can say "send this data to IP x.x.x.x" and the network knows which one is the destination machine. There are several ways to assign IP addresses to machines.
A connection is a conceptual link between two machines. It is conceptual because it is not necessary that the machines are directly wired to each other or anything: they are linked "through the network".
A socket is a programming interface that allows a programmer to manage connections, so the programmer can:
Establish (open) a new connection to another machine
Listen for incoming connections from other machines
Read data received through a connectino
Write data through the connection
Discard (close) the connection
A file descriptor is an element (typically a number) that allows the programmer to identify a "stream" of data: files on disk, sockets, pipes, named pipes... basically anything you can read / write bytes in a sequential manner. They are called file descriptors because initially they identified files only.
While reading the FTP protocol specification from : (http://www.pcvr.nl/tcpip/ftp_file.htm). I came across this "FTP differs from the other applications that we've described because it uses two TCP connections to transfer a file". My question is, can FTP have multiple TCP connection for multiple parallel file transfer, for example can I transfer two files in parallel over two TCP connections, is this a matter of customization or standardization?
While it would be theoretically possible to make an FTP server support multiple, concurrent transfers, it's not supported by the RFC or any known implementation.
The block is a simple one in that the control connection, after receiving a transfer request, does not return a final status or accept new commands until the data transfer is completed. Thus, though you could queue up another transfer request it wouldn't actually be processed by the server until the current one completes.
If you want multiple file transfers, just log into the FTP server multiple times using different programs or command-line windows and have each initiate a transfer.
No it can't. FTP uses a control connection for sending commands and a data connection that exists for the duration of the file transfer or directory listing retrieval, that's it.
For more information you can consult RFC 959, which defines the specs of the FTP protocol.
Since Socket is used in hadoop src, guess it's TCP connections to send/recv messages and files, right?
How JVM translate these Socket instances to Linux system calls, is that socket/send, or select/poll?
If it's all about select/poll, I still can get ip/port through relative socket system calls, right?
When I collect all the sys_calls during a terasort (1 master, 3 slaves), I got rare connect/accept/socket sys_calls, and they are without any LAN ip in the sockaddr struct (either 0 or strange ones, IPv4). There are bunches of select/poll sys_calls, is that reasonable?
you could use "netstat --tcp -n" commands to check current tcp connection. i guess Hadoop should use TCP.
you may need use strace to start your Hadoop JVM. The strace will print system calls used by the running application. Usual application use sys_poll to check connection FD's status and use read/write or sendto/recvfrom syscall to receive transmit packet.
?
Right, these system call is only called once during connection setup through sys_socket system call, then application does many polls, transmit or receives operation on that socket.
Does somebody knows how to close a TCP or UDP socket for a single connection via windows command line?
Googling about this, I saw some people asking the same thing. But the answers looked like a manual page of netstat or netsh commands focusing on how to monitor the ports. I don't want answers on how to monitor them (I already do this). I want to close/kill them.
EDIT, for clarification: Let's say that my server listens TCP port 80. A client makes a connection and port 56789 is allocated for it. Then, I discover that this connection is undesired (e.g. this user is doing bad things, we asked them to stop but the connection didn't get dropped somewhere along the way). Normally, I would add a firewall to do the job, but this would take some time, and I was in an emergency situation. Killing the process that owns the connection is really a bad idea here because this would take down the server (all users would lose functionality when we just want to selectively and temporally drop this one connection).
open cmd
type in netstat -a -n -o
find TCP [the IP address]:[port number] .... #[target_PID]# (ditto for UDP)
(Btw, kill [target_PID] didn't work for me)
CTRL+ALT+DELETE and choose "start task manager"
Click on "Processes" tab
Enable "PID" column by going to: View > Select Columns > Check the box for PID
Find the PID of interest and "END PROCESS"
Now you can rerun the server on [the IP address]:[port number] without a problem
If you know the port that you want to free you can sort your netstat list by looking for the specif port like this:
netstat -ano | findstr :8080
Then the pid will appear at the rigth which you can kill with taskkill.
taskkill /pid 11704 /F
Also you may want to look at this question which is specifically for localhost, but I think it is relevant:
Yes, this is possible. You don't have to be the current process owning the socket to close it. Consider for a moment that the remote machine, the network card, the network cable, and your OS can all cause the socket to close.
Consider also that Fiddler and Desktop VPN software can insert themselves into the network stack and show you all your traffic or reroute all your traffic.
So all you really need is either for Windows to provide an API that allows this directly, or for someone to have written a program that operates somewhat like a VPN or Fiddler and gives you a way to close sockets that pass through it.
There is at least one program (CurrPorts) that does exactly this and I used it today for the purpose of closing specific sockets on a process that was started before CurrPorts was started. To do this you must run it as administrator, of course.
Note that it is probably not easily possible to cause a program to not listen on a port (well, it is possible but that capability is referred to as a firewall...), but I don't think that was being asked here. I believe the question is "how do I selectively close one active connection (socket) to the port my program is listening on?". The wording of the question is a bit off because a port number for the undesired inbound client connection is given and it was referred to as "port" but it's pretty clear that it was a reference to that one socket and not the listening port.
For instance you want to free the port 8080
Then, follow these commands.
netstat -ano
taskkill /f /im [PID of the port 8080 got from previous command]
Done!
Use TCPView: http://technet.microsoft.com/en-us/sysinternals/bb897437.aspx
or CurrPorts: https://www.nirsoft.net/utils/cports.html
Alternatively, if you don't want to use EXTERNAL SOFTWARE (these tools don't require an installation by the way), you can simply FIRST run the netstat command (preferably netstat -b ) & then setup Local Security Policy to block the IP address of the user's machine in question, that's what I have been doing with unwanted or even unknown connections - that allows you doing everything WITHOUT ANY EXTERNAL SOFTWARE (everything comes with Windows)...
Try the tools TCPView (GUI) and Tcpvcon (command line) by Sysinternals/Microsoft.
https://learn.microsoft.com/en-us/sysinternals/downloads/tcpview
Use CurrPorts (it's free and no-install): http://www.nirsoft.net/utils/cports.html
/close <Local Address> <Local Port> <Remote Address> <Remote Port> {Process Name}
Examples:
# Close all connections with remote port 80 and remote address 192.168.1.10:
/close * * 192.168.1.10 80
# Close all connections with remote port 80 (for all remote addresses):
/close * * * 80
# Close all connections to remote address 192.168.20.30:
/close * * 192.168.20.30 *
# Close all connections with local port 80:
/close * 80 * *
# Close all connections of Firefox with remote port 80:
/close * * * 80 firefox.exe
It also has a nice GUI with search and filter features.
Note: This answer is huntharo and JasonXA's answer and comment put together and simplified to make it easier for readers. Examples come from CurrPorts' web page.
You can't close sockets without shutting down the process that owns those sockets. Sockets are owned by the process that opened them. So to find out the process ID (PID) for Unix/Linux. Use netstat like so:
netstat -a -n -p -l
That will print something like:
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1879/sendmail: acce
tcp 0 0 0.0.0.0:21 0.0.0.0:* LISTEN 1860/xinetd
Where -a prints all sockets, -n shows the port number, -p shows the PID, -l shows only what's listening (this is optional depending on what you're after).
The real info you want is PID. Now we can shutdown that process by doing:
kill 1879
If you are shutting down a service it's better to use:
service sendmail stop
Kill literally kills just that process and any children it owns. Using the service command runs the shutdown script registered in the init.d directory. If you use kill on a service it might not properly start back up because you didn't shut it down properly. It just depends on the service.
Unfortunately, Mac is different from Linux/Unix in this respect. You can't use netstat. Read this tutorial if you're interested in Mac:
http://www.tech-recipes.com/rx/227/find-out-which-process-is-holding-which-socket-open/
And if you're on Windows use TaskManager to kill processes, and services UI to shutdown services. You can use netstat on Windows just like Linux/Unix to identify the PID.
http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/netstat.mspx?mfr=true
If you know the particular port you want to kill, simply open Command Prompt as admin (on windows) and:
npx kill-port 1900
1900 above is the port number in my case. I use this most times when I want to close a port that React-Native developer tools (and Expo) is running on. Reason being that even after closing the developer window or stopping the server, the port still somehow remains in use.
You can't close sockets on your server without owning those sockets hence you can't actually close the socket down without having code running in the process that owns the server socket.
However, there is another option which is telling the client to close its socket. Sending a RST TCP packet to the port the client is connecting on will cause the client to drop their connection. You can do that with RST scanning using nmap.
http://nmap.org/
I found the right answer to this one. Try TCPView from Sysinternals, now owned by Microsoft. You can find it at http://technet.microsoft.com/en-us/sysinternals/bb897437
wkillcx is a reliable windows command line tool for killing tcp connections from the command line that hasn't been mentioned. It does have issues with servers with large number of connections sometimes though. I sometimes use tcpview for interactive kills but wkillcx can be used in scripts.
you can use program like tcpview from sysinternal. I guess it can help you a lot on both monitoring and killing unwanted connection.
In order to close the port you could identify the process that is listening on this port and kill this process.
CurrPorts did not work for us and we could only access the server through ssh, so no TCPView either. We could not kill the process either, as to not drop other connections. What we ended up doing and was not suggested yet was to block the connection on Windows' Firewall. Yes, this will block all connections that fit the rule, but in our case there was a single connection (the one we were interested in):
netsh advfirewall firewall add rule name="Conn hotfix" dir=out action=block protocol=T
CP remoteip=192.168.38.13
Replace the IP by the one you need and add other rules if needed.
instant/feasible/partial answer : https://stackoverflow.com/a/20130959/2584794
unlike from the previous answer where netstat -a -o -n was used incredibly long list was to be looked into without the name of application using those ports
If you're runnning on Windows 8,`Windows Server 2012 or above with PowerShell v4 of above installed, you can use the below script. This finds the processes associated with the port & terminates them (i.e. kills the process along with its connection; not just the connection).
Code
#which port do you want to kill
[int]$portOfInterest = 80
#fetch the process ids related to this port
[int[]]$processId = Get-NetTCPConnection -LocalPort $portOfInterest |
Select-Object -ExpandProperty OwningProcess -Unique |
Where-Object {$_ -gt 0}
#kill those processes
Stop-Process -Id $processId
Documentation:
Get-NetTCPConnection - PowerShell's NetStat equivalent
Select-Object - Pull back specific properties from an object / remove duplicates
Where-Object - Filter values based on some condition
Stop-Process - PowerShell's TaskKill equivalent
Yes there is possible to close TCP or UDP port there is a command in DOS
TASKKILL /f /pid 1234
I hope this will work for You
when I type wget http://yahoo.com:80 on unix shell. Can some one explain me what exactly happens from entering the command to reaching the yahoo server. Thank you very much in advance.
RFC provide you with all the details you need and are not tied to a tool or OS.
Wget uses in your case HTTP, which bases on TCP, which in turn uses IP, then it depends on what you use, most of the time you will encounter Ethernet frames.
In order to understand what happens, I urge you to install Wireshark and have a look at the dissected frames, you will get an overview of what data belongs to which network layer. That is the most easy way to visualize and learn what happens. Beside this if you really like (irony) funny documents (/irony) have a look at the corresponding RFCs HTTP: 2616 for example, for the others have a look at the external links at the bottom of the wikipedia articles.
The program uses DNS to resolve the host name to an IP. The classic API call is gethostbyname although newer programs should use getaddrinfo to be IPv6 compatible.
Since you specify the port, the program can skip looking up the default port for http. But if you hadn't, it would try a getservbyname to look up the default port (then again, wget may just embed port 80).
The program uses the network API to connect to the remote host. This is done with socket and connect
The program writes an http request to the connection with a call to write
The program reads the http response with one or more calls to read.