Connect to web server using python3 socket - networking

Question

I'm running Python3 running Django on localhost and listening to http://127.0.0.1:8000/
and I want to test it by different ways like request and socket
import requests
res = requests.request("GET", "http://127.0.0.1:8000/")
txt = res.content
print(res)
res.close()
# <Response [200]>
It runs well, and
on my running Django console, it shows [07/Jul/2018 16:04:26] "GET / HTTP/1.1" 200 6744
Now instead requests to socket
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1', 8000))
msg = b'GET / HTTP/1.1'
s.send(msg)
print(s)
s.close()
# <socket.socket fd=1324, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 55312), raddr=('127.0.0.1', 8000)>
No error shows in my program
But I got the following exception on my running Django console
Invalid HTTP_HOST header: 'tony.jhou-1.xxservice.com:8000'. You may need to add 'tony.jhou-1.xxservice.com:8000' to ALLOWED_HOSTS.
[07/Jul/2018 16:08:10] "GET / HTTP/1.1" 400 63266
Traceback (most recent call last):
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 138, in run
self.finish_response()
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 180, in finish_response
self.write(data)
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 274, in write
self.send_headers()
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 332, in send_headers
self.send_preamble()
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 255, in send_preamble
('Date: %s\r\n' % format_date_time(time.time())).encode('iso-8859-1')
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 453, in _write
result = self.stdout.write(data)
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\socketserver.py", line 775, in write
self._sock.sendall(b)
File "D:\emc\ve\lib\site-packages\gevent\_socket3.py", line 459, in sendall
return _socketcommon._sendall(self, data_memory, flags)
File "D:\emc\ve\lib\site-packages\gevent\_socketcommon.py", line 358, in _sendall
timeleft = __send_chunk(socket, chunk, flags, timeleft, end)
File "D:\emc\ve\lib\site-packages\gevent\_socketcommon.py", line 287, in __send_chunk
data_sent += socket.send(chunk, flags)
File "D:\emc\ve\lib\site-packages\gevent\_socket3.py", line 440, in send
return _socket.socket.send(self._sock, data, flags)
ConnectionAbortedError: [WinError 10053] 連線已被您主機上的軟體中止。
[07/Jul/2018 16:08:10] "GET / HTTP/1.1" 500 59
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 54587)
Traceback (most recent call last):
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 138, in run
self.finish_response()
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 180, in finish_response
self.write(data)
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 274, in write
self.send_headers()
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 332, in send_headers
self.send_preamble()
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 255, in send_preamble
('Date: %s\r\n' % format_date_time(time.time())).encode('iso-8859-1')
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 453, in _write
result = self.stdout.write(data)
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\socketserver.py", line 775, in write
self._sock.sendall(b)
File "D:\emc\ve\lib\site-packages\gevent\_socket3.py", line 459, in sendall
return _socketcommon._sendall(self, data_memory, flags)
File "D:\emc\ve\lib\site-packages\gevent\_socketcommon.py", line 358, in _sendall
timeleft = __send_chunk(socket, chunk, flags, timeleft, end)
File "D:\emc\ve\lib\site-packages\gevent\_socketcommon.py", line 287, in __send_chunk
data_sent += socket.send(chunk, flags)
File "D:\emc\ve\lib\site-packages\gevent\_socket3.py", line 440, in send
return _socket.socket.send(self._sock, data, flags)
ConnectionAbortedError: [WinError 10053] 連線已被您主機上的軟體中止。
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 141, in run
self.handle_error()
File "D:\emc\ve\lib\site-packages\django\core\servers\basehttp.py", line 88, in handle_error
super(ServerHandler, self).handle_error()
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 368, in handle_error
self.finish_response()
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 180, in finish_response
self.write(data)
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 274, in write
self.send_headers()
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 331, in send_headers
if not self.origin_server or self.client_is_modern():
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 344, in client_is_modern
return self.environ['SERVER_PROTOCOL'].upper() != 'HTTP/0.9'
TypeError: 'NoneType' object is not subscriptable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\socketserver.py", line 639, in process_request_thread
self.finish_request(request, client_address)
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\socketserver.py", line 361, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\socketserver.py", line 696, in __init__
self.handle()
File "D:\emc\ve\lib\site-packages\django\core\servers\basehttp.py", line 155, in handle
handler.run(self.server.get_app())
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\handlers.py", line 144, in run
self.close()
File "C:\Users\tony.jhou\AppData\Local\Continuum\Anaconda3\lib\wsgiref\simple_server.py", line 35, in close
self.status.split(' ',1)[0], self.bytes_sent
AttributeError: 'NoneType' object has no attribute 'split'
I'm new to socket programming and just start learning networking, I think maybe I should send more request head. Please tell me how to do that, thanks.

Answer 1

Yes, you need to send more data. A minimal HTTP 1.1 request requires a second line that contains a Host header specifying the name or address of the host that should handle the request, and the request header lines must be followed by an empty line to tell the server that the header has been completely received.
In HTTP the end of a line is indicated by a pair of characters, a carriage return and a line feed. In Python strings \r represents a carriage return and \n represents a line feed. (The line feed character is sometimes called a newline character, hence the \n code.) This means that you have to write \r\n to mark the end of each line in the request.
So, what you need is:
msg = b'GET / HTTP/1.1\r\nHost: 127.0.0.1\r\n\r\n'
The final \r\n produces the empty line that indicates the end of the headers.
After you make that change it's likely that your server will report a different error, because your client program closes its socket immediately. That breaks the connection between the client and the server. When the server tries to send its response, it will find that the connection has been broken and it will probably complain about that. To avoid that, change your client to use s.recv to wait for and read the response from the server after it does s.send(). Something like:
while True:
resp = s.recv(8000)
if resp:
print resp
else:
break