Super simple Python 3.x library for accessing http requests - http

Is there a very simple way to access the direct text/bytes of an http request in Python 3.x? Similar to what you would get out of Telnet or something like that. I'm looking for something I can set up to listen on a port, accept the request and directly read what comes across. It wouldn't define it was looking for POST or GET, etc., just the raw values:
Sample value:
GET /index.html/?=request HTTP/1.1
Host: www.example.com
User-Agent: Safari/4.0

The library I was looking for is here. The code in particular I was looking for is this:
import socketserver
class MyTCPHandler(socketserver.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).strip()
print("{} wrote:".format(self.client_address[0]))
print(self.data)
# just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
# Create the server, binding to localhost on port 9999
server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
server.serve_forever()

Related

How can I get the client IP address from a Sockets.TCPSocket object in Julia?

In the documentation for HTTP.jl, the HTTP.serve function has an optional keyword argument tcpisvalid::Function (::TCPSocket) -> Bool which says it can be used to
check accepted connection before processing requests. e.g. to implement source IP filtering, rate-limiting, etc.
How can I get the client IP address from a TCPSocket object?
You can use the function Sockets.getpeername. According to the Sockets docs, the function signature is:
getpeername(sock::TCPSocket) -> (IPAddr, UInt16)
And the description is:
Get the IP address and port of the remote endpoint that the given socket is connected to. Valid only for connected TCP sockets.
So, for example, if you run your simple server with the following code...
HTTP.serve(app, "127.0.0.1", 8081; tcpisvalid=sock -> begin
host, port = Sockets.getpeername(sock)
println("Request from... $host:$port")
true
end)
and then in a separate terminal run...
$ curl http://localhost:8081/
the output from your Julia server will be...
Request from... 127.0.0.1:40958

How to send a HTTP Request using a tun tap interface

I am working on a network proxy project and a newbie to this field. I want to create a tun-tap interface and send an HTTP request through this interface. Here is my approach.
use tun_tap::Iface;
use tun_tap::Mode;
use std::process:Command;
fn cmd(cmd: &str, args: &[&str]) {
let ecode = Command::new(cmd)
.args(args)
.spawn()
.unwrap()
.wait()
.unwrap();
assert!(ecode.success(), "Failed to execte {}", cmd);
}
fn main() {
let iface = Iface::new("tun1",Mode::Tun).unwrap();
cmd("ip", &["addr", "add", "dev", 'tun1', '192.168.0.54/24']);
cmd("ip", &["link", "set", "up", "dev", 'tun1']);
// 192.168.0.53:8000 is my development server created by python3 -m http.server command
let sent = iface.send(b"GET http://192.168.0.53:8000/foo?bar=898 HTTP/1.1").unwrap();
}
But my development server is not receiving any request. And not displaying any error.
A TUN interface sends and receives IP packets. This means that the data you give to iface.send must be an IP packet in order to be delivered. You can see in your code you're not indicating what server you are connecting to because at this layer connections "don't even exist". The IP in the HTTP request happens to be there because HTTP protocol says so, but you must already be connected to the server when you send this information.
In order to send and receive data from a tun interface you'll have to build an IP packet.
Once you can send and receive IP packets, you'll have to implement the TCP protocol on top of that to be able to open a connection to an HTTP server. On this layer (TCP) is where the concept of "connection" appears.
Once you can open and send/receive data over a TCP connection, you'll have to implement the HTTP protocol to be able to talk to the HTTP server, i.e. "GET http://192.168.0.53:8000/foo?bar=898 HTTP/1.1".

How to redirect all mitmproxy HTTP traffic to a specific HTTP server?

I want to redirect all HTTP traffic intercepted by mitmproxy to a particular HTTP server, regardless of where the HTTP traffic was destined too.
I know how to set an upstream proxy server for mitmserver, but in this case I don't want another proxy server, but a (destination) HTTP server instead.
Any ideas?
One way to do this would be inject a python script that overwrites the destination of every request. You add a -s script.py parameter to the mitmproxy/mitmdump command (or call master.addons.add(script.Script('script.py')) if you are using mitmproxy library) and add for example the following into your script:
from mitmproxy import http
def request(self, flow: http.HTTPFlow) -> None:
flow.request.host = 'google.com'
flow.request.path = '/'
... further customize request method, cookies, etc etc as needed

Python, Detect is a URL needs to be HTTPS vs HTTP

Using the python standard library, is there a way to determine if a given web address should use HTTP or HTTPS? If you hit a site using HTTP://.com is there a standard error code that says hey dummy it should be 'HTTPS' not http?
Thank you
Did u make any sort of testing?
The short, prematural answer of your questions is:
Does not exist should use... it's your preference, or a server decision at all, because of redirects.
Some servers does allow only https, and when you call http does return 302 code.
So, if you goal is to load https from a given url, just try it with a fallback to normal http.
I've recommend you to send only HEAD requests, so you can recognize very fast if the https connection is being listening or not. I do not recommend you to check for port 443 (ssl) because sometimes people do not follow that rule and https protocol will ensure that you is under https and not under a fake 443 port.
A bit of code:
#!/usr/bin/env python
#! -*- coding: utf-8 -*-
from urlparse import urlparse
import httplib, sys
def check_url(url):
url = urlparse(url)
conn = httplib.HTTPConnection(url.netloc)
conn.request("HEAD", url.path)
if conn.getresponse():
return True
else:
return False
if __name__ == "__main__":
url = "http://httpbin.org"
url_https = "https://" + url.split("//")[1]
if check_url(url_https):
print "Nice, you can load it with https"
else:
if check_url(url):
print "https didn't load, but you can use http"
if check_url(url):
print "Nice, it does load with http too"

Qt4 QSslSocket triggers HTTP 406 response

I have an application that is working as a proxy using QTcpSocket to intercept HTTP requests. I'm trying to extend that to work for HTTPS as well, but it seems that it's not working how I anticipated. Using QHttp is not possible for me due to the uniqueness of the application.
Currently, I have something like the following:
serverConnection = new QTcpSocket();
serverConnection->setProxy(proxy);
serverConnection->connectToHost(url_hostname, url_port);
serverConnection->write(request.toAscii());
connect(serverConnection, SIGNAL(readyRead()), this, SLOT(readServerData()), Qt::DirectConnection);
I've attempted to do something quite similar using QSslSocket, but unfortunately the outcome isn't what I expected.
serverSConnection = new QSslSocket();
serverSConnection->connectToHostEncrypted(url_hostname, url_port);
if (!serverSConnection->waitForEncrypted()) {
qDebug() << "waitForEncrypted failed";
}
serverSConnection->write(request.toAscii());
connect(serverSConnection, SIGNAL(readyRead()), this, SLOT(readSServerData()), Qt::DirectConnection);
Using the QSslSocket to do the request appears to trigger:
HTTP/1.1 406 Not Acceptable
The request I'm sending through looks something along the lines of the following:
Received Request: "CONNECT www.somesslhost.com:443 HTTP/1.1
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.13) Gecko/2009080315 Ubuntu/9.04 (jaunty) Firefox/3.0.13
Proxy-Connection: keep-alive
Host: www.somesslhost.com
Is there something special about sending an HTTPS request through my application using QSslSocket that I'm missing?
With current information I can only get the idea that you're doing it wrong.
Your method is fine if your client is supposed to send plain HTTP requests to your application and it is going to forward them to HTTPS server.
If your client is instead capable of using HTTPS and uses your application as a standard proxy server, you aren't supposed to connect to the server like that. You get 406 because you're trying to convince the destination server to act as a proxy between you and itself.
You can take a look at Tunneling SSL Through a WWW Proxy, I think. Basically, you should parse the incoming request, establish a ‘raw’ connection to server, respond the client with establishment confirmation and then just forward the packets forth and back.
The whole SSL negotiation is to be done between your client and destination server, proxy server is only supposed to forward encrypted packets without being able to decrypt them.
I'm confused - you say:
I have an application that is working
as a proxy using QTcpSocket to
intercept HTTP requests.
What do you mean by "intercept"? Are you trying to write an HTTP proxy application? Please be more clear & specific, and we can probably help you out.

Resources