I want to write a http proxy with authentication in golang but I couldn't find any example.
Here is what I trid but didn't work: (I get Error parsing basic auth)
server := &http.Server{
Addr: "0.0.0.0:8080",
ReadTimeout: 15 * time.Second,
WriteTimeout: 15 * time.Second,
Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
b, err := httputil.DumpRequest(r, true)
if err == nil {
fmt.Println("dump", string(b))
} else {
fmt.Println("dump error", err)
}
u, p, ok := r.BasicAuth()
if !ok {
fmt.Println("Error parsing basic auth")
w.WriteHeader(401)
return
}
if u != "USERNAME"{
fmt.Printf("Username provided is correct: %s\n", u)
w.WriteHeader(401)
return
}
if p != "PASSWORD" {
fmt.Printf("Password provided is correct: %s\n", u)
w.WriteHeader(401)
return
}
if r.Method == http.MethodConnect {
handleTunneling(w, r)
} else {
handleHTTP(w, r)
}
}),
// Disable HTTP/2.
TLSNextProto: make(map[string]func(*http.Server, *tls.Conn, http.Handler)),
}
log.Fatal(server.ListenAndServe())
I tested the application with firefox and foxyproxy
enter image description here
HTTP Authentication has two headers for providing authentication information: Authorization and Proxy-Authorization.
Authorization header:
The "Authorization" header field allows a user agent to authenticate
itself with an origin server -- usually, but not necessarily, after
receiving a 401 (Unauthorized) response. Its value consists of
credentials containing the authentication information of the user
agent for the realm of the resource being requested.
Proxy-Authorization:
The "Proxy-Authorization" header field allows the client to identify
itself (or its user) to a proxy that requires authentication. Its
value consists of credentials containing the authentication
information of the client for the proxy and/or realm of the resource
being requested.
Request.BasicAuth() is for "Authorization" header, not for "Proxy-Authorization" header.
BasicAuth returns the username and password provided in the request's Authorization header, if the request uses HTTP Basic Authentication. See RFC 2617, Section 2.
For parsing "Proxy-Authorization" header you can copy parseBasicAuth() function from request.go.
func ProxyBasicAuth(header http.Header) (username, password string, ok bool) {
auth := header.Get("Proxy-Authorization")
if auth == "" {
return "", "", false
}
return parseBasicAuth(auth)
}
// parseBasicAuth parses an HTTP Basic Authentication string.
// "Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==" returns ("Aladdin", "open sesame", true).
func parseBasicAuth(auth string) (username, password string, ok bool) {
const prefix = "Basic "
// Case insensitive prefix match. See Issue 22736.
if len(auth) < len(prefix) || !asciiEqualFold(auth[:len(prefix)], prefix) {
return "", "", false
}
c, err := base64.StdEncoding.DecodeString(auth[len(prefix):])
if err != nil {
return "", "", false
}
cs := string(c)
username, password, ok = strings.Cut(cs, ":")
if !ok {
return "", "", false
}
return username, password, true
}
// EqualFold is strings.EqualFold, ASCII only. It reports whether s and t
// are equal, ASCII-case-insensitively.
func asciiEqualFold(s, t string) bool {
if len(s) != len(t) {
return false
}
for i := 0; i < len(s); i++ {
if asciiLower(s[i]) != asciiLower(t[i]) {
return false
}
}
return true
}
// lower returns the ASCII lowercase version of b.
func asciiLower(b byte) byte {
if 'A' <= b && b <= 'Z' {
return b + ('a' - 'A')
}
return b
}
Here is the source code with minor changes. You can also check out packages like elazarl/goproxy and snail007/goproxy.
Related
I saw this code.
go func() {
var err error
if hasCert(s.TLSConfig) {
err = s.ServeTLS(ln, "" /*certFile*/, "" /*keyFile*/)
} else {
err = s.Serve(ln)
}
if err != http.ErrServerClosed {
errs <- err
}
}()
The ServeTLS is located in net/http. Why are there comments in the arguments? If the ServeTLS function receives certificates from the config, why add it to the arguments.
ServeTLS prototype
func (srv *Server) ServeTLS(l net.Listener, certFile, keyFile string) error
Take a look at https://pkg.go.dev/crypto/tls#Config
It configures many things for TLS, but not server key and cert. So it's not actually redundant to specify them to ServeTLS
The goal was to add customer headers to the reverse proxy. Found a way to do it using ModifyResponse. Added 3 response headers mentioned below.
Golang version
go version go1.8 linux/amd64
package middleware
import (
"context"
"net/http"
"net/http/httputil"
"net/url"
"time"
"github.com/Sirupsen/logrus"
"github.com/felixge/httpsnoop"
)
var customHeader1 string
var customHeader2 string
var customHeader3 string
func UpdateResponse(r *http.Response) error {
r.Header.Set("Custom-Header1", customHeader1)
r.Header.Set("Custom-Header2", customHeader2)
r.Header.Set("Custom-Header3", customHeader3)
return nil
}
func ReverseProxy(w http.ResponseWriter, r *http.Request, serverURL *url.URL, path string) {
proxy := httputil.NewSingleHostReverseProxy(serverURL)
ctx, cancel := context.WithTimeout(context.Background(), 65*time.Second)
defer cancel()
go func() {
select {
case <-ctx.Done():
Log.Error(ctx.Err()) // prints "context deadline exceeded"
return
}
}()
proxy.Director = func(req *http.Request) {
targetQuery := r.URL.RawQuery
req.Host = serverURL.Host
req.URL.Scheme = serverURL.Scheme
req.URL.Host = serverURL.Host
req.URL.Path = path
req.URL.RawQuery = targetQuery //+ "&" + req.URL.RawQuery
if _, ok := req.Header["User-Agent"]; !ok {
req.Header.Set("User-Agent", "")
}
}
// Printing it out in log
CustomHeader1 = r.Header.Get("Custom-Header1")
CustomHeader2 = r.Header.Get("Custom-Header2")
CustomHeader3 = r.Header.Get("Custom-Header3")
var Uid string
Uid = r.Header.Get("Resource-Owner-Uid")
Log.Printf("---daily--- %s =-= %s", CustomHeader1, Uid)
Log.Printf("---before daily--- %s =-= %s", CustomHeader3, Uid)
proxy.ModifyResponse = UpdateResponse
Log.Printf("---after-daily--- %s =-= %s", CustomHeader1, Uid)
Log.Printf("---after-daily--- %s =-= %s", CustomHeader3, Uid)
//If Server have already path with it.
if len(serverURL.Path) > 0 {
path = serverURL.Path + path
}
r.URL.Path = path
Log.Formatter = &logrus.JSONFormatter{}
r = r.WithContext(ctx)
m := httpsnoop.CaptureMetrics(proxy, w, r)
Log.Printf(
"host=%s remoteaddr=%s url=%s rawquery=%s Method=%s URL=%s resourceOwnerID=%s resourceOwnerType=%s resourceOwnerUID=%s (StatusCode=%d duration=%s written=%d) customheader1=%s customerheader3=%s respcustomheader1=%s respcustomheader3=%s",
r.Host,
r.RemoteAddr,
r.URL.Path,
r.URL.RawQuery,
r.Method,
r.URL,
r.Header["Resource-Owner-Id"],
r.Header["Resource-Owner-Type"],
r.Header["Resource-Owner-Uid"],
m.Code,
m.Duration,
m.Written,
r.Header["Custom-Header1"],
r.Header["Custom-Header3"],
w.Header().Get("Custom-Header1"),
w.Header().Get("Custom-Header3"),
)
}
// Log
{"level":"info","msg":"---daily--- false =-= uid","time":""}
{"level":"info","msg":"---daily --- false =-= uid","time":""}
{"level":"info","msg":"---after-daily--- false =-= uid","time":""}
{"level":"info","msg":"---after-daily --- false =-= uid","time":""}
time="" level=info msg="host= remoteaddr= url= rawquery= Method=POST URL= resourceOwnerID=[] resourceOwnerType=[] resourceOwnerUID=[] (StatusCode=200 duration=26.098272ms written=78) customheader1=[false] customheader3=[false] respcustomheader1=false respcustomheader3=true"
Expected was respcustomheader3= false
As you can see in the log, custom printed logs are false for the same but printed inside the Log.Printf is true. This value is set in the backend server which is sent back to reverse-proxy in response header and customheader1 is false in backend servers too. Even same data i'm storing in DB is also false from where im reading it.
If I retry again the respcustomheader3 is false.
Is there anything I'm doing wrong? Is my approach correct any guidance would be helpful.
Go version: go1.8.1 windows/amd64
Sample code for HTTP request is:
func (c *Client) RoundTripSoap12(action string, in, out Message) error {
fmt.Println("****************************************************************")
headerFunc := func(r *http.Request) {
r.Header.Add("Content-Type", fmt.Sprintf("text/xml; charset=utf-8"))
r.Header.Add("SOAPAction", fmt.Sprintf(action))
r.Cookies()
}
return doRoundTrip(c, headerFunc, in, out)
}
func doRoundTrip(c *Client, setHeaders func(*http.Request), in, out Message) error {
req := &Envelope{
EnvelopeAttr: c.Envelope,
NSAttr: c.Namespace,
Header: c.Header,
Body: Body{Message: in},
}
if req.EnvelopeAttr == "" {
req.EnvelopeAttr = "http://schemas.xmlsoap.org/soap/envelope/"
}
if req.NSAttr == "" {
req.NSAttr = c.URL
}
var b bytes.Buffer
err := xml.NewEncoder(&b).Encode(req)
if err != nil {
return err
}
cli := c.Config
if cli == nil {
cli = http.DefaultClient
}
r, err := http.NewRequest("POST", c.URL, &b)
if err != nil {
return err
}
setHeaders(r)
if c.Pre != nil {
c.Pre(r)
}
fmt.Println("*************", r)
resp, err := cli.Do(r)
if err != nil {
fmt.Println("error occured is as follows ", err)
return err
}
fmt.Println("response headers are: ", resp.Header.Get("sprequestguid"))
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
// read only the first Mb of the body in error case
limReader := io.LimitReader(resp.Body, 1024*1024)
body, _ := ioutil.ReadAll(limReader)
return fmt.Errorf("%q: %q", resp.Status, body)
}
return xml.NewDecoder(resp.Body).Decode(out)
I will call the RoundTripSoap12 function on the corresponding HTTP client.
When I send a request for the first time I will be getting some headers in the HTTP response, so these HTTP response headers should be sent as-is in my next HTTP request.
You may be interested in the httputil package and the reverse proxy example provided if you wish to proxy requests transparently:
https://golang.org/src/net/http/httputil/reverseproxy.go
You can copy the headers from one request to another one fairly easily - the Header is a separate object, if r and rc are http.Requests and you don't mind them sharing a header (you may need to clone instead if you want independent requests):
rc.Header = r.Header // note shallow copy
fmt.Println("Headers", r.Header, rc.Header)
https://play.golang.org/p/q2KUHa_qiP
Or you can look through keys and values and only copy certain headers, and/or do a clone instead to ensure you share no memory. See the http util package here for examples of this - see the functions cloneHeader and copyHeader inside reverseproxy.go linked above.
How do I disable common name validation inside of a go http client. I am doing mutual TLS with a common CA and hence common name validation means nothing.
The tls docs say,
// ServerName is used to verify the hostname on the returned
// certificates unless InsecureSkipVerify is given. It is also included
// in the client's handshake to support virtual hosting unless it is
// an IP address.
ServerName string
I don't want to do InsecureSkipVerify but I don't want to validate the common name.
You would pass a tls.Config struct with your own VerifyPeerCertificate function, and then you would check the certificate yourself.
VerifyPeerCertificate func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error
If normal verification fails then the handshake will abort before
considering this callback. If normal verification is disabled by
setting InsecureSkipVerify then this callback will be considered but
the verifiedChains argument will always be nil.
You can look here for an example of how to verify a certificate. Iif you look here, you'll see that part of even this verification process includes checking the hostname, but luckily you'll see that it skips it if it's set to the empty string.
So, basically you write your own VerifyPeerCertificate function, convert the rawCerts [][]byte, which I think would look something like:
customVerify := func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error {
roots := x509.NewCertPool()
for _, rawCert := range rawCerts {
cert, _ := x509.ParseCertificate(rawCert)
roots.AddCert(cert)
}
opts := x509.VerifyOptions{
Roots: roots,
}
_, err := cert.Verify(opts)
return err
}
conf := tls.Config{
//...
VerifyPeerCertificate: customVerify,
}
Normal https post like this
pool := x509.NewCertPool()
caStr, err := ioutil.ReadFile(serverCAFile)
if err != nil {
return nil, fmt.Errorf("read server ca file fail")
}
pool.AppendCertsFromPEM(caStr)
tr := &http.Transport{
TLSClientConfig: &tls.Config{
RootCAs: pool,
},
}
client := &http.Client{Transport: tr}
client.Post(url, bodyType, body)
But if your url is use ip(ex. https://127.0.0.1:8080/api/test) or you URL is not match certificate common name, and you want to only ignore certificate common name check, should do like this
pool := x509.NewCertPool()
caStr, err := ioutil.ReadFile(serverCAFile)
if err != nil {
return nil, fmt.Errorf("read server ca file fail")
}
block, _ := pem.Decode(caStr)
if block == nil {
return nil, fmt.Errorf("Decode ca file fail")
}
if block.Type != "CERTIFICATE" || len(block.Headers) != 0 {
return nil, fmt.Errorf("Decode ca block file fail")
}
cert, err := x509.ParseCertificate(block.Bytes)
if err != nil {
return nil, fmt.Errorf("ParseCertificate ca block file fail")
}
pool.AddCert(cert)
tr := &http.Transport{
TLSClientConfig: &tls.Config{
RootCAs: pool,
ServerName: cert.Subject.CommonName, //manual set ServerName
},
}
client := &http.Client{Transport: tr}
client.Post(url, bodyType, body)
I am trying to pass an additional parameter in the request I am trying to send to the Go server -
websocket.create_connection("ws://<ip>:port/x/y?token="qwerty")
The Go server implementation is as follows -
func main() {
err := config.Parse()
if err != nil {
glog.Error(err)
os.Exit(1)
return
}
flag.Parse()
defer glog.Flush()
router := mux.NewRouter()
http.Handle("/", httpInterceptor(router))
router.Handle("/v1/x", common.ErrorHandler(stats.GetS)).Methods("GET")
router.Handle("/v1/x/y", common.ErrorHandler(stats.GetS)).Methods("GET")
var listen = fmt.Sprintf("%s:%d", config.Config.Ip, config.Config.Port)
err = http.ListenAndServe(listen, nil)
if err != nil {
glog.Error(err)
os.Exit(1)
}
}
func httpInterceptor(router http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
startTime := time.Now()
if !auth.Auth(w, req) {
http.Error(w, "Failed authentication", 401)
return
}
router.ServeHTTP(w, req)
finishTime := time.Now()
elapsedTime := finishTime.Sub(startTime)
switch req.Method {
case "GET":
case "POST":
}
})
}
How should I look and parse for the token in the Go server so that the authentication is successful?
Library function
func ParseFromRequest(req *http.Request, keyFunc Keyfunc) (token *Token, err error) {
// Look for an Authorization header
if ah := req.Header.Get("Authorization"); ah != "" {
// Should be a bearer token
if len(ah) > 6 && strings.ToUpper(ah[0:6]) == "BEARER" {
return Parse(ah[7:], keyFunc)
}
}
// Look for "access_token" parameter
req.ParseMultipartForm(10e6)
if tokStr := req.Form.Get("access_token"); tokStr != "" {
return Parse(tokStr, keyFunc)
}
return nil, ErrNoTokenInRequest
}
Call FormValue to get a query parameter:
token := req.FormValue("token")
req is a the *http.Request
An alternative is to call ParseForm and access req.Form directly:
if err := req.ParseForm(); err != nil {
// handle error
}
token := req.Form.Get("token")
The OP asks in a comment how to map "token" to "access_token" for an external package that's looking "access_token". Execute this code before calling the external package:
if err := req.ParseForm(); err != nil {
// handle error
}
req.Form["access_token"] = req.Form["token"]
When the external package calls req.Form.Get("access_token"), it will get the same value as the "token" parameter.
Depending on the way you want to parse the token , if its coming from the form or the URL.
The first answer can be used if the token is being sent from the form while in case of a URL, I would suggest using this. This works for me
token := req.URL.Query().Get("token")
For url query parameters:
mux.Vars(r)["token"]