I want to connect to a server through proxy server that I have.
I am searching for something that is similar to Python's HTTPConnection.set_tunnel, is there something like this in golang?
----edit-----
I'm trying to create a connection to a server that allows self signed certificates & transfers through proxy, will this code work properly?
func CreateProxyClient(serverProxy string, sid string, portProxy int) (*Client, error) {
http.DefaultTransport.(*http.Transport).TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
proxyUrl, _ := url.Parse(serverProxy+":"+strconv.Itoa(portProxy))
tr := &http.Transport{
Proxy: http.ProxyURL(proxyUrl),
}
var netClient = &http.Client{
Timeout: time.Second * 10,
Transport: tr,
}
return &Client{netClient, serverProxy, sid}, nil
}
You can set environment variable HTTP_PROXY for HTTP or HTTPS_PROXY for HTTPS so the default http transport will use it.
Also as an alternative you can create http.Transport by yourself with Proxy field set to http.ProxyURL function call or use you custom implementation.
Example:
proxyURL, _ := url.Parse("http://proxy.example.com:port")
http.DefaultTransport = &http.Transport{
Proxy: http.ProxyURL(proxyURL),
}
// request using proxy
resp, _ := http.Get("https://google.com"))
Related
I got http: server gave HTTP response to HTTPS client in Golang httptest, when testing a get request of https url using httptest server below.
It works fine when I use URL start with "http://"
func testingHTTPClient(handler http.Handler) (*http.Client, func()) {
s := httptest.NewServer(handler)
cli := &http.Client{
Transport: &http.Transport{
DialContext: func(_ context.Context, network, _ string) (net.Conn, error) {
return net.Dial(network, s.Listener.Addr().String())
},
},
}
return cli, s.Close
}
refer from
Code snippet
How to stub requests to remote hosts with Go
Turns out that instead of
s := httptest.NewServer(handler),
I should use
s := httptest.NewTLSServer(handler)
to get a https server.
I am trying to create an HTTP client that can send self-signed HTTP requests through a proxy server.
I tried this code but I am not sure if there is a problem here, will the following code work?
func CreateProxyClient(serverProxy string, sid string, portProxy int) (*Client, error) {
http.DefaultTransport.(*http.Transport).TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
proxyUrl, _ := url.Parse(serverProxy+":"+strconv.Itoa(portProxy))
tr := &http.Transport{
Proxy: http.ProxyURL(proxyUrl),
}
var netClient = &http.Client{
Timeout: time.Second * 10,
Transport: tr,
}
return &Client{netClient, serverProxy, sid}, nil
}
"Is there a problem"? Only if you consider blindly trusting the certificate a problem (that's why it's called InsecureSkipVerify).
The better option is to configure the client to trust the specific certificate that the server is using, so you get MITM protection in addition to encryption.
To do this, get a copy of the server's certificate via a trusted channel (e.g. copy it from the server's filesystem), then add it to the client's CA pool (this will also trust all certificates signed by the server's cert, if applicable).
Here is an example for the test certificate in net/http, which is used by httptest.NewTLSServer:
package main
import (
"crypto/tls"
"crypto/x509"
"fmt"
"log"
"net/http"
"net/http/httptest"
)
// cert is used by httptest.NewTLSServer.
//
// In a real application you're going to want to load the certificate from
// disk, rather than hard-coding it. Otherwise you have to recompile the program
// when the certificate is updated.
var cert = []byte(`-----BEGIN CERTIFICATE-----
MIICEzCCAXygAwIBAgIQMIMChMLGrR+QvmQvpwAU6zANBgkqhkiG9w0BAQsFADAS
MRAwDgYDVQQKEwdBY21lIENvMCAXDTcwMDEwMTAwMDAwMFoYDzIwODQwMTI5MTYw
MDAwWjASMRAwDgYDVQQKEwdBY21lIENvMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCB
iQKBgQDuLnQAI3mDgey3VBzWnB2L39JUU4txjeVE6myuDqkM/uGlfjb9SjY1bIw4
iA5sBBZzHi3z0h1YV8QPuxEbi4nW91IJm2gsvvZhIrCHS3l6afab4pZBl2+XsDul
rKBxKKtD1rGxlG4LjncdabFn9gvLZad2bSysqz/qTAUStTvqJQIDAQABo2gwZjAO
BgNVHQ8BAf8EBAMCAqQwEwYDVR0lBAwwCgYIKwYBBQUHAwEwDwYDVR0TAQH/BAUw
AwEB/zAuBgNVHREEJzAlggtleGFtcGxlLmNvbYcEfwAAAYcQAAAAAAAAAAAAAAAA
AAAAATANBgkqhkiG9w0BAQsFAAOBgQCEcetwO59EWk7WiJsG4x8SY+UIAA+flUI9
tyC4lNhbcF2Idq9greZwbYCqTTTr2XiRNSMLCOjKyI7ukPoPjo16ocHj+P3vZGfs
h1fIw3cSS2OolhloGw/XM6RWPWtPAlGykKLciQrBru5NAPvCMsb/I1DAceTiotQM
fblo6RBxUQ==
-----END CERTIFICATE-----`)
func main() {
pool, err := x509.SystemCertPool()
if err != nil {
log.Fatal(err)
}
if !pool.AppendCertsFromPEM(cert) {
log.Fatal("Cannot append self-signed cert to CA pool")
}
c := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
RootCAs: pool,
},
},
}
s := httptest.NewTLSServer(nil)
res, err := c.Get(s.URL)
if err != nil {
log.Fatal(err)
}
fmt.Println(res.Status)
}
Try it on the playground: https://play.golang.org/p/HsI2RyOd5qd
I need to use a proxy with auth using PostForm method.
If I use something like (simplified):
request, err := http.NewRequest("GET", url.String(), nil)
response, err := client.Do(request)
I can with ease do request.Header.Add("Proxy-Authorization", basicAuth) and it works fine.
But now, I am editing third-party package, and I try to add proxy to the existing code:
proxyStr := "http://proxy.com:8080"
proxyURL, _ := url.Parse(proxyStr)
transport := &http.Transport{
Proxy: http.ProxyURL(proxyURL),
}
bot.Client = &http.Client{
Transport: transport,
}
resp, err := bot.Client.PostForm(method, params)
auth := "username:password"
basicAuth := "Basic " + base64.StdEncoding.EncodeToString([]byte(auth))
resp.Header.Add("Proxy-Authorization", basicAuth)
It does not work, and it fails, to my mind, at string resp.Header.Add("Proxy-Authorization", basicAuth).
Proxy without auth works fine, in this example.
Does anybody know, can I use proxy with auth in this case?
You can create the client once by using the following code. Then substitute your HTTP client in the third-party package.
&http.Client{
Transport: &http.Transport{
Proxy: http.ProxyURL(&url.URL{
Scheme: "http",
User: url.UserPassword("username", "password"),
Host: "146.137.9.45:65233",
}),
},
}
or you can parse the URL as well
url, _ := url.Parse("http://username:password#146.137.9.45:65233")
&http.Client{
Transport: &http.Transport{
Proxy: http.ProxyURL(url),
}
}
You are trying to add a header to a response, which isn't what you send to the server but what you receive. You have to add headers and data to the request, which you have to assemble first and then execute it like this:
data := url.Values{} // the form data
data.Add("foo-key", "some data")
req, err := http.NewRequest("POST","https://yoururl", strings.NewReader(data.Encode()))
auth := "username:password"
basicAuth := "Basic " + base64.StdEncoding.EncodeToString([]byte(auth))
req.Header.Add("Proxy-Authorization", basicAuth)
req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
resp, err := bot.Client.Do(req)
Then you just use the response (resp)
Thanks to all!
I found such a solution (may be it would be useful to someone):
// Uncomment to use proxy with auth
/*
proxyStr := "http://proxy.com:3128"
proxyURL, _ := url.Parse(proxyStr)
auth := "username:password"
basicAuth := "Basic " + base64.StdEncoding.EncodeToString([]byte(auth))
hdr := http.Header{}
hdr.Add("Proxy-Authorization", basicAuth)
transport := &http.Transport{
Proxy: http.ProxyURL(proxyURL),
ProxyConnectHeader: hdr,
}
bot.Client = &http.Client{
Transport: transport,
}
*/
resp, err := bot.Client.PostForm(method, params)
I found, implementation of transport in http2 package doesn't support proxy/socks for http2 client. Is there an simple way to make it work?
Yes I seem to recall something a while ago about the http2 client not working with HTTP/SOCKS5 proxies. Anyway time has moved on and the below works fine (if that's what you're after). Note that if using a custom DialTLS within the Transport, proxied connections still aren't supported.
package main
import (
"log"
"net/http"
"net/url"
)
func main() {
var addressString = "https://www.facebook.com/"
var proxyString = "socks5://127.0.0.1:9150"
req, _ := http.NewRequest("GET", addressString, nil)
tr := &http.Transport{
Proxy: func(req *http.Request) (*url.URL, error) {
proxyURI, err := url.Parse(proxyString)
return proxyURI, err
},
}
// The http client is equipped to handle http/2 connections
hc := &http.Client{Transport: tr}
resp, _ := hc.Do(req)
log.Print(resp)
}
If you're looking to communicate over sockets something like this should work:
socket := "<socket-path>"
// server
sock, err := net.Listen("unix", socket)
go http.Serve(s, nil)
//client
httpc := http.Client{
Transport: &http.Transport{
DialContext: func(_ context.Context, _, _ string) (net.Conn, error) {
return net.Dial("unix", socket)
},
},
}
I want my golang http client to use a proxy only if the user provides the proxy value.
// Make HTTP GET/POST request
proxyUrl, err := url.Parse(proxy)
tr := &http.Transport{
DisableKeepAlives: true,
Proxy: http.ProxyURL(proxyUrl),
}
The above code always tries to connect through proxy even if the proxy variable is blank.
Thanks for the suggestion. Now I am able to make it work. Below is the modified code.
tr := &http.Transport{}
tr.DisableKeepAlives = true
if len(proxy) != 0 { // Set the proxy only if the proxy param is specified
proxyUrl, err := url.Parse(proxy)
if err == nil {
tr.Proxy = http.ProxyURL(proxyUrl)
}
}