I'm trying TCP in Go and trying to make an HTTP call from a TCP client to another HTTP server.
I run another HTTP server locally.
I noticed that whenever the header connection: close is present in the request, the TCP client can read the whole response from the HTTP server.
However, when the header connection: close is not present in the request, only the header part is read from the connection. The body is not read even though the HTTP server returns a body.
Here is the client code
package main
import (
"bufio"
"bytes"
"fmt"
"log"
"net"
)
func main() {
con, err := net.Dial("tcp", "127.0.0.1:8080")
if err != nil {
log.Fatalln(err)
}
defer con.Close()
serverReader := bufio.NewScanner(con)
buf := bytes.Buffer{}
buf.WriteString("GET / HTTP/1.1\n")
buf.WriteString("Content-Type: text/hmtl\n")
//buf.WriteString("Connection: close\n")
buf.WriteString("User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.134 Safari/537.36\n")
buf.WriteString("Accept: */*\n")
buf.WriteString("Host: 127.0.0.1\n")
buf.WriteString("Accept-Encoding: gzip, deflate, br\n\n")
if _, err = con.Write(buf.Bytes()); err != nil {
log.Printf("failed to send the client request: %v\n", err)
}
//Waiting for the server response
for serverReader.Scan() {
fmt.Println(serverReader.Text())
}
}
server code
package main
import (
"log"
"net/http"
"time"
)
func main() {
var handler http.HandlerFunc = func(w http.ResponseWriter, r *http.Request) {
log.Println("request received")
if r.URL.String() == "/path" {
w.WriteHeader(400)
return
}
w.WriteHeader(200)
w.Header().Set("Content-Type", "application/json")
w.Write([]byte("hello"))
w.Write([]byte("hi"))
time.Sleep(time.Millisecond * 10)
w.Write([]byte("how're you"))
return
}
http.ListenAndServe(":8080", handler)
}
Could you help me with reading the response body ven
Related
I am trying to write a simple Unix domain socket based http server in golang. Here is the code:
package main
import (
"fmt"
"log"
"net/http"
"net"
"os"
"encoding/json"
"os/signal"
"syscall"
)
type UnixListener struct {
ServerMux *http.ServeMux
SocketPath string
UID int
GID int
SocketFileMode os.FileMode
}
//////////////////////////////////////////////////////////////////////
// Start the HTTP server with UNIX socket.
//////////////////////////////////////////////////////////////////////
func (l *UnixListener) Start() error {
listener, err := net.Listen("unix", l.SocketPath)
if err != nil {
return err
}
if err := os.Chown(l.SocketPath, l.UID, l.GID); err != nil {
return err
}
if err := os.Chmod(l.SocketPath, l.SocketFileMode); err != nil {
return err
}
go func(){
shutdown(listener)
}()
svr := http.Server{
Handler: l.ServerMux,
}
if err := svr.Serve(listener); err != nil {
return err
} else {
return nil
}
}
//////////////////////////////////////////////////////////////////////
// Shutdown the HTTP server.
//////////////////////////////////////////////////////////////////////
func shutdown(listener net.Listener) {
c := make(chan os.Signal, 1)
signal.Notify(c, syscall.SIGHUP, syscall.SIGINT, syscall.SIGTERM, syscall.SIGQUIT)
s := <-c
listener.Close()
log.Fatalf("[FATAL] Caught a signal. sinal=%s", s)
}
func hello(w http.ResponseWriter, r *http.Request) {
log.Println("Received request %s", r.RequestURI)
fmt.Fprintf(w, "Hello World!\n")
w.Header().Set("ContentType", "application/json")
w.WriteHeader(http.StatusOK)
data := map[string]string { "name" : "satish"}
resp, _ := json.Marshal(data)
if _, err := w.Write(resp); err != nil {
fmt.Printf("Error writing response")
}
}
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/hello", hello)
unixListener := UnixListener{
ServerMux: mux,
SocketPath: "/temp/test.sock",
UID: 501,
GID: 20,
SocketFileMode: 0644,
}
if err := unixListener.Start(); err != nil {
log.Fatalf("[FATAL] HTTP server error: %s", err)
}
}
But when I send the request to this UDS based server like below, the handler does not seem to be invoked, giving 404 not found error. What is the right way to implement UDS based http server and how to send the http request to it?
curl -vvv --unix-socket /temp/test.sock http:/hello
* Trying /Users/sburnwal/Projects/ACI/temp/test.sock:0...
* Connected to hello (/temp/test.sock) port 80 (#0)
> GET / HTTP/1.1
> Host: hello
> User-Agent: curl/7.84.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< Date: Mon, 14 Nov 2022 02:38:11 GMT
< Content-Length: 19
<
404 page not found
* Connection #0 to host hello left intact
It appears it's interpreting http:/hello in your command line as the host name and using that for the header. And then requesting / as the path
Try replacing http:/hello in your command line with http://hello/hello. I'm not sure if the single slash vs double-slash is significant. It just looks suspicious.
curl -vvv --unix-socket /temp/test.sock http://hello/hello
Or change your Go code to have a response handler for "/" instead of just for "/hello"
mux.HandleFunc("/hello", hello)
mux.HandleFunc("/", hello)
I'm trying to run this example of code but it just hangs and doesn't print anything out - any ideas?
package main
import (
"net/http"
"fmt"
)
func Hello(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello World"))
fmt.Println("Hi")
}
func main() {
http.HandleFunc("/", Hello)
if err := http.ListenAndServe(":8080", nil); err != nil {
panic(err)
}
}
It's simple.Open up your browser such as Chrome and go to localhost:8080 or 127.0.1:8080 and you will see the output
Your code listens on a local port 8080, accepts a request from the client, and responds to the client with the corresponding data.
Your responds to client is "Hello World".
w.Write([]byte("Hello World"))
The key is the http.HandleFunc function,http.HandleFunc binds to a route that executes the Hello function whenever you access 127.0.0.1:8080 from your browser .The browser will response you "Hello World" and Program will print "Hi".
When I use http://localhost:8080/login?id=ddfd#vcv.com&pwd=dccccf in postman or use it in android app I am getting 404. On curl I get
{"name":"Miss Moneypenny","email":"ddfd#vcv.com","password":"dccccf","mobile":27,"address":"dscsdacc"}
I am not able to understand what can I do to achieve json output in postman and on other platforms like Apps in ios as well as android when I use this api and also on the browser window.
My Main.go code
func getSession() *mgo.Session {
s, err := mgo.Dial("mongodb://localhost")
if err != nil {
panic(err)
}
return s
}
func main() {
r := httprouter.New()
uc := controllers.NewUserController(getSession())
r.GET("/login", uc.LoginUser)
http.ListenAndServe(":8080", r)
}
code in controller/user.go
type UserController struct {
session *mgo.Session
}
func NewUserController(s *mgo.Session) *UserController {
return &UserController{s}
}
func (uc UserController) LoginUser(w http.ResponseWriter, request *http.Request, params httprouter.Params) {
dump,err :=httputil.DumpRequest(request, true)
if err != nil {
http.Error(w, fmt.Sprint(err), http.StatusInternalServerError)
return
}
fmt.Println("Request Dump:\n", string(dump))
encodedValue := request.URL.Query().Get("id")
pwd := request.URL.Query().Get("pwd")
emailId, err := url.QueryUnescape(encodedValue)
if err != nil {
log.Fatal(err)
return
}
u := models.User{}
if err := uc.session.DB("go-web-dev-db").C("users").FindId(emailId + pwd).One(&u); err != nil {
w.WriteHeader(404)
return
}
uj, err := json.Marshal(u)
if err != nil {
fmt.Println(err)
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK) // 200
fmt.Fprintf(w, "%s\n", uj)
}
code in model/user.go
type User struct {
Name string `json:"name" bson:"name"`
Email string `json:"email" bson:"_id"`
Password string `json:"password" bson:"password"`
Mobile int `json:"mobile" bson:"mobile"`
Address string `json:"address" bson:"address"`
}
After using dump when I am using i am using curl 'http://localhost:8080/login?id=ddfd#vcv.com&pwd=dccccf' I get :-
Request Dump:
GET /login?id=ddfd#vcv.com&pwd=dccccf HTTP/1.1
Host: localhost:8080
Accept: */*
User-Agent: curl/7.69.1
After using dump when I am using i am using http://localhost:8080/login?id=ddfd#vcv.com&pwd=dccccf in postman I get :-
Request Dump:
GET /login?id=ddfd#vcv.com&pwd=dccccf HTTP/1.1
Host: localhost:8080
Accept: */*
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Postman-Token: 8e925738-b8db-4656-9f53-813f4cd53a80
User-Agent: PostmanRuntime/7.24.1
Why can't I run both HTTP and HTTPS from the same golang program?
Here is the code where the two servers are initiated.. The server which is initiated first will run - the second won't.. If they are switched arround the other will run and the other won't..
No errors are returned when running the program, but the requests http://www.localhost or https://secure.localhost times out
// Start HTTP
err_http := http.ListenAndServe(fmt.Sprintf(":%d", port), http_r)
if err_http != nil {
log.Fatal("Web server (HTTP): ", err_http)
}
// Start HTTPS
err_https := http.ListenAndServeTLS(fmt.Sprintf(":%d", ssl_port), "D:/Go/src/www/ssl/public.crt", "D:/Go/src/www/ssl/private.key", https_r)
if err_https != nil {
log.Fatal("Web server (HTTPS): ", err_https)
}
Here is the complete code
package main
import (
"net/http"
"fmt"
"log"
"os"
"io"
"runtime"
// go get github.com/gorilla/mux
"github.com/gorilla/mux"
)
const (
HOST = "localhost"
)
func Handler_404(w http.ResponseWriter, r *http.Request){
fmt.Fprint(w, "Oops, something went wrong!")
}
func Handler_www(w http.ResponseWriter, r *http.Request){
fmt.Fprint(w, "Hello world :)")
}
func Handler_api(w http.ResponseWriter, r *http.Request){
fmt.Fprint(w, "This is the API")
}
func Handler_secure(w http.ResponseWriter, r *http.Request){
fmt.Fprint(w, "This is Secure")
}
func redirect(r *mux.Router, from string, to string){
r.Host(from).Subrouter().HandleFunc("/", func (w http.ResponseWriter, r *http.Request){
http.Redirect(w, r, to, 301)
})
}
func main(){
port := 9000
ssl_port := 443
runtime.GOMAXPROCS(runtime.NumCPU())
http_r := mux.NewRouter()
https_r := mux.NewRouter()
// HTTP 404
http_r.NotFoundHandler = http.HandlerFunc(Handler_404)
// Redirect "http://HOST" => "http://www.HOST"
redirect(http_r, HOST, fmt.Sprintf("http://www.%s:%d", HOST, port))
// Redirect "http://secure.HOST" => "https://secure.HOST"
redirect(http_r, "secure."+HOST, fmt.Sprintf("https://secure.%s", HOST))
www := http_r.Host("www."+HOST).Subrouter()
www.HandleFunc("/", Handler_www)
api := http_r.Host("api."+HOST).Subrouter()
api.HandleFunc("/", Handler_api)
secure := https_r.Host("secure."+HOST).Subrouter()
secure.HandleFunc("/", Handler_secure)
// Start HTTP
err_http := http.ListenAndServe(fmt.Sprintf(":%d", port), http_r)
if err_http != nil {
log.Fatal("Web server (HTTP): ", err_http)
}
// Start HTTPS
err_https := http.ListenAndServeTLS(fmt.Sprintf(":%d", ssl_port), "D:/Go/src/www/ssl/public.crt", "D:/Go/src/www/ssl/private.key", https_r)
if err_https != nil {
log.Fatal("Web server (HTTPS): ", err_https)
}
}
ListenAndServe and ListenAndServeTLS open the listening socket and then loop forever serving client connections. These functions only return on an error.
The main goroutine never gets to the starting the TLS server because the main goroutine is busy waiting for HTTP connections in ListenAndServe.
To fix the problem, start the HTTP server in a new goroutine:
// Start HTTP
go func() {
err_http := http.ListenAndServe(fmt.Sprintf(":%d", port), http_r)
if err_http != nil {
log.Fatal("Web server (HTTP): ", err_http)
}
}()
// Start HTTPS
err_https := http.ListenAndServeTLS(fmt.Sprintf(":%d", ssl_port), "D:/Go/src/www/ssl/public.crt", "D:/Go/src/www/ssl/private.key", https_r)
if err_https != nil {
log.Fatal("Web server (HTTPS): ", err_https)
}
As previously said, both ListenAndServe and ListenAndServeTLS are blocking. That being said, I would agree that examples above are in fact resolving your issue as the point is to be in goroutine BUT same examples are not quite following go idioms.
You should be using error channels here as you want to capture ALL errors that are sent to you instead of having just one error returned back. Here's fully working sample that starts HTTP as HTTPS servers and return errors as channel that's later on used just to display errors.
package main
import (
"log"
"net/http"
)
func Run(addr string, sslAddr string, ssl map[string]string) chan error {
errs := make(chan error)
// Starting HTTP server
go func() {
log.Printf("Staring HTTP service on %s ...", addr)
if err := http.ListenAndServe(addr, nil); err != nil {
errs <- err
}
}()
// Starting HTTPS server
go func() {
log.Printf("Staring HTTPS service on %s ...", addr)
if err := http.ListenAndServeTLS(sslAddr, ssl["cert"], ssl["key"], nil); err != nil {
errs <- err
}
}()
return errs
}
func sampleHandler(w http.ResponseWriter, req *http.Request) {
w.Header().Set("Content-Type", "text/plain")
w.Write([]byte("This is an example server.\n"))
}
func main() {
http.HandleFunc("/", sampleHandler)
errs := Run(":8080", ":10443", map[string]string{
"cert": "/path/to/cert.pem",
"key": "/path/to/key.pem",
})
// This will run forever until channel receives error
select {
case err := <-errs:
log.Printf("Could not start serving service due to (error: %s)", err)
}
}
Hope this helps! :)
func serveHTTP(mux *http.ServeMux, errs chan<- error) {
errs <- http.ListenAndServe(":80", mux)
}
func serveHTTPS(mux *http.ServeMux, errs chan<- error) {
errs <- http.ListenAndServeTLS(":443", "fullchain.pem", "privkey.pem", mux)
}
func main() {
mux := http.NewServeMux()
// setup routes for mux // define your endpoints
errs := make(chan error, 1) // a channel for errors
go serveHTTP(mux, errs) // start the http server in a thread
go serveHTTPS(mux, errs) // start the https server in a thread
log.Fatal(<-errs) // block until one of the servers writes an error
}
The ListenAndServe (and ListenAndServeTLS) functions do not return to their caller (unless an error is encountered). You can test this by trying to print something in between the two calls.
i use
resp, err := http.Get("http://example.com/")
get a http.Response, and i want to exactly write to a http handler, but only http.ResponseWriter, so i hijack it.
...
webConn, webBuf, err := hj.Hijack()
if err != nil {
// handle error
}
defer webConn.Close()
// Write resp
resp.Write(webBuf)
...
Write raw request
But When i hijack, http connection can't reuse (keep-alive), so it slow.
How to solve?
Thanks! Sorry for my pool English.
update 12/9
keep-alive, It keep two tcp connection, and can reuse.
but when i hijack, and conn.Close(), It can't reuse old connection, so it create a new tcp connection when i each refresh.
Do not use hijack, Because once hijack, the HTTP server library will not do anything else with the connection, So can't reuse.
I change way, copy Header and Body, look like reverse proxy (http://golang.org/src/pkg/net/http/httputil/reverseproxy.go), Is works.
Example:
func copyHeader(dst, src http.Header) {
for k, w := range src {
for _, v := range w {
dst.Add(k, v)
}
}
}
func copyResponse(r *http.Response, w http.ResponseWriter) {
copyHeader(w.Header(), r.Header)
w.WriteHeader(r.StatusCode)
io.Copy(w, r.Body)
}
func handler(w http.ResponseWriter, r *http.Response) {
resp, err := http.Get("http://www.example.com")
if err != nil {
// handle error
}
copyResponse(resp, w)
}
It seem that once the connection is closed the keep-alive connection closes as well.
One possible solution would be to prevent the connection from closing until desired, but I'm not sure if that good advise.
Maybe the correct solution involves creating a instance of net.TCPConn, copying the connection over it, then calling .SetKeepAlive(true).
Before running the below example, launch another terminal with netstat -antc | grep 9090.
Routes in example:
localhost:9090/ok is a basic (non-hijacked) connection
localhost:9090 is a hijacked connection, lasting for 10 seconds.
Example
package main
import (
"fmt"
"net/http"
"sync"
"time"
)
func checkError(e error) {
if e != nil {
panic(e)
}
}
var ka_seconds = 10
var conn_id = 0
func main() {
http.HandleFunc("/ok", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "ok")
})
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
conn_id++
fmt.Printf("Connection %v: Keep-alive is enabled %v seconds\n", conn_id, ka_seconds)
hj, ok := w.(http.Hijacker)
if !ok {
http.Error(w, "webserver doesn't support hijacking", http.StatusInternalServerError)
return
}
conn, bufrw, err := hj.Hijack()
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
// Don't forget to close the connection:
time.AfterFunc(time.Second* time.Duration(ka_seconds), func() {
conn.Close()
fmt.Printf("Connection %v: Keep-alive is disabled.\n", conn_id)
})
resp, err := http.Get("http://www.example.com")
checkError(err)
resp.Write(bufrw)
bufrw.Flush()
})
fmt.Println("Listing to localhost:9090")
http.ListenAndServe(":9090", nil)
}
Related issue: http://code.google.com/p/go/issues/detail?id=5645