I've read about asynchronous response and,
I am trying to return multi response using golang,
but I don't know how I can do that.
my example:
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
)
func main() {
http.HandleFunc("/", multiResponse)
http.ListenAndServe(":8080", nil)
}
func multiResponse(w http.ResponseWriter, r *http.Request) {
go func() {
resp, err := http.Get("https://www.google.com.eg/search?q=hello")
if err != nil {
log.Println(err)
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Println(err)
}
// w.WriteHeader(http.StatusOK)
// w.Write(body)
fmt.Fprintln(w, string(body))
resp.Body.Close()
fmt.Println("done...")
}()
w.WriteHeader(http.StatusAccepted)
}
is there any examples to understand asynchronous response?
Edit:
I meaning about multiple response,
when I receive request from client I will send StatusAccepted immediately,
and in another goroutine other process work, then after finishing send StatusOK with some data.
Related
I am having issue in keeping websocket connection alive in go. In my code below, I assign 2 different ports to handle websocket (:8080) and for API request (:3300).
There is no issue when I am using websocket handler directly, but using API handler request and making new external request to the websocker handler, the connection closed directly. Any help is appreciated.
package main
import (
"bytes"
"fmt"
"io/ioutil"
"net/http"
"time"
"github.com/gobwas/ws"
"github.com/gobwas/ws/wsutil"
)
func main() {
go websocket()
http.HandleFunc("/ws", func(rw http.ResponseWriter, r *http.Request) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
http.Error(rw, err.Error(), http.StatusInternalServerError)
return
}
url := fmt.Sprintf("http://127.0.0.1:8080?%s", r.URL.RawQuery)
req, err := http.NewRequest(r.Method, url, bytes.NewReader(body))
if err != nil {
fmt.Println(err)
panic(err)
}
req.Header = make(http.Header)
for h, val := range r.Header {
req.Header[h] = val
}
httpClient := &http.Client{Timeout: time.Second * 10}
httpClient.Do(req)
})
http.ListenAndServe(":3300", nil)
}
func websocket() {
http.ListenAndServe(":8080", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
conn, _, _, err := ws.UpgradeHTTP(r, w)
if err != nil {
fmt.Println(err)
return
}
go func() {
defer conn.Close()
for {
msg, op, err := wsutil.ReadClientData(conn)
if err != nil {
fmt.Println(err)
return
}
err = wsutil.WriteServerMessage(conn, op, msg)
if err != nil {
fmt.Println(err)
return
}
}
}()
}))
}
The code in the question connects to the websocket endpoint using an HTTP request. Upgrade fails as a result.
Use the standard library reverse proxy to proxy the request.
A simpler approach is to is to call the websocket handler directly. Move the handler to a top-level function:
func handleWS(w http.ResponseWriter, r *http.Request) {
conn, _, _, err := ws.UpgradeHTTP(r, w)
if err != nil {
fmt.Println(err)
return
}
go func() {
defer conn.Close()
for {
msg, op, err := wsutil.ReadClientData(conn)
if err != nil {
fmt.Println(err)
return
}
err = wsutil.WriteServerMessage(conn, op, msg)
if err != nil {
fmt.Println(err)
return
}
}
}()
}
Use the handler in both servers.
func main() {
go websocket()
http.HandleFunc("/ws", handleWS)
http.ListenAndServe(":3300", nil)
}
func websocket() {
http.ListenAndServe(":8080", http.HandlerFunc(handleWS))
}
I try to send the POST a request into my web server, but when i try to get response body occurs error. Also I tried to send request with Postman and everything worked fine. Response from server is JSON data which give some information about loaded picture.
package main
import (
"fmt"
"bytes"
"mime/multipart"
"os"
"path/filepath"
"io"
"net/http"
"io/ioutil"
)
func main() {
url := "localhost:6000/..."
method := "POST"
payload := &bytes.Buffer{}
writer := multipart.NewWriter(payload)
file, errFile1 := os.Open("/home/...")
defer file.Close()
part1, errFile1 := writer.CreateFormFile("Image",filepath.Base("/home/..."))
_, errFile1 = io.Copy(part1, file)
if errFile1 !=nil {
fmt.Println(errFile1)
}
err := writer.Close()
if err != nil {
fmt.Println(err)
}
client := &http.Client {
}
req, err := http.NewRequest(method, url, payload)
if err != nil {
fmt.Println(err)
}
req.Header.Set("Content-Type", writer.FormDataContentType())
res, err := client.Do(req)
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
fmt.Println(string(body))
}
This is your line #42-43 where the error occurs:
res, err := client.Do(req)
defer res.Body.Close()
If client.Do() returns an error, res may be nil, and so res.Body.Close() is a runtime panic. You have to first check the error, and only proceed to close the body if error is nil:
res, err := client.Do(req)
if err != nil {
// Handle error and return:
return
}
defer res.Body.Close()
See related: Do we need to close the response object if an error occurs while calling http.Get(url)?
Also: Is resp.Body.Close() required if I don't need response?
Hey there I would like to parse a http.resquest two times like below. When I parsed the Body the first time, the body will be closed. I need some help/hint what the best way is to handle this, do I have to create a copy of the request or is there a better way?
func myfunc(w http.ResponseWriter, req *http.Request) {
err := parseBody(req, &type1){
.....
}
err := parseBody(req, &type2){
.....
}
}
Thanks for help
It's true that you can read body only once and it's ok because to parse body more than once you don't have to read it more that one time. Let's consider simple example:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
)
type RequestData1 struct {
Code string `json:"code"`
Status string `json:"status"`
}
type RequestData2 struct {
Status string `json:"status"`
Message string `json:"message"`
}
func main() {
http.HandleFunc("/post", post)
http.ListenAndServe(":8080", nil)
}
If we use this code:
func post(w http.ResponseWriter, r *http.Request) {
body1, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err)
}
rd1 := RequestData1{}
err = json.Unmarshal(body1, &rd1)
if err != nil {
panic(err)
}
body2, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err)
}
rd2 := RequestData2{}
err = json.Unmarshal(body2, &rd2)
if err != nil {
panic(err) // panic!!!
}
fmt.Printf("rd1: %+v \nrd2: %+v", rd1, rd2)
w.WriteHeader(http.StatusOK)
w.Write([]byte(`Look into console.`))
}
we will have panic: http: panic serving [::1]:54581: unexpected end of JSON input
but with next code:
func post(w http.ResponseWriter, r *http.Request) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err)
}
rd1 := RequestData1{}
err = json.Unmarshal(body, &rd1)
if err != nil {
panic(err)
}
rd2 := RequestData2{}
err = json.Unmarshal(body, &rd2)
if err != nil {
panic(err)
}
fmt.Printf("rd1: %+v \nrd2: %+v", rd1, rd2)
w.WriteHeader(http.StatusOK)
w.Write([]byte(`Look into console.`))
}
all works! You can test it by issuing request:
curl -X POST 'http://localhost:8080/post' \
-H 'Content-Type: application/json' -d '{"code":"200", "status": "OK", "message": "200 OK"}'
Result will be:
rd1: {Code:200 Status:OK}
rd2: {Status:OK Message:200 OK}
When you read request.Body, you're reading the stream from the client (e.g. web browser). The client only sends the request once. If you want to parse it multiple times, read the whole thing out into a buffer (e.g. a []byte) and then parse that as many times as you want. Just be mindful of the potential memory use of many concurrent requests with large payloads, as you'll be holding the full payload in memory at least until you're fully done parsing it.
This is a stripped-down version of the code I want to use for a page-specific web crawler. The idea is to have a function that gets a URL, deals with HTTP and returns a Reader to the response body http.Response:
package main
import (
"io"
"log"
"net/http"
"os"
)
func main() {
const url = "https://xkcd.com/"
r, err := getPageContent(url)
if err != nil {
log.Fatal(err)
}
f, err := os.Create("out.html")
if err != nil {
log.Fatal(err)
}
defer f.Close()
io.Copy(f, r)
}
func getPageContent(url string) (io.Reader, error) {
res, err := http.Get(url)
if err != nil {
return nil, err
}
return res.Body, nil
}
The response body is never closed, which is bad. Closing it inside of the getPageContent function won't work, of course, for io.Copy won't be able to read anything from a closed resource.
My question is rather of general interest than for the specific use case: How can I use functions to abstract the gathering of external resources without having to store the whole resource in a temporary buffer? Or should I better avoid such abstractions?
As pointed out by the user leaf bebop in the comment section, the function getPageCount should return an io.ReadCloser instead of just an io.Reader:
package main
import (
"io"
"log"
"net/http"
"os"
)
func main() {
const url = "https://xkcd.com/"
r, err := getPageContent(url)
if err != nil {
log.Fatal(err)
}
defer r.Close()
f, err := os.Create("out.html")
if err != nil {
log.Fatal(err)
}
defer f.Close()
io.Copy(f, r)
}
func getPageContent(url string) (io.ReadCloser, error) {
res, err := http.Get(url)
if err != nil {
return nil, err
}
return res.Body, nil
}
Another solution is you can directly return the response and close it in main function. In general you can put checks on response StatusCode etc. if new requirements come. Here is the updated code:
package main
import (
"io"
"log"
"net/http"
"os"
)
func main() {
const url = "https://xkcd.com/"
r, err := getPageContent(url)
if err != nil {
log.Fatal(err)
}
defer r.Body.Close()
if r.StatusCode !=http.StatusOK{
//some operations
}
f, err := os.Create("out.html")
if err != nil {
log.Fatal(err)
}
defer f.Close()
io.Copy(f, r.Body)
}
func getPageContent(url string) (*http.Response, error) {
res, err := http.Get(url)
if err != nil {
return nil, err
}
return res, nil
}
I was trying to use the httptest package in golang. I found out something I don't understand. Here is the code:
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"net/http/httptest"
)
func main() {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello1"))
}))
ts.Close()
ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello2"))
}))
ts.Close()
ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(100)
w.Write([]byte("Hello3"))
}))
res, err := http.Get(ts.URL)
if err != nil {
log.Fatal(err)
}
greeting, err := ioutil.ReadAll(res.Body)
res.Body.Close()
if err != nil {
log.Fatal(err)
}
ts.Close()
fmt.Printf("%s", greeting)
}
In this code example, I was trying to open and close httptest servers several times. Somehow it caused deadlock in The Go Playground. I tried on my own environment (Go version: go1.7.4 darwin/amd64) and it caused hanging without responding at all.
My question is: Why w.WriteHeader(100) caused deadlock but w.WriteHeader(200) doesn't? Is it the bug from the core library of Golang or just I misunderstood some usage? Tks!
If you slightly modify code:
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"net/http/httptest"
)
func main() {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello1"))
}))
ts.Close()
ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello2"))
}))
ts.Close()
ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(100)
w.Write([]byte("Hello3"))
}))
fmt.Println("before get") ///// . <----
res, err := http.Get(ts.URL)
fmt.Println("after get") ///// . <----
if err != nil {
log.Fatal(err)
}
greeting, err := ioutil.ReadAll(res.Body)
res.Body.Close()
if err != nil {
log.Fatal(err)
}
ts.Close()
fmt.Printf("%s", greeting)
}
and run it - you'll see only first line.
That means Go http client want more data from you. So it hangs on line
res, err := http.Get(ts.URL)
and cannot get to the ts.Close() below.
Next - lets modify a test, so it will close a connection and this way release a clients waiting lock:
ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(100)
w.Write([]byte("Hello3"))
hj, _ := w.(http.Hijacker)
conn, _, _ := hj.Hijack()
conn.Close()
}))
I close connection explicitly but this way you get both check strings and test finishes ok. Try it.
Of course it's a HTTP protocol violation so I get an error:
Get http://127.0.0.1:54243: net/http: HTTP/1.x transport connection broken: unexpected EOF