multipart writer CreateFormFile stuck - http

trying to post multipart/form-data image using go
image file receive from request client and already saved as multipart.File
here my code
func postImage(file multipart.File, url string, filename string) (*http.Response, error) {
r, w := io.Pipe()
defer w.Close()
m := multipart.NewWriter(w)
defer m.Close()
errchan := make(chan error)
defer close(errchan)
go func() {
part, err := m.CreateFormFile("file", filename)
log.Println(err)
if err != nil {
errchan <- err
return
}
if _, err := io.Copy(part, file); err != nil {
errchan <- err
return
}
}()
merr := <-errchan
if merr != nil {
return nil, merr
}
resp, err := http.Post(url, m.FormDataContentType(), r)
if err != nil {
return nil, err
}
defer resp.Body.Close()
return resp, err
}
when i try using it, it stuck at part, err := m.CreateFormFile("file", filename) never return anything
any solution?
Thanks

Use the pipe error to propagate the error back to the main goroutine. Close the write side of the pipe to prevent the client from blocking forever on read. Close the read side of the pipe to ensure that the goroutine exits.
func postImage(file multipart.File, url string, filename string) (*http.Response, error) {
r, w := io.Pipe()
// Close the read side of the pipe to ensure that
// the goroutine exits in the case where http.Post
// does not read all of the request body.
defer r.Close()
m := multipart.NewWriter(w)
go func() {
part, err := m.CreateFormFile("file", filename)
if err != nil {
// The error is returned from read on the pipe.
w.CloseWithError(err)
return
}
if _, err := io.Copy(part, file); err != nil {
// The error is returned from read on the pipe.
w.CloseWithError(err)
return
}
// The http.Post function reads the pipe until
// an error or EOF. Close to return an EOF to
// http.Post.
w.Close()
}()
resp, err := http.Post(url, m.FormDataContentType(), r)
if err != nil {
return nil, err
}
defer resp.Body.Close()
return resp, err
}

Related

Go http request redirect

I am writing an API whichs has to redirect incoming requests to another service, the response must be forwarded to the original requester.
I figured a simple function like below should do the trick, but I was wrong.
I receive the data from my redirected response, however when I send it back to the initial request I receive this response without any data Could not get response. Error: socket hang up
If I try to execute the very same request using postman straight to the redirect URL it works perfectly fine.
func initialAssetsHandler(w http.ResponseWriter, r *http.Request) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
fmt.Println(err)
return
}
resp, err := http.Post(conf.redirectURL, "application/json", bytes.NewReader(body))
if err != nil {
log.Error(err)
}
defer resp.Body.Close()
buf := new(bytes.Buffer)
buf.ReadFrom(resp.Body)
log.Info(string(buf.Bytes()))
var data json.RawMessage
if err = json.NewDecoder(resp.Body).Decode(&data); err != nil {
fmt.Println(err)
return
}
helper.SendJsonRaw(w, 200, data)
}
Here is the SendJsonRaw function:
func SendJsonRaw(w http.ResponseWriter, status int, r json.RawMessage) error {
w.Header().Set(HeaderContentType, MimeApplicationJSON)
w.WriteHeader(status)
_, err := w.Write(r)
return err
}
The r.Body is read by the json decoder up to EOF, then when you pass it to the redirect request it looks empty to the http.Client and therefore it sends no body. You need to retain the content of the body.
For example you can do the following:
func initialAssetsHandler(w http.ResponseWriter, r *http.Request) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
fmt.Println(err)
return
}
var initialAssets TagAssets
if err := json.Unmarshal(&initialAssets, body); err != nil {
if !strings.Contains(err.Error(), "json: invalid use of ,string struct tag, trying to unmarshal") {
helper.SendJsonError(w, http.StatusBadRequest, err)
return
}
}
resp, err := http.Post(conf.redirectURL, "application/json", bytes.NewReader(body))
if err != nil {
log.Error(err)
}
defer resp.Body.Close()
log.Info(resp)
var data json.RawMessage
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
fmt.Println(err)
return
}
helper.SendJsonOk(w, data)
}

request.FormValue is empty in the 'net/http' in Go

I'm trying to create a simple http service with the endpoint to download file to the local system in Go. The link comes in ?uri tag, but when I want to get it I receive an empty string. I tried to parse the form of my request but it didn't help. Here is my code:
func main() {
http.HandleFunc("/download", DownloadHandler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
func DownloadHandler(writer http.ResponseWriter, request *http.Request) {
prsErr := request.ParseForm()
if prsErr != nil{
panic(prsErr)
}
uri := request.FormValue("?uri")
_, _ = writer.Write([]byte(uri))
err := DownloadFile("img.png", uri)
if err != nil {
panic(err)
}
}
func DownloadFile(filepath string, url string) error {
// Create the file
out, err := os.Create(filepath)
if err != nil {
return err
}
defer out.Close()
// Get the data
resp, err := http.Get(url)
if err != nil {
return err
}
defer resp.Body.Close()
// Write the body to file
_, err = io.Copy(out, resp.Body)
if err != nil {
return err
}
return nil
}
I will appreciate any help! Thank you!
invalid at request.FormValue("?uri")
uri := request.FormValue("uri")

Stream HAR events

I have a long running app that I'd like to monitor in real time. HAR files allow me to do this after the fact, but as they are an "archive", they don't allow me to do this in real time.
Is their anyway to stream the "events" array of the HAR file so I can process them as they are generated?
This can be firefox or chrome.
So with some help from https://github.com/mafredri/cdp/tree/master/example/screencast I figured out how to do this in go with chrome's debugger api
What this code doesn't do is tie the request body to the response (where it isn't available), but as I show the RequestID will be consistent so if one serializes event processing (say via locking) one can save the body and use it when the response event is seen.
package main
import (
"context"
"log"
"github.com/mafredri/cdp"
"github.com/mafredri/cdp/cdpcmd"
"github.com/mafredri/cdp/devtool"
"github.com/mafredri/cdp/rpcc"
)
func main() {
if err := run(); err != nil {
panic(err)
}
}
func run() error {
ctx, cancel := context.WithCancel(context.TODO())
defer cancel()
devt := devtool.New("http://localhost:9222")
page, err := devt.Get(ctx, devtool.Page)
if err != nil {
return err
}
conn, err := rpcc.DialContext(ctx, page.WebSocketDebuggerURL)
if err != nil {
return err
}
defer conn.Close()
c := cdp.NewClient(conn)
err = c.Page.Enable(ctx)
if err != nil {
return err
}
loadEventFired, err := c.Page.LoadEventFired(ctx)
if err != nil {
return err
}
_, err = c.Page.Navigate(ctx, cdpcmd.NewPageNavigateArgs("https://github.com/"))
if err != nil {
return err
}
_, err = loadEventFired.Recv()
if err != nil {
return err
}
loadEventFired.Close()
a := &cdpcmd.NetworkEnableArgs{}
a.SetMaxResourceBufferSize(32000)
a.SetMaxTotalBufferSize(96000)
err = c.Network.Enable(ctx, a)
responseEvents, err := c.Network.ResponseReceived(ctx)
requestEvents, err := c.Network.RequestWillBeSent(ctx)
go func() {
defer responseEvents.Close()
for {
ev, err := responseEvents.Recv()
if err != nil {
log.Printf("Failed to receive network event: %v", err)
return
}
log.Printf("requestid = %v, url = %v", ev.RequestID, ev.Response.URL)
}
}()
go func() {
defer requestEvents.Close()
for {
ev, err := requestEvents.Recv()
if err != nil {
log.Printf("Failed to receive network event: %v", err)
return
}
log.Printf("requestid = %v, url = %v", ev.RequestID, ev.Request.URL)
}
}()
select {}
return nil
}

How can you upload files as a []byte in go?

I would like to use golang post request, upload pictures, but I do not want to pass filepath, just want to pass [] byte
The following article are not what I need because they are used os.Open
golang POST data using the Content-Type multipart/form-data
func Upload(url, file string) (err error) {
// Prepare a form that you will submit to that URL.
var b bytes.Buffer
w := multipart.NewWriter(&b)
// Add your image file
f, err := os.Open(file)
if err != nil {
return
}
defer f.Close()
fw, err := w.CreateFormFile("image", file)
if err != nil {
return
}
if _, err = io.Copy(fw, f); err != nil {
return
}
// Add the other fields
if fw, err = w.CreateFormField("key"); err != nil {
return
}
if _, err = fw.Write([]byte("KEY")); err != nil {
return
}
// Don't forget to close the multipart writer.
// If you don't close it, your request will be missing the terminating boundary.
w.Close()
// Now that you have a form, you can submit it to your handler.
req, err := http.NewRequest("POST", url, &b)
if err != nil {
return
}
// Don't forget to set the content type, this will contain the boundary.
req.Header.Set("Content-Type", w.FormDataContentType())
// Submit the request
client := &http.Client{}
res, err := client.Do(req)
if err != nil {
return
}
// Check the response
if res.StatusCode != http.StatusOK {
err = fmt.Errorf("bad status: %s", res.Status)
}
return
}
Since you use
if _, err = io.Copy(fw, f); err != nil {
return
}
You may as well edit your code to:
Add new import: "bytes"
Change the method signature to func Upload(url string, file []byte) (err error)
Use io.Copy(fw, bytes.NewReader(f))

Reading content from http.Get in Golang

I'm having a tough time reading XML from a GET request in Go. I just started to learn Go and haven't found any resources on this topic. What I tried:
response, err := http.Get(url)
if err != nil {
log.Fatal(err)
} else {
defer response.Body.Close()
xml, _ := ioutil.ReadAll(response.Body)
if err != nil {
log.Fatal(err)
}
}
_, err := io.Copy(os.Stdout, response.Body) works but I'd like to store the XML for further processing.
Any help is greatly appreciated.
What you've tried is mostly good. Few things to improve it:
http.Get() returns an http.Response and an optional error. If there is no error, that only means that the HTTP GET operation succeeded, but the server might have responded with an error document. So you still have to check the response HTTP status code.
Also io.ReadAll() also returns an error (besides the read data), don't forget to check that too.
Let's wrap it in a function:
func getXML(url string) (string, error) {
resp, err := http.Get(url)
if err != nil {
return "", fmt.Errorf("GET error: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return "", fmt.Errorf("Status error: %v", resp.StatusCode)
}
data, err := ioutil.ReadAll(resp.Body)
if err != nil {
return "", fmt.Errorf("Read body: %v", err)
}
return string(data), nil
}
Testing / using the above function:
if xmlStr, err := getXML("http://somehost.com/some.xml"); err != nil {
log.Printf("Failed to get XML: %v", err)
} else {
log.Println("Received XML:")
log.Println(xmlStr)
}
Also note that it would be the same to get the content of any other responses, so it's worth not "encoding" the string conversion and return type. This one is more general:
func getContent(url string) ([]byte, error) {
resp, err := http.Get(url)
if err != nil {
return nil, fmt.Errorf("GET error: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("Status error: %v", resp.StatusCode)
}
data, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("Read body: %v", err)
}
return data, nil
}
Using this to get an XML doc:
if data, err := getContent("http://somehost.com/some.xml"); err != nil {
log.Printf("Failed to get XML: %v", err)
} else {
log.Println("Received XML:")
log.Println(string(data))
}

Resources