this is a textbook example I try to put in use.
I get "BAD" as a result, it means that resp is nil, though I don't know how to fix it.
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
)
func main() {
resp, _ := http.Get("http://example.com/")
if resp != nil {
body, _ := ioutil.ReadAll(resp.Body)
fmt.Println(string(body))
resp.Body.Close()
} else {
fmt.Println("BAD")
}
}
I would recommend to check your Internet settings first, as I cannot reproduce the problem.
Also, error handling in Go is crucial, so change your code to the one below and see if you get any error when making the request.
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
)
func main() {
resp, err := http.Get("http://example.com/")
if err != nil {
log.Fatalln(err)
}
if resp != nil {
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(body))
resp.Body.Close()
} else {
fmt.Println("BAD")
}
}
Related
I'm trying to download a file fro the url to a local file.
I wanted to test whether the requesting url is only file, if it is not a file it should return bad request
Any help could be appreciated
package main
import (
"fmt"
"io"
"net/http"
"os"
)
func main() {
fileUrl := "http://example.com/file.txt"
err := DownloadFile("./example.txt", fileUrl)
if err != nil {
panic(err)
}
fmt.Println("Downloaded: " + fileUrl)
}
// DownloadFile will download a url to a local file.
func DownloadFile(filepath string, url string) error {
// Get the data
resp, err := http.Get(url)
if err != nil {
return err
}
defer resp.Body.Close()
// Create the file
out, err := os.Create(filepath)
if err != nil {
return err
}
defer out.Close()
// Write the body to file
_, err = io.Copy(out, resp.Body)
return err
}
Below is the way to check whether the URL is downloadable or not. Hope this helps someone :)
package main
import (
"fmt"
"io"
"net/http"
"os"
)
func main() {
fileUrl := "http://example.com/file.txt"
err := DownloadFile("./example.txt", fileUrl)
if err != nil {
panic(err)
}
fmt.Println("Downloaded: " + fileUrl)
}
// DownloadFile will download a url to a local file.
func DownloadFile(filepath string, url string) error {
// Get the data
resp, err := http.Get(url)
contentType = resp.Header.Get("Content-Type")
if err != nil {
return err
}
defer resp.Body.Close()
if contentType == "application/octet-stream" {
// Create the file
out, err := os.Create(filepath)
if err != nil {
return err
}
defer out.Close()
// Write the body to file
_, err = io.Copy(out, resp.Body)
return err
} else {
fmt.Println("Requested URL is not downloadable")
return nil
}
}
I'm confused about the HTTP API docs of IPFS。next is part of it。
/api/v0/add
Add a file or directory to IPFS.
//but how to add a directory by golang? it look like so simple but no a example to finish it
#cURL Example
curl -X POST -F file=#myfile "http://127.0.0.1:5001/api/v0/add?quiet=&quieter=&silent=&progress=&trickle=&only-hash=&wrap-with-directory=&chunker=size-262144&pin=true&raw-leaves=&nocopy=&fscache=&cid-version=&hash=sha2-256&inline=&inline-limit=32"
I worked on the same issue and found this working shell solution:
https://community.infura.io/t/ipfs-http-api-add-directory/189/8
you can rebuild this in go
package main
import (
"bytes"
"github.com/stretchr/testify/assert"
"io"
"io/ioutil"
"mime/multipart"
"net/http"
"os"
"strings"
"testing"
)
func TestUploadFolderRaw(t *testing.T) {
ct, r, err := createForm(map[string]string{
"/file1": "#/my/path/file1",
"/dir": "#/my/path/dir",
"/dir/file": "#/my/path/dir/file",
})
assert.NoError(t, err)
resp, err := http.Post("http://localhost:5001/api/v0/add?pin=true&recursive=true&wrap-with-directory=true", ct, r)
assert.NoError(t, err)
respAsBytes, err := ioutil.ReadAll(resp.Body)
assert.NoError(t, err)
t.Log(string(respAsBytes))
}
func createForm(form map[string]string) (string, io.Reader, error) {
body := new(bytes.Buffer)
mp := multipart.NewWriter(body)
defer mp.Close()
for key, val := range form {
if strings.HasPrefix(val, "#") {
val = val[1:]
file, err := os.Open(val)
if err != nil { return "", nil, err }
defer file.Close()
part, err := mp.CreateFormFile(key, val)
if err != nil { return "", nil, err }
io.Copy(part, file)
} else {
mp.WriteField(key, val)
}
}
return mp.FormDataContentType(), body, nil
}
or use https://github.com/ipfs/go-ipfs-http-client which seems to be a better way. I'm working on it and tell you when I know how to use it
Greetings
This is a stripped-down version of the code I want to use for a page-specific web crawler. The idea is to have a function that gets a URL, deals with HTTP and returns a Reader to the response body http.Response:
package main
import (
"io"
"log"
"net/http"
"os"
)
func main() {
const url = "https://xkcd.com/"
r, err := getPageContent(url)
if err != nil {
log.Fatal(err)
}
f, err := os.Create("out.html")
if err != nil {
log.Fatal(err)
}
defer f.Close()
io.Copy(f, r)
}
func getPageContent(url string) (io.Reader, error) {
res, err := http.Get(url)
if err != nil {
return nil, err
}
return res.Body, nil
}
The response body is never closed, which is bad. Closing it inside of the getPageContent function won't work, of course, for io.Copy won't be able to read anything from a closed resource.
My question is rather of general interest than for the specific use case: How can I use functions to abstract the gathering of external resources without having to store the whole resource in a temporary buffer? Or should I better avoid such abstractions?
As pointed out by the user leaf bebop in the comment section, the function getPageCount should return an io.ReadCloser instead of just an io.Reader:
package main
import (
"io"
"log"
"net/http"
"os"
)
func main() {
const url = "https://xkcd.com/"
r, err := getPageContent(url)
if err != nil {
log.Fatal(err)
}
defer r.Close()
f, err := os.Create("out.html")
if err != nil {
log.Fatal(err)
}
defer f.Close()
io.Copy(f, r)
}
func getPageContent(url string) (io.ReadCloser, error) {
res, err := http.Get(url)
if err != nil {
return nil, err
}
return res.Body, nil
}
Another solution is you can directly return the response and close it in main function. In general you can put checks on response StatusCode etc. if new requirements come. Here is the updated code:
package main
import (
"io"
"log"
"net/http"
"os"
)
func main() {
const url = "https://xkcd.com/"
r, err := getPageContent(url)
if err != nil {
log.Fatal(err)
}
defer r.Body.Close()
if r.StatusCode !=http.StatusOK{
//some operations
}
f, err := os.Create("out.html")
if err != nil {
log.Fatal(err)
}
defer f.Close()
io.Copy(f, r.Body)
}
func getPageContent(url string) (*http.Response, error) {
res, err := http.Get(url)
if err != nil {
return nil, err
}
return res, nil
}
I've read about asynchronous response and,
I am trying to return multi response using golang,
but I don't know how I can do that.
my example:
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
)
func main() {
http.HandleFunc("/", multiResponse)
http.ListenAndServe(":8080", nil)
}
func multiResponse(w http.ResponseWriter, r *http.Request) {
go func() {
resp, err := http.Get("https://www.google.com.eg/search?q=hello")
if err != nil {
log.Println(err)
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Println(err)
}
// w.WriteHeader(http.StatusOK)
// w.Write(body)
fmt.Fprintln(w, string(body))
resp.Body.Close()
fmt.Println("done...")
}()
w.WriteHeader(http.StatusAccepted)
}
is there any examples to understand asynchronous response?
Edit:
I meaning about multiple response,
when I receive request from client I will send StatusAccepted immediately,
and in another goroutine other process work, then after finishing send StatusOK with some data.
My very simple code snippet:
import "net/http"
import "io"
import "os"
func main() {
resp, err := http.Get("http://example.com")
if err == nil {
io.Copy(os.Stdout, resp.Body)
}
}
When example.com is charset=iso-8859-1 encoded my output is faulty. Umlauts for example are not displayed correctly:
Hällo Wörld --> H?llo W?rld
Whats a good solution to display umlauts correctly??
You can use the package golang.org/x/net/html/charset to determine the encoding of the website, and also create a reader that converts the content to UTF-8.
Below is a working example:
package main
import (
"io"
"net/http"
"os"
"golang.org/x/net/html/charset"
)
func main() {
resp, err := http.Get("http://example.com")
if err != nil {
os.Exit(1)
}
r, err := charset.NewReader(resp.Body, resp.Header.Get("Content-Type"))
if err != nil {
os.Exit(1)
}
io.Copy(os.Stdout, r)
}