Using FileServer to serve my single html page - http

I'm trying to build a sample web application demonstrating rest techniques using go at the back-end, serving json based requests and javascript, jquery in the front-end (I'm not using html/template package).
FileServer "returns a handler that serves HTTP requests with the contents of the file system rooted at root."
supose that I'm publishing my static folder that contains index.html and scripts folder holding some javascript files.
How can I prevent the client from viewing my js files (publishing just the index.html at /) ?

You can easily restrict the FileServer, which is a HttpHandler by wrapping another HttpHandler around that. For example, take this wrapper which ONLY allows *.js files to be served:
func GlobFilterHandler(h http.Handler, pattern string) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
fileName := filepath.Base(path)
if ok, err := filepath.Match(pattern, fileName); !ok || err != nil {
if err != nil {
log.Println("Error in pattern match:", err)
}
http.NotFound(w, r)
return
}
h.ServeHTTP(w, r)
})
}
func main() {
fileHandler := http.FileServer(http.Dir("/tmp/dtest"))
wrappedHandler := GlobFilterHandler(fileHandler, "*.js")
}
You can find a blog post here which describes the basic idea pretty good.
Another option you have is to extend on http.Dir and make your own http.FileSystem implementation which does exactly what you want:
type GlobDir struct {
Dir http.Dir
Pattern string
}
func (d GlobDir) Open(name string) (http.File, error) {
baseName := filepath.Base(name)
if ok, err := filepath.Match(d.Pattern, baseName); !ok || err != nil {
if err != nil {
return nil, err
}
return nil, fmt.Errorf("%s not match GlobDir pattern.", baseName)
}
return d.Dir.Open(name)
}
func main() {
fileHandler := http.FileServer(GlobDir{
Dir: http.Dir("/tmp/dtest"),
Pattern: "*.js",
})
http.ListenAndServe(":8080", fileHandler)
}
The second solution implements the http.FileSystem interface which is accepted by http.FileServer.
It checks whether the input file name matches the supplied pattern and then hands control down to the original http.Dir. This is probably the way you want to go here.

Related

Remove the .html extension from every file in a simple HTTP server

I want to make it so when someone visits a page on my Go HTTP server, they won't see the .html extension.
E.g. when they visit https://example.org/test they will see the content of https://example.org/test.html.
My code:
package main
import (
"net/http"
)
func main() {
fs := http.FileServer(http.Dir("public/"))
http.Handle("/", http.StripPrefix("/", fs))
http.ListenAndServe(":8000", nil)
}
One option is to implement http.FileSystem using http.Dir. The advantage of this approach is that it takes advantage of the carefully written code in http.FileServer.
It will look something like this:
type HTMLDir struct {
d http.Dir
}
func main() {
fs := http.FileServer(HTMLDir{http.Dir("public/")})
http.Handle("/", http.StripPrefix("/", fs))
http.ListenAndServe(":8000", nil)
}
The implementation of the Open method depends on the application requirements.
If you always want to tack on the .html extension, then use this code:
func (d HTMLDir) Open(name string) (http.File, error) {
return d.d.Open(name + ".html")
}
If you want to fallback to the .html extension, then use this code:
func (d HTMLDir) Open(name string) (http.File, error) {
// Try name as supplied
f, err := d.d.Open(name)
if os.IsNotExist(err) {
// Not found, try with .html
if f, err := d.d.Open(name + ".html"); err == nil {
return f, nil
}
}
return f, err
}
Flip the previous one around to start with the .html extension and fallback to the name as supplied:
func (d HTMLDir) Open(name string) (http.File, error) {
// Try name with added extension
f, err := d.d.Open(name + ".html")
if os.IsNotExist(err) {
// Not found, try again with name as supplied.
if f, err := d.d.Open(name); err == nil {
return f, nil
}
}
return f, err
}
So basically you want the http.FileServer functionality, but you don't want clients having to enter the trailing .html extension.
Another simple solution is to add it yourself, at the server side. This is how it can be done:
fs := http.FileServer(http.Dir("public"))
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
r.URL.Path += ".html"
fs.ServeHTTP(w, r)
})
panic(http.ListenAndServe(":8000", nil))
And that's all.
If you want this file server to also serve other files (such as images and CSS files), only append the .html extension if it has no extension:
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if ext := path.Ext(r.URL.Path); ext == "" {
r.URL.Path += ".html"
}
fs.ServeHTTP(w, r)
})
It's possible, but it cannot be done by serving the files using http.FileServer().
Instead, create a custom handler for the / route. Inside the handler, serve the requested file directly using http.ServeFile().
viewPath := "public/"
http.Handle("/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// hack, if requested url is / then point towards /index
if r.URL.Path == "/" {
r.URL.Path = "/index"
}
requestedPath := strings.TrimLeft(filepath.Clean(r.URL.Path), "/")
filename := fmt.Sprintf("%s/%s.html", viewPath, requestedPath)
http.ServeFile(w, r, filename)
}))
http.ListenAndServe(":8000", nil)
The .html suffix is added to every request path, so it'll point correctly towards the html files.
path / -> ./public/index.html
path /index -> ./public/index.html
path /some/folder/about -> ./public/some/folder/about.html
...

How can I serve a CSS file and have dynamic routing?

I'm trying to set up an HTTP server in Go using only the standard library. The server should be able to accept requests of the form /:uuid and should be able to serve an index.html file as well as a css file imported in it. This is what my code looked like:
func indexHandler(w http.ResponseWriter, r *http.Request) {
// serve index.html
if r.URL.Path == "/" {
http.ServeFile(w, r, "./web/index.html")
} else {
// if the path is /randomwords, redirect to mapped URL
randomwords := r.URL.Path[1:]
url := getMappedURL(randomwords)
http.Redirect(w, r, url, http.StatusFound)
}
}
func main() {
http.HandleFunc("/", indexHandler)
log.Println("listening on port 5000")
http.ListenAndServe(":5000", nil)
}
This serves the html file and is able to accept requests like /:something but the problem is that it doesn't include the CSS file. After some googling, I changed the main function to this:
func main() {
fs := http.FileServer(http.Dir("web"))
http.Handle("/", fs)
log.Println("listening on port 5000")
http.ListenAndServe(":5000", nil)
}
This serves both the HTML and the CSS files but it doesn't allow routes of the form :something. I can't figure out how to have both of these features.
Your original solution was nearly there, all you have to do is add a branch:
if r.URL.Path == "/" {
http.ServeFile(w, r, "./web/index.html")
} else if r.URL.Path == "/styles.css" {
http.ServeFile(w, r, "./web/styles.css")
} else {
// ...
Of course this can be tailored as needed - you could check for any file ending in ".css" using strings.HasSuffix, for example.

HTTP server and client to download multiple files

I have tried setting up a download server and a download client for individual files. How can I modify them to serve/download all the files from a directory?
Following are my server and client codes:
//server.go
func main() {
http.HandleFunc("/dlpath", handle)
err := http.ListenAndServe(":10001", nil)
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
}
func handle(writer http.ResponseWriter, r *http.Request) {
filename := "C:\\Users\\aarvi\\GolandProjects\\src\\Practice\\download\\serve\\send.txt"
http.ServeFile(writer, r, filename)
}
//client.go
func main() {
downloadFile("res_out.txt", "http://localhost:10001/dlpath")
}
func downloadFile(dirname string, url string) error {
// Create the file
out, err := os.OpenFile(dirname, os.O_WRONLY | os.O_CREATE | os.O_APPEND, 0666)
if err != nil {
fmt.Println(err)
}
defer out.Close()
// get data
request, err := http.NewRequest("GET", url, nil)
if err != nil {
fmt.Println(err)
}
client := http.Client{}
resp, err := client.Do(request)
if err != nil {
return err
}
defer resp.Body.Close()
// Write the body to file
_, err = io.Copy(out, resp.Body)
if err != nil {
return err
}
return nil
}
I tried serving the directory in the handle function like so:
dirname := "C:\\Users\\aarvi\\GolandProjects\\src\\Practice\\download\\serve"
http.FileServer(http.Dir(dirname))
and tried to print out the response on the client side, but I got nothing. How can I serve all the files from the /serve directory, and download them in the client?
EDIT:
Following are the contents of the serve directory:
serve
---sample.txt
---send.txt
---dir2
------abc.txt
How can I download all these files on the client side as separate files, with the directory structure intact?
Update: When I call the http.Handle function (as mentioned in the answer) directly in the main function, I am able to serve all the files, and the file within the inner directory too.
However, when I call the same within the handle function, it doesn't serve anything. I am guessing this has something to do with the path?
The problem can be in the file path you are requesting. It is prefixed with /dlpath/. You must strip this prefix and pass the rest of the text as a path. See: https://godoc.org/net/http#FileServer
Could you try this code snippet:
package main
import (
"log"
"net/http"
)
func main() {
dirName := "C:\\Users\\aarvi\\GolandProjects\\src\\Practice\\download\\serve"
http.Handle("/dlpath/", http.StripPrefix("/dlpath", http.FileServer(http.Dir(dirName))))
err := http.ListenAndServe(":8001", nil)
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
}
Hope this helps.
P.S.
This is from the case when you are serving the directory in the handler function.
The accepted answer and some amount of googling helped me write FileServer with a different handler (http.HandleFunc). Using HandleFunc allowed me to set up other things like authentication etc.,
The key was to call ServeHTTP()
So, the handler function can be modified in the following manner to serve all the files in a directory:
func handle(writer http.ResponseWriter, r *http.Request) {
{
dirName := "C:\\Users\\aarvi\\GolandProjects\\src\\Practice\\download\\serve"
http.StripPrefix("/dlpath", http.FileServer(http.Dir(dirName))).ServeHTTP(writer, r)
}
}

Creating a tar file to be served via Google App Engine

I'm writing an application for Google AppEngine using Go, and need to tar together a bunch of files to serve to the user when they navigate to a particular URL. At the moment the files are static, and so I could solve this problem by tarring them before upload to the server. In the future I would like to dynamically alter them before tarring, and so would like to learn how to tar & serve the static files on request.
In my init() function I have the following line:
http.HandleFunc("/download.tar", tarit)
The function tarit is the one I am having a problem with, and it currently looks like the following:
func tarit(w http.ResponseWriter, r *http.Request) {
tarball := tar.NewWriter(w)
defer tarball.Close()
info, err := os.Stat("/files")
if err != nil {
return
}
var baseDir string
if info.IsDir() {
baseDir = filepath.Base("/files")
}
filepath.Walk("/files", func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
header, err := tar.FileInfoHeader(info, info.Name())
if err != nil {
return err
}
if baseDir != "" {
header.Name = filepath.Join(baseDir, strings.TrimPrefix(path, "/files"))
}
if err := tarball.WriteHeader(header); err != nil {
return err
}
if info.IsDir() {
return nil
}
file, err := os.Open(path)
if err != nil {
return err
}
defer file.Close()
_, err = io.Copy(tarball, file)
return err
})
}
The files I am trying to add to the tarball are located in /files, and I've added this folder as a static_dir in the app.yaml.
When navigating to the appropriate URL, the browser downloads a tar file that is only 1 KB in size, and appears to be empty.
I would very much appreciate if someone could point out where I am going wrong, or what I am misunderstanding. I'd also be very happy to provide any other details that you would like.
Thanks!
Specify directories using paths relative to the directory containing app.yaml. The path in the posted code is the absolute path "/files". Perhaps you should change it to "files".
Log the errors returned from os.Stat and filepath.Walk. The errors will probably lead you to the problem.

Asynchronous Testing With Stream Processing

I'm very new to Go, so I may be misunderstanding something foundational about Go's async/stream handling, but here goes...
I'm trying to write some tests using ginkgo on a function I wrote that processes streams.
The processing side reads in newline-delimited text from a File until it encounters a special delimiter line at which point it tries to parse the text as JSON. The code looks like this:
func ParseConfig(inStream *os.File) (Config, error){
var header string
var stdin = bufio.NewScanner(inStream)
for stdin.Scan() {
line := stdin.Text()
if line == "|||" {
break;
}
header += line
}
// parse JSON here and return
}
My test looks something like this
Describe("ParseConfig()", func() {
It("should pass for a valid header", func(){
_, err := io.WriteString(stream, "{\"Key\": \"key\", \"File\": \"file\"}\n|||\n")
Expect(err).NotTo(HaveOccurred())
conf, err := parser.ParseConfig(stream)
Expect(err).NotTo(HaveOccurred())
Expect(conf.Key).To(Equal("key"))
})
})
Unfortunately, this yields a JSON parsing error, as it's trying to parse an empty string. I'm assuming that my problem is that I'm sending the string on the stream before I've told the ParseConfig() function to listen on that string for data? But I'm not entirely clear how I could refactor this to use proper go routines to first listen for data then send it.
Some of the potential solutions I saw were around the use of "channels" (with which I'm unfamiliar) but I was worried that this one need might not be worth a major refactor to introduce a whole new paradigm of concurrency.
Thanks!
Not sure if I understood correctly, but your ParseConfig should probably take an io.Reader instead of a *os.File. That way you can test it directly without worrying about concurrency.
file t_test.go:
package main
import (
"strings"
"testing"
"github.com/onsi/ginkgo"
"github.com/onsi/gomega"
)
var _ = ginkgo.Describe("ParseConfig()", func() {
ginkgo.It("should pass for a valid header", func() {
// really don't know what you were doing with your 'stream' variable
// This is a test, you should forge a test scenario and pass it to your config function
stream := strings.NewReader(`{"Key": "key", "File": "file"}` + "\n|||\n")
conf, err := ParseConfig(stream)
gomega.Expect(err).NotTo(gomega.HaveOccurred())
gomega.Expect(conf.Key).To(gomega.Equal("key"))
})
})
func TestParseConfig(t *testing.T) {
ginkgo.RunSpecs(t, "Parse Config")
}
file main.go
package main
import (
"bufio"
"encoding/json"
"io"
"log"
"os"
)
type Config struct {
Key string
File string
}
func ParseConfig(inStream io.Reader) (*Config, error) {
var header string
var stdin = bufio.NewScanner(inStream)
for stdin.Scan() {
line := stdin.Text()
if line == "|||" {
break
}
header += line
}
c := &Config{}
// parse JSON here and return
if err := json.Unmarshal([]byte(header), c); err != nil {
return nil, err
}
return c, nil
}
func main() {
f, err := os.Open("config.json")
if err != nil {
log.Fatal(err)
}
ParseConfig(f)
}

Resources