I am trying to write a simple web app that will listen for UDP packets.
But I can either only listen for UDP packets, or run the web app...
I am not familiar with GoLang, but here's the code I'm using to...
listen for UDP:
ServerConn, _ := net.ListenUDP("udp", &net.UDPAddr{IP:[]byte{#,#,#,#},Port:####,Zone:""})
defer ServerConn.Close()
buf := make([]byte, 1024)
for {
n, addr, _ := ServerConn.ReadFromUDP(buf)
fmt.Println("Received ", string(buf[0:n]), " from ", addr)
}
Server logic:
package main
We import 4 important libraries
1. “net/http” to access the core go http functionality
2. “fmt” for formatting our text
3. “html/template” a library that allows us to interact with our html file.
4. "time" - a library for working with date and time.
import (
"net/http"
"fmt"
"time"
"html/template"
)
//Create a struct that holds information to be displayed in our HTML file
type Welcome struct {
Name string
Time string
}
//Go application entrypoint
func main() {
//Instantiate a Welcome struct object and pass in some random information.
//We shall get the name of the user as a query parameter from the URL
welcome := Welcome{"Anonymous", time.Now().Format(time.Stamp)}
//We tell Go exactly where we can find our html file. We ask Go to parse the html file (Notice
// the relative path). We wrap it in a call to template.Must() which handles any errors and halts if there are fatal errors
templates := template.Must(template.ParseFiles("templates/welcome-template.html"))
//Our HTML comes with CSS that go needs to provide when we run the app. Here we tell go to create
// a handle that looks in the static directory, go then uses the "/static/" as a url that our
//html can refer to when looking for our css and other files.
http.Handle("/static/", //final url can be anything
http.StripPrefix("/static/",
http.FileServer(http.Dir("static")))) //Go looks in the relative "static" directory first using http.FileServer(), then matches it to a
//url of our choice as shown in http.Handle("/static/"). This url is what we need when referencing our css files
//once the server begins. Our html code would therefore be <link rel="stylesheet" href="/static/stylesheet/...">
//It is important to note the url in http.Handle can be whatever we like, so long as we are consistent.
//This method takes in the URL path "/" and a function that takes in a response writer, and a http request.
http.HandleFunc("/" , func(w http.ResponseWriter, r *http.Request) {
//Takes the name from the URL query e.g ?name=Martin, will set welcome.Name = Martin.
if name := r.FormValue("name"); name != "" {
welcome.Name = name;
}
//If errors show an internal server error message
//I also pass the welcome struct to the welcome-template.html file.
if err := templates.ExecuteTemplate(w, "welcome-template.html", welcome); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
})
//Start the web server, set the port to listen to 8080. Without a path it assumes localhost
//Print any errors from starting the webserver using fmt
fmt.Println("Listening");
fmt.Println(http.ListenAndServe(":8080", nil));
}
taken from(https://medium.com/google-cloud/building-a-go-web-app-from-scratch-to-deploying-on-google-cloud-part-1-building-a-simple-go-aee452a2e654)
I tried putting both of these extracts in 1 file, as well as running 2 files at the same time using
go run *.go
Any help would be appreciated!
You're going to need to start looking into goroutines - since you're asking to do two things concurrently. I suggest doing some reading into channels, goroutines, and concurrency in general :)
Related
Consider the following very basic "net/http"-program:
package main
import (
"net/http"
"log"
)
func entry(w http.ResponseWriter, req *http.Request) {
log.Println(req.URL.Path)
path := []byte(req.URL.Path)
w.Write(path)
}
func main() {
http.HandleFunc("/", entry)
err := http.ListenAndServe("localhost:10000", nil)
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
}
When accessing the URL, http://localhost:10000/a//b, the DefaultServerMux, which is what the default HandleFunc() and ListenAndServe() uses, redirects it to /a/b, effectively removing the double slash.
The documentation for ServerMux does specify that it 'sanitises' URL requests:
ServeMux also takes care of sanitizing the URL request path, redirecting any request containing . or .. elements or repeated slashes to an equivalent, cleaner URL.
But what if I don't want it to? I have a scenario where // != / in my URLs. I could find another solution. But is there a way to still use Go's "net/http" package, while it not sanitising my URLs like this? Preferably with as little re-writing as possible.
(I will probably find a different solution than having // and / being distinct, since I am probably happy with other features that ServerMux provides (in case a solution requires me to use another multiplexer), but now I am curious whether there is a solution with Go's standard "net/http" package.)
Go for gorilla mux to route the server using the url. By default the DefaultSeveMux uses Clean which modify the url against various unwanted characters. That's the reason the url changed whn you are using // double slash.
func Clean(path string) string
While if you do not want to sanitize the url. Gorilla mux provide a method SkipClean() which when set to true. It will not sanitize the url.
func (r *Router) SkipClean(value bool) *Router {
r.skipClean = value
return r
}
It is mentioned in the document of gorilla mux for SkipClean() as:
SkipClean defines the path cleaning behaviour for new routes. The
initial value is false. Users should be careful about which routes are
not cleaned When true, if the route path is "/path//to", it will
remain with the double slash. This is helpful if you have a route
like: /fetch/http://xkcd.com/534/ When false, the path will be
cleaned, so /fetch/http://xkcd.com/534/ will become
/fetch/http/xkcd.com/534
I have been using golang's default http.ServeMux for http route handling.
wrap := func(h func(t *MyStruct, w http.ResponseWriter, r *http.Request)) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
h(t, w, r)
}
}
// Register handlers with default mux
httpMux := http.NewServeMux()
httpMux.HandleFunc("/", wrap(payloadHandler))
Assume this server is accessible via http://example.com/
Very few of my client's requests were of path http://example.com/api//module (note the extra slash) which is redirected as 301 Moved Permanently. Exploring inside golang's http ServeMux.Handler(r *Request) function, seems it's intended.
path := cleanPath(r.URL.Path)
// if there is any change between actual and cleaned path, the request is 301 ed
if path != r.URL.Path {
_, pattern = mux.handler(host, path)
url := *r.URL
url.Path = path
return RedirectHandler(url.String(), StatusMovedPermanently), pattern
}
I've looked into other similar issue.
go-web-server-is-automatically-redirecting-post-requests
Above qn has problem with redundant / in register pattern itself, but my use case is not with register pattern (in some nested path which is irrelevent to register pattern)
Problem is, since my client's requests are POST, browsers handle 301 with new GET request with exact query params and POST body. But change in the HTTP method causes the request to fail.
I have already instructed client to fix the redundant / in url, but the fix might take few (?) weeks to be deployed in all client locations.
Also these redundant / are handled fine in Apache Tomcat, but fails only in golang server. So is this the intended behaviour in my use case (redundant / in nested path) with golang or possible bug?
I am thinking of way to override the Handler func of ServeMux, but it won't be useful since Handler calls are made internally. Looking to disable this 301 behaviour, help would be appreciated.
Relevant links
http-post-method-is-actally-sending-a-get
The clean and redirect is intended behavior.
Wrap the mux with a handler that removes the double slashes:
type slashFix struct {
mux http.Handler
}
func (h *slashFix) ServeHTTP(w http.ResponseWriter, r *http.Request) {
r.URL.Path = strings.Replace(r.URL.Path, "//", "/", -1)
h.mux.ServeHTTP(w, r)
}
Use it like this:
httpMux := http.NewServeMux()
httpMux.HandleFunc("/", wrap(payloadHandler))
http.ListenAndServe(addr, &slashFix{httpMux})
Accepeted answer solved the problem
One more way is to use Gorilla mux and setting SkipClean(true). But be sure to know about the side effects in its doc
SkipClean defines the path cleaning behaviour for new routes. The initial value is false. Users should be careful about which routes are not cleaned. When true, if the route path is "/path//to", it will remain with the double slash. This is helpful if you have a route like: /fetch/http://xkcd.com/534/
When false, the path will be cleaned, so /fetch/http://xkcd.com/534/ will become /fetch/http/xkcd.com/534
func (r *Router) SkipClean(value bool) *Router {
r.skipClean = value
return r
}
I'm building an api that also serves my react front end app, but am having an issue serving my index.html
Given that it's not really a go template I'm not using html/template.
I'm not seeing a strait forward way to serve the static html root of my app on all pages that do not start /api in the route.
I'm purposely trying not to use any go frameworks beyond gorilla's mux
My handler.go:
func Index(w http.ResponseWriter, r *http.Request) {
http.FileServer(http.Dir("./views"))
}
Routes.go:
type Route struct {
Name string
Method string
Pattern string
HandlerFunc http.HandlerFunc
}
type Routes []Route
var routes = Routes{
Route{
"Index",
"GET",
"/",
Index,
},
}
router.go
import (
"net/http"
"github.com/gorilla/mux"
)
func NewRouter() *mux.Router {
router := mux.NewRouter().StrictSlash(true)
for _, route := range routes {
var handler http.Handler
handler = route.HandlerFunc
handler = Logger(handler, route.Name)
router.
Methods(route.Method).
Path(route.Pattern).
Name(route.Name).
Handler(handler)
}
return router
}
main:
package main
import (
"./server"
"log"
"net/http"
)
func main() {
router := server.NewRouter()
log.Fatal(http.ListenAndServe(":8080", router))
}
Currently a blank page shows up, and thats it. My index.html is located in /views/index.html in relation to the executable (but I've tried it in relation to the handler as well)
Update
I was able to serve the html file using the method shown in this question: How do you serve a static html file using a go web server? However using mux and the more modularized file structure still yields a nice pretty, clean blank page.
In handler.go, your Index function is actually a no-op, since http.FileServer() returns a Handler, which is never passed the ResponseWriter or Request, hence the blank page.
Maybe try something like this to at least get past that:
func Index(w http.ResponseWriter, r *http.Request) {
http.FileServer(http.Dir("./views")).ServeHTTP(w, r)
}
It's common practice in some cases to pass plain URIs as suffix of the path instead of a query parameter. Here is an example from Internet Archive's Wayback Machine.
https://web.archive.org/web/20150825082012/http://example.com/
In this example, user is requesting a copy of http://example.com/ as captured at 2015-08-25 08:20:12. If we were to implement similar service in Go, we probably would have a router as follows:
http.HandleFunc("/web/", returnArchivedCopy)
Then in the returnArchivedCopy handler function, we will split r.URL.Path (where r is the Request object) to extract the date-time and the target URL. However there is a problem in this style of URL scheme; Go's net/http package calls cleanPath function on the path portion to sanitize it. This sanitization process does various cleanup tasks such as eeliminating . and .. from the path and replace multiple slashes with a single one. This later operation makes sense when because in Unix systems // in the file path are same as /. However this causes an issue in the above described use case as http://example becomes http:/example and the server internally returns a redirect response to the client with the sanitized path.
I am wondering, what are my options in this case? Is there a way to ask HTTP not to sanitize the request path while still utilizing all the default behavior that is shipped with the default (or slightly modified) server, multiplexer, and handler? Or is there a way to modify the request parameters (path in this case) before it hits the multiplexer's routing patterns. If the later is possible, we might try to perform something like URL encoding to avoid the redirect and later decode the URL back in the handler function before extracting desired bits.
I have experimented with some custom handlers and multiplexers, but I am new to Go, hence I was not quite sure how to delegate the routing back to the default handlers after making changes in the request.
You can implement a wrapper mux, that falls back to the default one, here's a very simple example:
func main() {
http.HandleFunc("/blah", func(w http.ResponseWriter, req *http.Request) {
w.Write([]byte("w00t"))
})
http.ListenAndServe(":9090", http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
p := strings.SplitN(req.URL.RequestURI()[1:] /*trim the first slash*/, "/", 3)
if len(p) != 3 || p[0] != "web" {
http.DefaultServeMux.ServeHTTP(w, req)
return
}
t, err := time.Parse("20060102150405", p[1])
if err != nil {
http.Error(w, "invalid time", 400)
return
}
url := p[2]
fmt.Fprintf(w, "requested url %v # %v", url, t)
}))
}
I've been using the gocraft-web package so far to do some development on an HTTP service. It's really great because you can stick middleware in it to check for stuff like the presence of a Cookie in the header.
At the moment I am wanting to implement request signing. Getting the client to sign the request is easy enough, but I am wanting to check it for all endpoints with a common piece of middleware. Basically the middleware needs to find the key to check against, compute the request HMAC, and check it against the supplied HMAC (presumably in the Authorization Header).
Computing the actual HMAC is really easy in go.
The problem is: reading the message in middleware makes it unavailable to the final endpoint.
The best solution I have come up with (example shown below) is to read everything from the Request in the middleware and stuffing it back into a bytes.Buffer for later reading. Is there a better way to do this? The current implementation seems a bit hackish.
Reading everything into memory sucks, but I can probably just put my service behind a proxy and limit the size of requests anyways. The actual content will always be pretty small(under 5 kilobytes). The extra copy introduced by this approach is likely to be quite slow, but computing the HMAC of a message is not exactly cheap to begin with.
The advantage to this is that it is transparent: it will work with any other go http code that just expects to read from Request.Body without any magic.
I suppose I could be a bit slicker and use a io.TeeReader.
This is my solution so far. If you post to localhost:3300 some JSON it prints the sha512 to the terminal in the server process, but also the response is able to contain a listing of the keys & values in it.
package main
import "fmt"
import "github.com/gocraft/web"
import "net/http"
import "bytes"
import "crypto/sha512"
import "io"
import "encoding/hex"
import "encoding/json"
type Context struct{}
type echoer struct {
*bytes.Buffer
}
func (e echoer) Close() error {
//Just do nothing to make the interface happy
return nil
}
func middlewareThatLooksAtBody(rw web.ResponseWriter, req *web.Request, next web.NextMiddlewareFunc) {
var replacement echoer
replacement.Buffer = &bytes.Buffer{}
hash := sha512.New()
hash.Write([]byte(req.Method))
reader := req.Body
var bytes []byte = make([]byte, 64)
for {
amount, err := reader.Read(bytes)
fmt.Printf("Read %d bytes\n", amount)
if err != nil {
if err == io.EOF {
break
}
panic(err)
}
if amount == 0 {
break
}
hash.Write(bytes)
replacement.Write(bytes)
}
//Is this needed?
reader.Close()
//replacement.Seek(0, 0)
req.Body = replacement
fmt.Printf("%v\n", hex.EncodeToString(hash.Sum(nil)))
next(rw, req)
}
func echoJson(rw web.ResponseWriter, req *web.Request) {
dec := json.NewDecoder(req.Body)
var obj map[string]interface{}
err := dec.Decode(&obj)
if err != nil {
rw.WriteHeader(http.StatusBadRequest)
fmt.Fprintf(rw, "%v\n", err)
return
}
for k, v := range obj {
fmt.Fprintf(rw, "%v = %v\n", k, v)
}
}
func main() {
router := web.New(Context{})
router.Middleware(middlewareThatLooksAtBody)
router.Post("/", echoJson)
http.ListenAndServe("localhost:3300", router)
}
From your description, it looks like you need to read all the bytes from the request body, regardless of what your handlers will do.
If so, then you have at least a couple of options that would avoid the extra copy:
1) Store the read contents inside your gocraft context.
2) Do all body data processing and validation in the middleware and store the results of the processing in the context.
Granted, this means that your handlers now must know that they should look for the contents in the context instead of the req.Body.
I think it's a decent trade-off though, given your requirements.