Check errors when calling http.ResponseWriter.Write() - http

Say I have this http handler:
func SomeHandler(w http.ResponseWriter, r *http.Request) {
data := GetSomeData()
_, err := w.Write(data)
}
Should I check the error returned by w.Write? Examples I've seen just ignore it and do nothing. Also, functions like http.Error() do not return an error to be handled.

It's up to you. My advice is that unless the documentation of some method / function explicitly states that it never returns a non-nil error (such as bytes.Buffer.Write()), always check the error and the least you can do is log it, so if an error occurs, it will leave some mark which you can investigate should it become a problem later.
This is also true for writing to http.ResponseWriter.
You might think ResponseWriter.Write() may only return errors if sending the data fails (e.g. connection closed), but that is not true. The concrete type that implements http.ResponseWriter is the unexported http.response type, and if you check the unexported response.write() method, you'll see it might return a non-nil error for a bunch of other reasons.
Reasons why ResponseWriter.Write() may return a non-nil error:
If the connection was hijacked (see http.Hijacker): http.ErrHijacked
If content length was specified, and you attempt to write more than that: http.ErrContentLength
If the HTTP method and / or HTTP status does not allow a response body at all, and you attempt to write more than 0 bytes: http.ErrBodyNotAllowed
If writing data to the actual connection fails.
Even if you can't do anything with the error, logging it may be of great help debugging the error later on. E.g. you (or someone else in the handler chain) hijacked the connection, and you attempt to write to it later; you get an error (http.ErrHijacked), logging it will reveal the cause immediately.
Tip for "easy" logging errors
If you can't do anything with the occasional error and it's not a "showstopper", you may create and use a simple function that does the check and logging, something like this:
func logerr(n int, err error) {
if err != nil {
log.Printf("Write failed: %v", err)
}
}
Using it:
logerr(w.Write(data))
Tip for "auto-logging" errors
If you don't even want to use the logerr() function all the time, you may create a wrapper for http.ResponseWriter which does this "automatically":
type LogWriter struct {
http.ResponseWriter
}
func (w LogWriter) Write(p []byte) (n int, err error) {
n, err = w.ResponseWriter.Write(p)
if err != nil {
log.Printf("Write failed: %v", err)
}
return
}
Using it:
func SomeHandler(w http.ResponseWriter, r *http.Request) {
w = LogWriter{w}
w.Write([]byte("hi"))
}
Using LogWriter as a wrapper around http.ResponseWriter, should writes to the original http.ResponseWriter fail, it will be logged automatically.
This also has the great benefit of not expecting a logger function to be called, so you can pass a value of your LogWriter "down" the chain, and everyone who attempts to write to it will be monitored and logged, they don't have to worry or even know about this.
But care must be taken when passing LogWriter down the chain, as there's also a downside to this: a value of LogWriter will not implement other interfaces the original http.ResponseWriter might also do, e.g. http.Hijacker or http.Pusher.
Here's an example on the Go Playground that shows this in action, and also shows that LogWriter will not implement other interfaces; and also shows a way (using 2 "nested" type assertions) how to still get out what we want from LogWriter (an http.Pusher in the example).

I want to add to #icza solution. You don't need to create logging structure, you can use simple function:
func logWrite(write func([]byte) (int, error), body []byte) {
_, err := write(body)
if err != nil {
log.Printf("Write failed: %v", err)
}
}
Take a look on that approach based on the #icza code: https://play.golang.org/p/PAetVixCgv4

Related

Reliable thread-safe map

I was making a WaitForResponse function for my Discord bot, and it works, but the user can still use commands even when the bot is expecting a response. I combated this by using a map with the user and channel IDs, but I was then hit with the dreaded fatal error: concurrent map read and write. So I tried using a sync.Map, however it wouldn't always work when I spammed the command. I could sometimes still run commands when the bot was expecting a response. Is there any way I can ensure that the values are getting added and removed from the map when and as they should?
For these scenarios, sync.Mutex can be used to ensure that only one modification is allowed by acquiring a lock around the code that you want to be thread-safe.
var mu sync.Mutex
func readMap(key string) {
mu.Lock()
defer mu.Unlock()
return yourMap[key]
}
func updateMap(key, value string) {
mu.Lock()
defer mu.Unlock()
yourMap[key] = value
}
Mutex ensures that ONLY ONE goroutine can is allowed access to the locked code, which means for your case, only one operation, either read or write can be performed.
For better efficiency, you should consider using sync.RWMutex since you might not want to lock the map when it's being read. From GoDoc:
A RWMutex is a reader/writer mutual exclusion lock. The lock can be held by an arbitrary number of readers or a single writer. The zero value for a RWMutex is an unlocked mutex.
var mu sync.RWMutex
func readMap(key string) {
mu.RLock()
defer mu.RUnlock()
return yourMap[key]
}
func updateMap(key, value string) {
mu.Lock()
defer mu.Unlock()
yourMap[key] = value
}

GO API with SQLITE3 can't DELETE tuple from db

Hello fellow developers.
I am trying to learn GO while constructing a simple web API using sqlite3. I got stuck at somepoint where i am unable to delete rows from my table by sending a DELETE request from postman. I am trying to use the code below to delete a row. I have already verified that I have access to db and I can also delete rows by using command tool of sqlite3. I do not understand what is wrong!
func deleteArticle(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
params := mux.Vars(r) // get any params
db := connectToDB(dbName)
defer db.Close()
_, err := db.Query("DELETE FROM article WHERE id=" + params["id"])
if err != nil {
fmt.Fprintf(w, "article couldn't be found in db")
}
}
Here is the navigation part:
myRouter.HandleFunc("/articles/{id}", deleteArticle).Methods("DELETE")
No mather what I do I cannot delete an article from db using postman.
Thanks bunches.
Thanks to #mkopriva 's comments I have learned that
1.
It is very important that you do not use Query nor QueryRow for SQL
queries that do not return any rows, for these cases use the Exec
method. When you use Query you always have to assign the result to a
non-blank identifier, i.e. anything but _, and then invoke the Close
method on that once you're done with the result. If you do not do that
then your application will leak db connections and very soon will
start crashing.
2.
when you want to pass user input (including record ids) to your
queries you have to utilize, at all times, the parameter-reference
syntax supported by the sql dialect and/or dirver you are using, and
then pass the input separately. That means that you should never do
Exec("DELETE FROM article WHERE id=" + params["id"]),
instead you should always do
Exec("DELETE FROM article WHERE id= ?",params["id"])
If you do not do it the proper way and instead continue
using plain string concatenation your app will be vulnerable to SQL
injection attacks.
Regarding this information I have changed my code into:
func deleteArticle(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
params := mux.Vars(r) // get any params
db := connectToDB(dbName)
defer db.Close()
fmt.Printf("%q\n", params["id"])
statement, err := db.Prepare("DELETE FROM article WHERE id= ?")
if err != nil {
fmt.Fprintf(w, "article couldn't be found in db")
}
statement.Exec(params["id"])
}
Which has solved my problem. So thank you #mkopriva

Is it safe to use sync.Pool reference as context value?

I have struct that uses sync.Pool.
Is it safe to use this reference as context value?
type User struct {
ID string
}
var userPool = sync.Pool{
New: func() interface{} {
return &User{}
},
}
func getUser() *User {
return userPool.Get().(*User)
}
func recycleUser(user *User) {
userPool.Put(user)
}
The user struct is retrieved from pool in middleware.
func middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// get user from pool
user := getUser()
// user must be recycled
ctx := context.WithValue(r.Context(), "user", user)
}
}
And recycled in handler.
func getUser(w http.ResponseWriter, r *http.Request) {
user := r.Context().Value("user").(*User)
// TODO: do something with user
// put user struct back into pool
recycleUser(user)
}
Edit:
My question is more about how context deals with pointer to my object. Does it make a copy? Is it safe to use non-primitive object with context?
There are three points to make:
Usage of sync.Pool
Your usage of pool is as intended. From golang Documentation
A Pool is safe for use by multiple goroutines simultaneously.
Retrieving users
getUser uses func (p *Pool) Get() interface{} which removes the returned item from the pool, afterwards you can do whatever you please
with that value. In your case, thats a *User. Whether the associated
ressource is safe to use, well, depends on your usage of those values in the rest of the program.
Recycling users
Calling recycleUser within the hander is potentially dangerous, depending on your environment.
What could possibly go wrong?
After recycleUser returned your *User back to the pool, it could be retrieved and used by a different goroutine immediately. But at the same time, *User is still stored in the context assiociated with the request. So it depends if any of the functions of your middleware use the *User from the context, too and wether they store the *User value. Or if you later add some code after the recycleUser, which uses the *User value. After the recycleUser call, all those usages probably operate on the wrong user, which is already used by some different request.
How to solve those issues
coding convention
if you retrieve the user in middleware functions, don't store it
if you retrieve the user in middleware functions, don't use it after the call to the next handler in the chain.
don't call recycleUser in middleware functions
retrieve the user in the leaf handler (last handler in the chain, doesn't calls any further handlers propagating the context and thus the *User) and use defer recycleUser(user) to put back the *User in the pool and make it impossible that later added code uses *User after the call to recycleUser(user).
in the leaf handler, don't use *User in code which is called by some defer usage in the leaf handler, other than recycleUser.
ensuring the coding convention by technical means. My only idea here would be to place the *User inside some struct and only use that struct for one request and mark it as empty, once *User is put back into the pool. Make all access methods of that struct check for emptyness and panic/log/whatever, if an empty struct is accessed. Place a pointer to that struct in the context. But that is probably pointless, as you now allocate that struct for each request, instead of a User, which probably was, what you tried to avoid.
A minor remark
You probably meant your middleware function to read like:
func middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// get user from pool
user := getUser()
// user must be recycled
ctx := context.WithValue(r.Context(), "user", user)
// probably do something more
next(w, r.WithContext(ctx))
}
}

Get HTTP Status using swift

I am sorry, I haven't found an answer for my question(( Please, don't be very harsh, I am not a professional programmer, but I keep learning and hope once I will be able to answer someone's question))
I am trying to get HTTP Status of the link (I am sort of generating links depending on one database entries code, like ABCDEF, I keep them in an array and then generate a link to second database, like www.blablabla.ABCDEF.net), so I can see whether the page exists in database or not.
I have written this code, but something is wrong. So maybe it's a question like: "What's wrong with my code?" But on the stack they also say, you have to show your attempts of problem solving...
I wish I could keep it all swift, without any additional modules or something, I think NSHTTPURLResponse must be enough, but I am using it somehow wrong.
Looking forward to help and replies))
var err: NSError!
NSURLConnection.sendAsynchronousRequest(request, queue: queue, completionHandler:{ (response: NSURLResponse?, data: NSData!, error: err) -> Void in
if (err != nil) {
let httpStatus: NSHTTPURLResponse = response as NSHHTPURLResponse
for myLink in allLinks {
println("HERE IS THE CURRENT STATUS CODE" + httpStatus.statusCode + "OF A LINK:" + myLink)
if httpStatus.statusCode == 200 {println("SUCCESS!!!!!")}
}
}
}
The fundamental issue here is that you appear to be looking at the statusCode only if the err is not nil. But if you have error, you probably don't have status code. The error parameter to the closure indicates fundamental network issue that probably prevented you from getting to the page in question. The statusCode is generally only meaningful if the error was nil (i.e. you succeeded in connecting to the server), in which case the statusCode is how the server informs you of its ability to service the HTTP request.
A couple of minor things:
You don't need the var err: NSError! line (because the error object is passed as parameter to the closure). This variable you've declared is not used here.
I don't see how error: err as the third parameter to the closure could have worked. The syntax is "variableName: variableType", but there is no type of err.
Likewise, your code is referring to a non-existent class, NSHHTPURLResponse. In Swift 3 and later, it's HTTPURLResponse.
It's probably prudent to do if let for the retrieval of the HTTPURLResponse in order to get the statusCode.
I'm unclear as to your intent in iterating through allLinks, because this connection is just for a given request, not a bunch of links. Just look at the statusCode in light of the particular request. If you need to test multiple URLs, then you do a separate request for each.
We should consider any codes between 200 and 299 as success, not just 200. I'd suggest using the range 200 ..< 300.
Thus:
let task = URLSession.shared.dataTask(with: request) { data, response, error in
guard let data = data, let httpResponse = response as? HTTPURLResponse, error == nil else {
print("No valid response")
return
}
guard 200 ..< 300 ~= httpResponse.statusCode else {
print("Status code was \(httpResponse.statusCode), but expected 2xx")
return
}
// everything OK, process `data` here
}
task.resume()
I also made a few other changes (updated for Swift 3 and later; use URLSession rather than URLConnection; I think error is fine choice for the variable name; I prefer the trailing closure syntax; I tend to use inferred types for closure parameters to make the declaration a little more concise, etc.), but all of that is immaterial to the question at hand: Hopefully this illustrates how one checks the status code.
For Swift 2 rendition, see previous revision of this answer.
[Swift 5.2]
Hi there, you can try this one:
let task = URLSession.shared.dataTask(with: yourRequest) {
(data, response, error) in
guard let response = response else {
print("Cannot found the response")
return
}
let myResponse = response as! HTTPURLResponse
print("Status Code:", myResponse.statusCode)
}
task.resume()
Here is the output
Status Code: 200
(This is your StatusCode)

How to store or cache values to be served in http requests in Golang?

I am writing some Go web services (also implementing the webserver in Go with http.ListenAndServe).
I have a map of structs which I would like to keep in memory (with an approximate data size of 100Kb) to be used by different HTTP requests.
How can this be achieved in Go? I am thinking to use global package variables or caching systems (like memcache/groupcache).
In addition to the answers you've already received, consider making use of receiver-curried method values and http.HandlerFunc.
If your data is data that is loaded before the process starts, you could go with something like this:
type Common struct {
Data map[string]*Data
}
func NewCommon() (*Common, error) {
// load data
return c, err
}
func (c *Common) Root(w http.ResponseWriter, r *http.Request) {
// handler
}
func (c *Common) Page(w http.ResponseWriter, r *http.Request) {
// handler
}
func main() {
common, err := NewCommon()
if err != nil { ... }
http.HandleFunc("/", common.Root)
http.HandleFunc("/page", common.Page)
http.ListenAndServe(...)
}
This works nicely if all of the Common data is read-only. If the Common data is read/write, then you'll want to have something more like:
type Common struct {
lock sync.RWMutex
data map[string]Data // Data should probably not have any reference fields
}
func (c *Common) Get(key string) (*Data, bool) {
c.lock.RLock()
defer c.lock.RUnlock()
d, ok := c.data[key]
return &d, ok
}
func (c *Common) Set(key string, d *Data) {
c.lock.Lock()
defer c.lock.Unlock()
c.data[key] = *d
}
The rest is basically the same, except instead of accessing the data through the receiver's fields directly, you'd access them through the getters and setters. In a webserver where most of the data is being read, you will probably want an RWMutex, so that reads can be executed concurrently with one another. Another advantage of the second approach is that you've encapsulated the data, so you can add in transparent writes to and/or reads from a memcache or a groupcache or something of that nature in the future if your application grows such a need.
One thing that I really like about defining my handlers as methods on an object is that it makes it much easier to unit test them: you can easily define a table driven test that includes the values you want and the output you expect without having to muck around with global variables.
Don't indulge in premature optimization. Define a Go package API to encapsulate the data and then you can change the implementation at any time. For example, just scribbling,
package data
type Key struct {
// . . .
}
type Data struct {
// . . .
}
var dataMap map[Key]Data
func init() {
dataMap = make(map[Key]Data)
}
func GetData(key Key) (*Data, error) {
data := dataMap[key]
return &data, nil
}

Resources