I am trying to handle incoming HTTP requests by Nginx and Lua. I need to read a blue from Redis in each request and currently, I open a Redis connection in every request by this code:
local redis = require "resty.redis"
local red = redis:new()
local ok, err = red:connect("redis", 6379)
if not ok then
ngx.say("failed to connect: ", err)
return
end
local res, err = red:auth("abcd")
if not res then
ngx.log(ngx.ERR, err)
return
end
Is there any way to make this connection static or singleton to increase my request handler performance?
It is impossible to share a cosocket object (and, therefore, a redis object, check this answer for details) between different requests:
The cosocket object created by this API function has exactly the same lifetime as the Lua handler creating it. So never pass the cosocket object to any other Lua handler (including ngx.timer callback functions) and never share the cosocket object between different Nginx requests.
However, nginx/ngx_lua uses a connection pool internally:
Before actually resolving the host name and connecting to the remote backend, this method will always look up the connection pool for matched idle connections created by previous calls of this method
That being said, you just need to use sock:setkeepalive() instead of sock:close() for persistent connections. The redis object interface has a corresponding method: red:set_keepalive().
You'll still need to create a redis object on a per request basis, but this will help to avoid a connection overhead.
Related
In this situation, I'm using all standard Go libraries -- net/http, most importantly.
The application consists of two layers. The first layer is the basic web application. The web application serves out the UI, and proxies a bunch of API calls back to the second layer based on username -- so, it's effectively a load balancer with consistent hashing -- each user is allocated to one of these second-layer nodes, and any requests pertaining to that user must be sent to that particular node.
Quick details
These API endpoints in the first layer effectively read in a JSON body, check the username, use that to figure out which of the layer 2 nodes to send the JSON body to, and then it sends it there. This is done using a global http.Client that has timeouts set on it, as appropriate.
The server side does a defer request.Body.Close() in each of the handlers after ensuring no error comes back from decoder.Decode(&obj) calls that unmarshal the JSON. If there is any codepath where that could happen, it isn't one that's likely to get followed very often.
Symptoms
On the node in the second layer (the application server) I get log lines like this because it's leaking sockets presumably and sucking up all the FDs:
2019/07/15 16:16:59 http: Accept error: accept tcp [::]:8100: accept4: too many open files; retrying in 1s
2019/07/15 16:17:00 http: Accept error: accept tcp [::]:8100: accept4: too many open files; retrying in 1s
And, when I do lsof 14k lines are output, of which 11,200 are TCP sockets. When I look into the contents of lsof, I see that nearly all these TCP sockets are in connection state CLOSE_WAIT, and are between my application server (second layer node) and the web server (the first layer node).
Interestingly, nothing seems to go wrong with the web application server (layer 1) during this timeframe.
Why does this happen?
I've seen lots of explanations, but most either point out that you need to specify custom defaults on a custom http.Client and not use the default, or they tell you to make sure to close the request bodies after reading from them in the layer 2 handlers.
Given all this information, does anyone have any idea what I can do to at least put this to bed once and for all? Everything I search on the internet is user error, and while I certainly hope that's the case here, I worry that I've nailed down every last quirk of the Go standard library I can find.
Been having trouble nailing down exactly how long it takes for this to happen -- the last time it happened, it was up for 3 days before I started to see this error, and at that point obviously nothing recovers until I kill and restart the process.
Any help would be hugely appreciated!
EDIT: example of client-side code
Here is an example of what I'm doing in the web application (layer 1) to call the layer 2 node:
var webHttpClient = &http.Client{
Transport: &http.Transport{
MaxIdleConnsPerHost: MaxIdleConnections,
},
Timeout: time.Second * 20,
}
// ...
uri := fmt.Sprintf("http://%s/%s", tsUri, "pms/all-venue-balances")
req, e := http.NewRequest("POST", uri, bytes.NewBuffer(b))
resp, err := webHttpClient.Do(req)
if err != nil {
log.Printf("Submit rebal error 3: %v\n", err)
w.WriteHeader(500)
return
}
defer resp.Body.Close()
body, _ := ioutil.ReadAll(resp.Body)
w.WriteHeader(200)
w.Write(body)
I'm using Akka's HTTP client to make a connection to an infinitely streaming HTTP endpoint. I am having difficulty getting the client to close the upstream to the HTTP server.
Here's my code (StreamRequest().stream returns a Source[T, Any]. It's generated by Http().outgoingConnectionHttps and then a Flow[HttpResponse, T, NotUsed] to convert HttpResponse to a stream of T):
(killSwitch, tFuture) = StreamRequest()
.stream
.takeWithin(timeToStreamFor)
.take(toPull)
.viaMat(KillSwitches.single)(Keep.right)
.toMat(Sink.seq)(Keep.both)
.run()
Then I have
tFuture.onComplete { _ =>
info(s"Shutting down the connection")
killSwitch.shutdown()
}
When I run the code I see the 'Shutting down the connection' log message but the server tells me that I'm still connected. It disconnects only when the JVM exits.
Any ideas what I'm doing wrong or what I should be doing differently here?
Thanks!
I suspect you should invoke Http().shutdownAllConnectionPools() when tFuture completes. The pool does not close connections because they can be reused by the different stream materialisations, so when the stream completes it does not close the pool. The shut connection you seen in the log can be because the idle timeout has triggered for one of the connections.
I've been implementing a kong plugin that needs to make HTTP requests to retrieve information to share it with the upstream services.
There is an excellent library called lua-resty-http that can be used to make HTTP requests.
The service that contains the information needed, it is configured behind the proxy, and it matches the path: /endpoint-providing-info.
The goal is to rely on the proxy capabilities to avoid having to parse the hostname which has some particular form that is not relevant to this question.
By playing around, I was able to achieve the desired behavior by doing the following:
local ok, err = http_client:connect("127.0.0.1", ngx.var.server_port)
if not ok and err then return nil, 'there was a failure opening a connection: ' .. err
local res, err = http_client:request({
method = 'GET',
path = '/endpoint-providing-info'
})
//parse the response, etc...
The request gets routed to the upstream service and works as expected.
My primary concern is the following:
By connecting to localhost, I assumed that the current Nginx node is the one attending the request. Will this affect the performance? Is it better/possible to connect to the cluster directly?
I suppose that you configure for current nginx a location which match /endpoint-providing-info, use proxy module and configure an upstream for a cluster.
If you would use lua-resty-http:
Pros - you may use body_reader - an iterator function for reading the body in a streaming fashion.
Cons - your request will go thru kernel boundary (loopback interface).
Another possibility is to issue a subrequest using ngx.location.capture API
Pros - subrequests just mimic the HTTP interface but there is no extra HTTP/TCP traffic nor IPC involved. Everything works internally, efficiently, on the C level.
Cons - it is full buffered approach, will not work efficiently for big responses.
Update - IMO:
If you expect from upstream server big responses -lua-resty-http is your choice.
If you expect from upstream server a lot of small responses - ngx.location.capture should be used.
From the introduction on gRPC:
In gRPC a client application can directly call methods on a server application on a different machine as if it was a local object, making it easier for you to create distributed applications and services. As in many RPC systems, gRPC is based around the idea of defining a service, specifying the methods that can be called remotely with their parameters and return types. On the server side, the server implements this interface and runs a gRPC server to handle client calls. On the client side, the client has a stub that provides exactly the same methods as the server.
The above paragraph talks about a client and a server, with the former being the one who is invoking methods to the other. What am I wondering is: can the server-end of the connection invoke methods that have been registered on the client?
No, a server cannot invoke calls on the client. gRPC works with HTTP, and HTTP has not had such semantics in the past.
There has been discussion as to various ways to achieve such a feature, but I'm unaware of any work having started or general agreement on a design. gRPC does support bidirectional streaming, which may get you some of what you need. With bidirectional streaming the client can respond to messages from server, but the client still calls the server and only one type of message can be sent for that call.
The protocol does not implement it, but you may pretend this situation.
Define a server method that returns a stream of a ServerRequest message:
import "google/protobuf/any.proto";
service FullDuplex {
rpc WaitRequests (google.protobuf.Any) returns (stream ServerRequest);
}
message ServerRequest {
float someValue = 1;
float anotherAnother = 1;
}
ServerRequest may be an Oneof, so your may receive different types of server requests.
If you need that your client sends back a response for each request, you may create a stream from your client to the server, but you will need to implement a logic in your server side that triggers a timeout waiting for that response.
service FullDuplex {
rpc WaitRequests (stream ClientResponse) returns (stream ServerRequest);
}
What you can do is start a HTTP server in both processes and use clients at each end to initiate communication. There's a bit of boilerplate involved and you have to design a simple handshaking protocol (one end registers with the other, advertising its listen address) but it's not too much work.
I use CloudFlare for one of my high volume websites, and it sits in front of my stack.
The thing is CloudFlare leaves idle connections open in addition to creating new ones, and it's not a setting I can change.
When I have Varnish or Nginx sitting in front listening on port 80 they have out of the box configuration to hang up the idle connections.
This is fine until I had to add a proxy written in Go to the front of my stack. It uses the net/http standard library.
I'm not a Go wizard but based on what people are telling me there are only read and write timeout settings but not hanging up idle connections.
Now my server will fill up with connections and die unless I set a set read and write timeouts, but the problem with this is my backend takes a long time sometimes and it's causing good requests to be cut off when they shouldn't.
What is the proper way to handle idle connections with Go http?
Edit 1: To be more clear, I'm using httputil.NewSingleHostReverseProxy to construct a proxy, which exposes transport options but only for the upstream. The problems I am having are downstream, they need to be set on the http.Server object that uses the ReverseProxy as a handler. http.Server does not expose transport.
Edit 2: I would prefer an idle timeout to a read timeout, since the later would apply to an active uploader.
Thanks
The proper way to hangup idle connections in the Go http server is to set the read timeout.
It is not necessary to set the write timeout to hang up on idle clients. Don't set this value or adjust it up if it's cutting off responses.
If you have long uploads, then use a connection state callback to implement separate idle and read timeouts:
server.ConnState = func(c net.Conn, cs http.ConnState) {
switch cs {
case http.StateIdle, http.StateNew:
c.SetReadDeadline(time.Now() + idleTimeout)
case http.StateActive:
c.SetReadDeadline(time.Now() + activeTimeout)
}
}
See the net/http.Transport docs. The Transport type has some options for dealing with idle HTTP connections in the keep-alive state. From reading your question, the option that seems most relevant to your problem is the MaxIdleConnsPerHost field:
MaxIdleConnsPerHost, if non-zero, controls the maximum idle (keep-alive) to keep per-host. If zero, DefaultMaxIdleConnsPerHost is used.
Reading the code, the default is 2 per host.
The Transport type also has a method to zap all idle connections: CloseIdleConnections.
CloseIdleConnections closes any connections which were previously connected from previous requests but are now sitting idle in a "keep-alive" state. It does not interrupt any connections currently in use.
You can specify a Transport on any http client:
tr := &http.Transport{
TLSClientConfig: &tls.Config{RootCAs: pool},
DisableCompression: true,
MaxIdleConnsPerHost: 1,
}
client := &http.Client{Transport: tr}
resp, err := client.Get("https://example.com")
Another thing worth noting: the docs recommend that you keep a single http client object that is re-used across all your requests (i.e. like a global variable).
Clients and Transports are safe for concurrent use by multiple goroutines and for efficiency should only be created once and re-used.
If you are creating many http client objects in your proxy implementation, it might explain unbounded growth of idle connections (just guessing at how you might be implementing this, though).
EDIT: Reading a little bit more, the net/httputil package has some convenience types for reverse proxies. See the ReverseProxy type. That struct also allows you to supply your own Transport object, allowing you to control your proxy's idle client behavior via this helper type.