I reach a limit in the number of requests I can get using the default httpClient that my API wrapper provides.
//I was getting something like this at the beginning
Head www.example.com:80/: lookup example.com: no such host
To solve this I want to change the MaxIdleConnsPerHost setting for the httpClient.Transport of my client. It looks much more like this:
However, the Transport my client uses is a RoundTripper and subsequently, it doesn't have a MaxIdleConnsPerHost param.
&oauth2.Transport{Source:(*oauth2.reuseTokenSource)(0xc2082ac0c0),
Base:http.RoundTripper(nil),
mu:sync.Mutex{state:0, sema:0x0},
modReq:map[*http.Request]*http.Request(nil)
}
The one I'm providing is mostly a default one and it lacks the proper configuration I suppose or simple what I want to do is not feasible.
&http.Transport{idleMu:sync.Mutex{state:0, sema:0x0},
idleConn:map[http.connectMethodKey][]*http.persistConn(nil),
idleConnCh:map[http.connectMethodKey]chan *http.persistConn(nil),
reqMu:sync.Mutex{state:0, sema:0x0},
reqCanceler:map[*http.Request]func()(nil),
altMu:sync.RWMutex{w:sync.Mutex{state:0, sema:0x0},
writerSem:0x0,
readerSem:0x0,
readerCount:0,
readerWait:0},
altProto:map[string]http.RoundTripper(nil),
Proxy:(func(*http.Request) (*url.URL, error))(nil),
Dial:(func(string, string) (net.Conn, error))(nil),
TLSClientConfig:(*tls.Config)(nil),
TLSHandshakeTimeout:0,
DisableKeepAlives:false,
DisableCompression:false,
MaxIdleConnsPerHost:200,
ResponseHeaderTimeout:0}
Can someone guide me on the right direction?
Related
In order to connect my bot to the local bot API server and in order to turn curl's SSL verify_host and verify_peer off, how can I change irazasyed/telegram-bot-sdk's default request values of BASE_BOT_URL and the handler's verify option, not hard coding?
I know that I can change the BASE_BOT_URL constant in telegram-bot-sdk's TelegramClient class. And I know that I can change the verify option of my request handler to be false in guzzle's CurlFactory class. But, I'm not interested in hard-coding or overwriting these vendor files to avoid any update challenges in the future. I am more interested in something like longman/telegram-bot's setCustomBotApiUri function. The closest I could get is working with telegram-bot-sdk's setHttpClientHandler function, but I'm having trouble how to work with that.
Appreciate any help
P.S. I am using irazasyed/telegram-bot-sdk:^3.6 in my laravel:^9.2 app.
I started to use LocustIO for load testing a 3rd party API which provides a way to do batch requests (http://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_BatchRequests).
How can this be done using LocustIO?
I tried with the following:
def batch(self):
response = self.client.request(method="POST", url="/$batch", auth=("ABC", "DEF"), headers={"ContentType": "multipart/mixed; boundary=batch_36522ad7-fc75-4b56-8c71-56071383e77b"}, data="Content-Type: application/http\nContent-Transfer-Encoding: binary\n\nGET putyoururlhere HTTP/1.1\nAccept: application/json\n\n\n")
Auth is something I need to have authentication to the API, but that's not the point of the question and "putyoururlhere" should be replaced with the actual url. Either way, it gives errors when executing the test, so I must be doing something wrong.
People with experience how to do this?
Kind regards!
The data parameter should be your POST body (only), you cant put additional headers in it the way you did. You probably just want to add them as additional entries in the dict you pass as headers
Se the documentation for python requests library for more details. https://requests.readthedocs.io/en/master/
I'm new to nginx and I'm trying to develope a simple nginx module,a handler module specifically. Although it's not what I really wanna do,I try to finish this task first. I'm trying to get the socketfd when a browser(or a client) connects to nginx.And I have get it successfully.However, when I tried to output something using dup2(),the nginx is always pending and just outputs nothing.Sometimes I can get output after a long time and once I stop nginx like nginx -s stop,and the output appears immediately.
Like this:
reach http://100.100.60.199/nc?search=123456
get
search=123456 HTTP/1.l
HOST
output
I have read some blogs about nginx module and I found that handler module has its own pattern (to my understanding?).For example, the output should be nginx_chain_t,and I should construct that chain instead of using dup2 like a regular c code.So I wonder if it's feasible to get output like the function below.
Here is my handler function:
static ngx_int_t ngx_http_nc_handler(ngx_http_request_t *r){
//ngx_int_t rc;
ngx_socket_t connfd = r->connection->fd;
int nZero=0;
//if(setsockopt(connfd,SOL_SOCKET,SO_SNDBUF,(const void*)&nZero,sizeof(nZero))==0)
if(setsockopt(connfd,IPPROTO_TCP,TCP_NODELAY,(const void*)&nZero,sizeof(int))==0){
setbuf(stdout,NULL);
setbuf(stdin,NULL);
setbuf(stderr,NULL);
dup2(connfd,STDOUT_FILENO);
dup2(connfd,STDERR_FILENO);
dup2(connfd,STDIN_FILENO);
printf("%s\n", r->args.data);
//close(connfd);
}
return NGX_OK;
}
So I wonder if it's feasible,how can I get things right using the method above or can anybody just say it's impossible and construct a chain is the only way?
I finally solved this problem by trying to understand how exactly nginx works.In short,all I need to do is to add a http header to the output.But it's not that easy like what I described.
When I attempt to use SWI-Prolog's http_post/4, as follows:
:- use_module(library(http/http_client).
update(URL, Arg) :-
http_post(URL, form([update = Arg), _, [status_code(204)]).
When I query this rule, and watch the TCP traffic, I see the HTTP POST request and reply with the expected 204 status code both occur immediately. However, Prolog hangs for up to 30 seconds before returning back 'true'. What is happening that prevents the rule from immediately returning?
I've tried this variant as well, but it also hangs:
:- use_module(library(http/http_client).
update(URL, Arg) :-
http_post(URL, form([update = Arg), Reply, [status_code(204)]),
close(Reply).
I have a similar issue with http_delete/3, but not with http_get/3.
library docs state that http_post
It is equivalent to http_get/3, except for providing an input document, which is posted using http_post_data/3.
http_get has timeout(+Timeout) in its options. That could help to lower the latency, but as it is set to +infinite by default, I fear will not solve the issue. Seems like the service you are calling keeps the connection alive up to some timeout.
Personally I had to use http_open, instead of http_post, when calling Google API services on https...
I'm looking for some kind of documentation that specifies how much time each browser (IE6/IE7/FF2/FF3, etc) will wait on a request before it just gives up and times out.
I haven't had any luck trying to get this.
Any pointers?
I managed to find network.http.connect.timeout for much older versions of Mozilla:
This preference was one of several
added to allow low-level tweaking of
the HTTP networking code. After a
portion of the same code was
significantly rewritten in 2001, the
preference ceased to have any effect
(as noted in all.js as early as
September 2001).
Currently, the timeout is determined
by the system-level connection
establishment timeout. Adding a way to
configure this value is considered
low-priority.
It would seem that network.http.connect.timeout hasn't done anything for some time.
I also saw references to network.http.request.timeout, so I did a Google search. The results include lots of links to people recommending that others include it in about:config in what appears to be a mistaken belief that it actually does something, since the same search turns up this about:config entries article:
Pref removed (unused).
Previously: HTTP-specific network
timeout. Default value is 120.
The same page includes additional information about network.http.connect.timeout:
Pref removed (unused).
Previously: determines how long to
wait for a response until registering
a timeout. Default value is 30.
Disclaimer: The information on the MozillaZine Knowledge Base may be incorrect, incomplete or out-of-date.
After the last Firefox update we had the same session timeout issue and the following setting helped to resolve it.
We can control it with network.http.response.timeout parameter.
Open Firefox and type in ‘about:config’ in the address bar and press Enter.
Click on the "I'll be careful, I promise!" button.
Type ‘timeout’ in the search box and network.http.response.timeout parameter will be displayed.
Double-click on the network.http.response.timeout parameter and enter the time value (it is in seconds) that you don't want your session not to timeout, in the box.
firstly I don't think there is just one solution to your problem....
As you know each browser is vastly differant.
But lets see if we can get any closer to the answer you need....
I think IE Might be easy...
Check this link
http://support.microsoft.com/kb/181050
For Firefox try this:
Open Firefox, and in the address bar, type "about:config" (without quotes). From there, scroll down to the Network.http.keep-alive and make sure that is set to "true". If it is not, double click it, and it will go from false to true. Now, go one below that to network.http.keep-alive.timeout -- and change that number by double clicking it. if you put in, say, 500 there, you should be good. let us know if this helps at all
For Google Chrome (Tested on ver. 62)
I was trying to keep a socket connection alive from the google chrome's fetch API to a remote express server and found the request headers have to match Node.JS's native <net.socket> connection settings.
I set the headers object on my client-side script with the following options:
/* ----- */
head = new headers();
head.append("Connnection", "keep-alive")
head.append("Keep-Alive", `timeout=${1*60*5}`) //in seconds, not milliseconds
/* apply more definitions to the header */
fetch(url, {
method: 'OPTIONS',
credentials: "include",
body: JSON.stringify(data),
cors: 'cors',
headers: head, //could be object literal too
cache: 'default'
})
.then(response=>{
....
}).catch(err=>{...});
And on my express server I setup my router as follows:
router.head('absolute or regex', (request, response, next)=>{
req.setTimeout(1000*60*5, ()=>{
console.info("socket timed out");
});
console.info("Proceeding down the middleware chain link...\n\n");
next();
});
/*Keep the socket alive by enabling it on the server, with an optional
delay on the last packet sent
*/
server.on('connection', (socket)=>socket.setKeepAlive(true, 10))
WARNING
Please use common sense and make sure the users you're keeping the socket connection open to is validated and serialized. It works for Firefox as well, but it's really vulnerable if you keep the TCP connection open for longer than 5 minutes.
I'm not sure how some of the lesser known browsers operate, but I'll append to this answer with the Microsoft browser details as well.