I'm trying to start a simple server in Julia. (Julia 1.7, HTTP v0.9.16, Windows 10).
By copying the snippet in the HTTP docs, I've written
using HTTP
using Sockets
HTTP.serve(Sockets.localhost, 8000) do request::HTTP.Request
#show HTTP.header(request, "Content-Type")
#show HTTP.payload(request)
return HTTP.Response("Helllllo")
end
in a julia terminal.
When i navigate to http://localhost:8000/ in Chrome, it Does show a page with the word "Helllllo".
But, when i try to send GET requests to it, it doesn't answer. Things I tried:
Opening a new julia terminal and writing
using HTTP
r = HTTP.request("GET", "https://127.0.0.1:8000");
What happens is that this command hangs, instead of producing a r.
Going in Chrome console and writing
fetch("https://127.0.0.1:8000")
.then(res => res.json())
.then(console.log)
Again, no answer.
What am I doing wrong?
Thanks!
A couple different issues:
You're trying to read from a https address, after having set up a plain http server. Once you fix this, Julia is able to get a request-response:
julia> r = HTTP.request("GET", "http://127.0.0.1:8000")
HTTP.Messages.Response:
"""
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Helllllo"""
But it still fails in the browser (in Firefox - I assume it will be the same in Chrome) with
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://127.0.0.1:8000/. (Reason: CORS request did not succeed).
This is simply because 127.0.0.1 != localhost for browser, for security purposes (so it doesn't allow you to read from the "different" host at 127.0.0.1 while you're on localhost). If you instead do:
>> fetch("http://localhost:8000")
.then(console.log)
Promise { <state>: "pending" }
Response { type: "basic", url: "http://localhost:8000/", redirected: false, status: 200, ok: true, statusText: "OK", headers: Headers, body: ReadableStream, bodyUsed: false }
Related
I have a flask server with and endpoint that processes some uploaded .csv files and returns a .zip (in a JSON reponse, as a base64 string)
This process can take up to 90 seconds
I've been setting it up for production using gunicorn and nginx and I'm testing the endpoint with smaller .csv s. They get processed fine and in a couple seconds I get the "got blob" log. But nginx doesn't return it to the client and finally it times out. I set up a longer fail-timeout of 10 minutes and the client WILL wait 10 minutes, then time out
the proxy read timeout offered as solution here is set to 3600s
Also the proxy connect timeout is set to 75s according to this
also the timeout for the gunicorn workers according to this
The error log says: "upstream timed out connection timed out while reading upstream"
I also see examples of nginx receiving an OPTIONS request and immediately after the POST request (some CORS weirdness from the client) where nginx passes the OPTIONS request but fails to pass the POST request to gunicorn despite nginx having received it
Question:
What am I doing wrong here?
Many thanks
http {
upstream flask {
server 127.0.0.1:5050 fail_timeout=600;
}
# error log
# 2022/08/18 14:49:11 [error] 1028#1028: *39 upstream timed out (110: Connection timed out) while reading upstream, ...
# ...
server {
# ...
location /api/ {
proxy_pass http://flask/;
proxy_read_timeout 3600;
proxy_connect_timeout 75s;
# ...
}
# ...
}
}
# wsgi.py
from main import app
if __name__ == '__main__':
app.run()
# flask endpoint
#app.route("/process-csv", methods=['POST'])
def process_csv():
def wrapped_run_func():
return blob, export_filename
# ...
try:
blob, export_filename = wrapped_run_func()
b64_file = base64.b64encode(blob.getvalue()).decode()
ret = jsonify(file=b64_file, filename=export_filename)
# return Response(response=ret, status=200, mimetype="application/json")
print("got blob")
return ret
except Exception as e:
app.logger.exception(f"0: Error processing file: {export_filename}")
return Response("Internal server error", status=500)
ps. getting this error from stackoverflow
"Your post appears to contain code that is not properly formatted as code. Please indent all code by 4 spaces using the code toolbar button or the CTRL+K keyboard shortcut. For more editing help, click the [?] toolbar icon."
for having perfectly well formatted code with language syntax, I'm sorry that I had to post it ugly
Sadly I got no response
See last lines for the "solution" finally implemented
CAUSE OF ERROR: I believe the problem is that I'm hosting the Nginx server on wsl1
I tried updating to wsl2 and see if that fixed it but I need to enable some kind of "nested virtualization", as the wsl1 is running already on a VM.
Through conf changes I got it to the point where no error is logged, gunicorn return the file then it just stays in the ether. Nginx never gets/sends the response
"SOLUTION":
I ended up changing the code for the client, the server and the nginx.conf file:
the server saves the resulting file and only returns the file name
the client inserts the filename into an href that then displays a link
on click a request is sent to nginx which in turn just sends the file from a static folder, leaving gunicorn alone
I guess this is the optimal way to do it anyway, though it still bugs me I couldn't (for sure) find the reason of the error
I need to perform a get request from nifi to couchbase. The curl command is:
curl http://HOST:PORT/query/service -d "statement=select item.Date from bucket unnest bucket as item" -u USER:PASSWORD
I tried using InvokeHttp and ExecuteStreamCommand but it keeps returning errors(status code 400). The full error message is:
{ "requestID": "bff62c0b-36fd-401d-bca0-0959e0944323", "errors":
[{"code":1050,"msg":"No statement or prepared value"}], "status":
"fatal", "metrics": {"elapsedTime": "113.31µs","executionTime":
"74.321µs","resultCount": 0,"resultSize": 0,"errorCount": 1
It's important to say that I prefer that the http request will be triggered by an incoming flowfile. I tried using the processors in various of ways but non of them worked.
When I run the command from the nifi server it works fine.
Thanks for the help
the -d parameter of the curl utility forces HTTP POST command
and application/x-www-form-urlencoded mime-type.
so, in the nifi InvokeHTTP select the following parameters
HTTP Method = POST
Remote URL = <your url here>
Basic Authentication Username = <username>
Basic Authentication Password = <password>
Content-Type = application/x-www-form-urlencoded
and the body of the flow file should be
statement=select item.Date from bucket unnest bucket as item
I don't know nifi, but based on the error message, the "statement=" part of the request isn't being included, or you are not sending the request as a POST command.
I'm trying to make a get request from Arduino to GAE using PHP.
Arduino code:
#define DST_IP "https://myservice-183310.appspot.com" // my gae address
String cmd;
Wifi.println("AT+CIPMUX=0");
cmd = "AT+CIPSTART=\"TCP\",\"";
cmd += DST_IP;
cmd += "\",80\r\n";
Wifi.println(cmd);
if(Wifi.find("OK")){
Serial.println("LINK SUCCESS");
}
else{
Serial.println("LINK ERROR");
}
String user_info="GET /register.php?nfc_id=";
user_info+=nfc_id;
user_info+="\r\nHTTP/1.0 HOST: ";
user_info+=DST_IP;
user_info+="\r\n\r\n";
cmd="AT+CIPSEND=";
cmd+=String(user_info.length());
Wifi.println(cmd);
Wifi.print(user_info);
app.yaml in GAE:
runtime: php55
api_version: 1
handlers:
- url: /register.php
script: register.php
And I get a 404 error message, but "https://myservice-183310.appspot.com/register.php?nfc_id=ooo" works in the browser.
Why do I get this error and how can I fix it?
A valid HTTP request should look like:
GET /request.php?nfc_id=MY_NFC_ID HTTP/1.0
Host: myservice-183310.appspot.com
I'm no Arduino coder, but it looks like the request you are sending is:
GET /register.php?nfc_id=MY_NFC_ID
HTTP/1.0 HOST: https://myservice-183310.appspot.com
Things to note
The HTTP/1.0 goes on the same line as the GET, not on its own line.
The Host: HTTP header should not have the protocol https. SSL or non-SSL will already have been handled before these headers can be read, so they are unneeded (and so might only cause trouble).
I think the issue is the "HTTPS" protocol, try with "HTTP".
I've just started working with the Quectel MC60 and I am having some issues:
About HTTP GET method, I make the following commands:
AT+QIFGCNT=0
AT+QICSGP=1,"my_apn"
AT+QIREGAPP
AT+QIACT
AT+QSSLCFG="https",1
AT+QHTTPURL=39,40
my_url_39_bytes_long
AT+QHTTPGET=60
AT+QHTTPREAD=30
AT+QIDEACT
When using the QCOM software, I make a script running all the above commands sequentially. When it comes to the AT+QHTTPREAD command, the response is always "+CME ERROR: 3822" (HTTP response failed). What can it be? I'm sure the HTTP server is working properly.
The answer is that it is necessary to configure the request header
AT+QIFGCNT=0
AT+QICSGP=1,"my_apn"
AT+QIREGAPP
AT+QIACT
AT+QHTTPURL=39,40
my_url_39_bytes_long
AT+QHTTPCFG="requestheader",1
AT+QHTTPPOST=77
GET path HTTP/1.1
User-Agent: Fiddler
Host: www.my_host.com
AT+QHTTPREAD=30
AT+QIDEACT
NOTE: in AT+HTTPPOST=77, 77 is the size of the POST message (last two \r\n are required and count)
NOTE2: after GET you're supposed to write the path to the url inserted in AT+QHTTPURL. For example, if you specified your URL as https://www.my_host.com/debug/main/port, your AT+HTTPPOST request should look like this (don't forget the last two \r\n):
GET /debug/main/port HTTP/1.1
User-Agent: Fiddler
Host: www.my_host.com
I have a service that redirects users to temporary pre-signed AWS downloads. These are large files, often 5-10gb. To prevent download sharing, we have a relatively short (30 seconds) valid lifespan.
Everything is working except that on slow internet connections, they tend to fail or get interrupted. wget has a feature that automatically retries the download. However, instead of retrying the original URL (eg: http://service.com/download/file.zip), wget retries the redirected pre-signed URL (eg: http://service.s3.amazonaws.com/file.zip?AWSAccessKeyId=XXXX&Signature=XXXX&Expires=1468000000)
Since these are large files, and the pre-signed lifespan is so short, that temporary url is no longer valid and the user gets a 403 Forbidden result.
Originally, when we noticed the problem, we were using 302 Found temporary redirects. A little research seemed to indicate we SHOULD have been using 307 Temporary Redirect. However, that didn't resolve the problem with wget. For grins and giggles, we tried 303 See Other, but that didn't work either.
Does anyone have any idea how get wget to retry the original URL instead of the redirected URL?
below is a wget example log:
--2016-07-06 10:29:51-- https://service.com/download/file.zip
Connecting to service.com (service.com)|10.0.0.1|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location:
https://service.s3.amazonaws.com/file.zip?AWSAccessKeyId=XXXX&Signature=XXXX&Expires=1468000000
[following]
--2016-07-06 10:29:52-- https://service.s3.amazonaws.com/file.zip?AWSAccessKeyId=XXXX&Signature=XXXX&Expires=1468000000
Resolving service.s3.amazonaws.com (service.s3.amazonaws.com)...
54.231.12.129
Connecting to service.s3.amazonaws.com
(service.s3.amazonaws.com)|54.231.12.129|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2070666907 (1.9G) [application/zip]
Saving to: ‘file.zip’
file.zip 53%[=========> ] 1.03G --.-KB/s in 18m 7s
2016-07-06 10:47:59 (995 KB/s) - Read error at byte
1107205784/2070666907 (The specified session has been invalidated for
some reason.). Retrying.
--2016-07-06 10:48:00-- (try: 2) https://service.s3.amazonaws.com/file.zip?AWSAccessKeyId=XXXX&Signature=XXXX&Expires=1468000000
Connecting to service.s3.amazonaws.com
(service.s3.amazonaws.com)|54.231.12.129|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2016-07-06 10:48:01 ERROR 403: Forbidden.
I had a similar issue, and a similar answer as #panzerito, but broke it up into a script i called loopdone
#!/bin/bash
until `$1`; do sleep 1; echo restarting; done
then I can just do loopdone "wget -c http://my.url/" (incl quotes) to force it to run again and again (and resume, unless server does not support it) until exit code is 0. (meaning no error)
Bash-code:
initial_error_EXIT_STATUS; until [ "$?" -eq "0" ]; do wget https://example.com/download/file.zip -c; done