I have the following get in my restlet app:
#Get
public String represent(Variant variant) throws ResourceException
{
String text = "returntext";
text+="\r\n";
return text;
}
When checking the response from invoking this service, I get the following:
CFG - HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Type: text/plain;charset=UTF-8
Date: Mon, 29 Jul 2013 19:59:37 GMT
Server: Restlet-Framework/2.0.9
Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
Content-Length: 118
Connection: keep-alive
Shouldn't the Connection be: closed as opposed to keep-alive?
I think is causing the connection to get tied up because I can only invoke the service once.
#Ab is correct. http 1.1 has a keep alive. This was the problem.
Related
I have a problem that relates to the authentication with the google.
I followed the tutorials and finally managed to reach google site where I can select the choose an account section. Then I select one and get redirect to the local address: https://localhost:44377/signin-google?state ...
but there I get:
An unhandled exception occurred while processing the request.
Exception: OAuth token endpoint failure: Status: Unauthorized;Headers: Cache-Control: max-age=0, private
Date: Sat, 10 Feb 2018 22:14:21 GMT
Transfer-Encoding: chunked
Accept-Ranges: none
Server: GSE
Vary: X-Origin, Origin, Accept-Encoding
WWW-Authenticate: Bearer realm="https://accounts.google.com/"
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Alt-Svc: hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35"
;Body: {
"error": "invalid_client",
"error_description": "Unauthorized"
}
;
Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler+d__12.MoveNext()
Any help in investigating this would be very helpful.
I have script which coomunicates between nodemcu and my server. It works good on my localhost and is parsing response retrieved from my server when I send GET request. Problem is when I upload it all on my website where transfer encoding is chunked. I am not able to retrieve content, although request is legitimate and correct. Code is written in Lua and I am trying to work on my NodeMCU device.
conn=net.createConnection(net.TCP, 0)
conn:on("connection",function(conn, payload)
conn:send("GET /mypath/node.php?id=1&update"..
" HTTP/1.1\r\n"..
"Host: www.mydomain.com\r\n"..
"Accept: */*\r\n"..
"User-Agent: Mozilla/4.0 (compatible; esp8266 Lua;)"..
"\r\n\r\n")
end)
conn:on("receive", function(conn, payload)
if string.find(payload, "UPDATE")~=nil then
node.restart()
end
conn:close()
conn = nil
end)
conn:connect(80,"www.mydomain.com")
end
Just to repeat that this GET request works and is tested manualy and on localhost. Only problem is with chunked content, I don't know how to parse it.
Update: I managed to remove chunked encoding by changing HTTP/1.1 to HTTP/1.0, but still I have problem
using this code
conn:on("receive", function(conn, payload)
print(payload)
I get this response
HTTP/1.1 200 OK
Date: Tue, 09 Jan 2018 02:34:25 GMT
Server: Apache
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=9m226vr20r4baa634bagk8k2k3; path=/
Connection: close
Content-Type: text/html; charset=utf-8
Update 2.
I have just created one file http.php with text included "php". I have uploaded it to localhost and to my domain. Once I tried to access my localhost from nodemcu, and then to domain. Results were different
This is the request
conn:send("GET /"..s.path.."/http.php"..
" HTTP/1.0\r\n"..
"Host: "..s.domain.."\r\n"..
"Accept: */*\r\n"..
"User-Agent: Mozilla/4.0 (compatible; esp8266 Lua;)"..
"\r\n\r\n")
end)
s.domain and s.path correcponds to different paths and domains on localhost and my domain
Result on domain
HTTP/1.1 200 OK
Date: Tue, 09 Jan 2018 03:09:28 GMT
Server: Apache
Connection: close
Content-Type: text/html; charset=UTF-8
result on localhost
TTP/1.1 200 OK
Date: Tue, 09 Jan 2018 03:08:48 GMT
Server: Apache/2.4.27 (Win64) PHP/7.0.23
X-Powered-By: PHP/7.0.23
Content-Length: 3
Connection: close
Content-Type: text/html; charset=UTF-8
php
As you can see, localhost is showing content "php", and domain is showing only header. When I type some file which does not exists domain is showing me html code.
I'm using the following code to put the chunks together. I'm wondering anyways, why your response from the server is missing the Content-Length header.
conn:on("receive", function(client, payload)
-- Inspired by https://github.com/marcoskirsch/nodemcu-httpserver/blob/master/httpserver.lua
-- Collect data packets until the size of HTTP body meets the Content-Length stated in header
if payload:find("Content%-Length:") or bBodyMissing then
if fullPayload then fullPayload = fullPayload .. payload else fullPayload = payload end
if (tonumber(string.match(fullPayload, "%d+", fullPayload:find("Content%-Length:")+16)) > #fullPayload:sub(fullPayload:find("\r\n\r\n", 1, true)+4, #fullPayload)) then
bBodyMissing = true
return
else
payload = fullPayload
fullPayload, bBodyMissing = nil
end
end
if (bBodyMissing == nil) then
local _, headerEnd = payload:find("\r\n\r\n")
local body = payload:sub(headerEnd + 1)
print (body)
end
end)
I am trying to simply download a page with python.
http://webapps.rrc.state.tx.us/CMPL/viewPdfReportFormAction.do?method=cmplP4FormPdf&packetSummaryId=97770
If i get the response code from the server i get 200
import urllib2
url = 'http://webapps.rrc.state.tx.us/CMPL/viewPdfReportFormAction.do?method=cmplP4FormPdf&packetSummaryId=97770'
file_pointer = urllib2.urlopen(url)
print file_pointer.getcode()
However if i get the url i get the redirect page
file_pointer.geturl()
I have tried urllib, urllib2,requests, and mechanize all separately and can not get any to work. I am obviously missing something because other people in the office have code that works. SOS
Also here is more information provided by requests
import requests
url = 'http://webapps.rrc.state.tx.us/CMPL/viewPdfReportFormAction.do?method=cmplP4FormPdf&packetSummaryId=97770'
proxy = { 'https': '200.35.152.93:1212'}
response = requests.get(url, proxies=proxy)
send: 'GET /CMPL/viewPdfReportFormAction.do?method=cmplP4FormPdf&packetSummaryId=97770 HTTP/1.1\r\nHost: webapps.rrc.state.tx.us\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.7.0 CPython/2.7.10 Windows/7\r\n\r\n'
reply: 'HTTP/1.1 302 Found\r\n'
header: Date: Wed, 26 Aug 2015 19:33:12 GMT
header: Server: Apache/2.2.15 (Red Hat)
header: Location: http://www.rrc.state.tx.us/site-policies/railroad-commission-of-texas-site-policies/?method=cmplP4FormPdf&packetSummaryId=97770
header: Content-Length: 405
header: Connection: close
header: Content-Type: text/html; charset=iso-8859-1
send: 'GET /site-policies/railroad-commission-of-texas-site-policies/?method=cmplP4FormPdf&packetSummaryId=97770 HTTP/1.1\r\nHost: www.rrc.state.tx.us\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.7.0 CPython/2.7.10 Windows/7\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Cache-Control: private
header: Content-Type: text/html; charset=utf-8
header: server: one
header: Date: Wed, 26 Aug 2015 19:33:11 GMT
header: Content-Length: 41216
The problem is that this specific site is looking for your User Agent header, and since you're a python client, it disallows you to get the PDF and redirect you.
Therefore you need to mask your user agent.
Look at the following example:
url = 'http://webapps.rrc.state.tx.us/CMPL/viewPdfReportFormAction.do?method=cmplP4FormPdf&packetSummaryId=97770'
req = urllib2.Request(url)
req.add_unredirected_header('User-Agent', 'Mozilla/5.0')
file_pointer = urllib2.urlopen(req)
print file_pointer.getcode()
print file_pointer.geturl();
Okay so all one has to do with the requests module is to disable redirection.Here is my working code that is also using a proxy server.
import requests
url = 'http://webapps.rrc.state.tx.us/CMPL/viewPdfReportFormAction.do?method=cmplP4FormPdf&packetSummaryId=97770'
proxy = { 'https': '200.35.152.93:1212'}
r = requests.get(url, proxies=proxy,allow_redirects=False)
print r.url
I'm making the following request to a local website running in IIS
var httpRequestMessage = new HttpRequestMessage();
httpRequestMessage.RequestUri = new Uri("http://localhost:8081/");
httpRequestMessage.Method = HttpMethod.Get;
var response = new HttpClient().SendAsync(httpRequestMessage).Result;
This produces the following response headers:
HTTP/1.1 200 OK
Accept-Ranges: bytes
Date: Mon, 03 Jun 2013 22:34:25 GMT
ETag: "50c7472eb342ce1:0"
Server: Microsoft-IIS/8.0
X-Powered-By: ASP.NET
An identical request made via Fiddler produces the following response headers (I've highlighted the differences):
HTTP/1.1 200 OK
Content-Type: text/html
Last-Modified: Fri, 26 Apr 2013 19:20:58 GMT
Accept-Ranges: bytes
ETag: "50c7472eb342ce1:0"
Server: Microsoft-IIS/8.0
X-Powered-By: ASP.NET
Date: Mon, 03 Jun 2013 22:29:34 GMT
Content-Length: 10
Why is there a difference in response headers?
Am I using HttpClient correctly (aside from the fact I am calling Send synchronously)?
TL;DR;
To access all response headers you need to read both HttpResponseMessage.Headers and HttpResponseMessage.Content.Headers properties.
Long(er) answer:
This, basically:
var response = new HttpClient().GetAsync("http://uri/").Result;
var allHeaders = response.Headers.Union(response.Content.Headers);
foreach (var header in allHeaders)
{
// do stuff
}
I see two issues with this:
The Headers property is not appropriately named: it should really be SomeHeaders or AllHeadersExceptContentHeaders. (I mean, really, when you see a property named Headers, do you expect it to return all headers or some headers? I am pretty sure they are in violation of their own framework design guidelines on this one.)
The MSDN page does not mention at any point the fact this is a subset of all headers and developers should also inspect Content.Headers.
I get an error when initializing a RavenDB documentstore that contains a machinename in the URL, combined with specifying a non default database i.e Multitenant.
I can only get store.Initialize() to work if I do not specify a default database.
My ConnectionString is http://somemachinename:8080;ResourceManagerId=d5723e19-92ad-4531-adad-8611e6e05c8b
Following code fails
var store = new DocumentStore
{
ConnectionStringName="RavenDB,
DefaultDatabase="TNW_Grc_CLient",
EnlistInDistributedTransactions=false
}
store.Initialize(); // fails with the following exception
with this message
Message = "Value cannot be null.\r\nParameter name: g"
Scenario:
a) The first time (if the database does not yet exist) this is called,
the call succeeds and I note that the raw http response is:
HTTP/1.1 201 Created
Transfer-Encoding: chunked
Content-Type: application/json; charset=utf-8
Content-Encoding: gzip
Location: /docs/Raven/Databases/TNW_Grc_Client
Server: Microsoft-HTTPAPI/2.0
Raven-Server-Build: 427
Date: Fri, 26 Aug 2011 19:45:23 GMT
X-Charles-Received-Continue: HTTP/1.1 100 Continue
Expires: 0
Cache-Control: no-cache
{"Key":"Raven/Databases/TNW_Grc_Client","ETag":"00000000-0000-0300-0000-000000000001"}
b) The second time the call fails. Debugging some, it appears the etag header is not sent from
the server but is somehow expected by the client code:
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/json; charset=utf-8
Content-Encoding: gzip
Server: Microsoft-HTTPAPI/2.0
Raven-Server-Build: 427
Date: Fri, 26 Aug 2011 19:48:10 GMT
Expires: 0
Cache-Control: no-cache
{"Id":null,"Settings":{"Raven/DataDir":"~\\Tenants\\TNW_Grc_Client"}}
You can specify values on the connection string or on the document store properties, not both at the same time.