can anyone please tell me why i am not able to get the range of contents using content-range header,instead i am getting the whole content in given url.
import requests
url="https://tools.ietf.org/rfc/rfc2822.txt "
content = requests.get(url)
content_len = int(content.headers['Content-Length'])
print(content_len)
headers = {f"Content-Range": f"bytes=0-100/{content_len}"}
r = requests.get(url, headers=headers)
print(r.content)
Usually you would use Range and not Content-Range for requesting a subset of a resource. The server will then respond with a HTTP 206 (Partial Content) status and serve you with the requested range. The response will then contain a Content-Range as a header.
The following works for example:
import requests
url = "https://tools.ietf.org/html/rfc7233"
headers = {"Range": "bytes=0-500"}
r = requests.get(url, headers=headers)
Related
Good afternoon!
For many months I have not been able to generate a POST request to send the firmware.
My method:
Used fiddler to keep track of titles.
With disabled authorization, urllib3 sends the necessary files, but the fiddler could not see it
import urllib3
http = urllib3.PoolManager()
ip="10.254.255.105"
name_file="boot_v1.5_app_v2.27.8.fw"
uri = "/firmware/update_from_page.htm"
with open(name_file, 'rb') as fp:
file_data = fp.read()
send = http.request(
'POST',
'http://' + ip + uri,
fields={'new_firmware': (name_file, file_data),}
)
POSTMAN also worked even with enabled (digest) authorization. I found the correct header line ("Content-Type": "multipart/form-data; border=--------------------7e6a5e075c") needed to send a file with form function -data
Please help to form a request to send a file through the requests library. it is used in the rest of the code and supports the desired digital authorization.
Tried to use the requests library in different ways. I decided to reproduce the sending with disabled authorization based on the working code urllib3, but same errors.
"fields=" missing in the requests library replaced it with others and changed arguments with headers
send = requests.post(url='http://'+ip+uri,
# data={'new_firmware': file_data},
files={'new_firmware': file_data}
# verify=False,
)
date= {'new_firmware': file_data}or {'new_firmware': fp} and analogues - leads to freezing
files= {'new_firmware': (name_file, file_data),} and analogues = requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
I noticed the urllib3 header in Wireshark
POST /firmware/update_from_page.htm HTTP/1.1
Host: 10.254.255.105
Accept-Encoding: identity
Content-Length: 524476
Content-Type: multipart/form-data; boundary=4cf5492caddccc65a2b42d8bc691c30f
User-Agent: python-urllib3/1.26.14
Maybe it sends a file in jason format, but so far it has not been possible to figure it out, and so far I have not found working options
I want to use the python request to access the below url, while I get the 401. while I can open the below url in the website. I don't know how to use the python request to access it.
https://data.10jqka.com.cn/funds/gnzjl/field/tradezdf/order/desc/page/2/ajax/1/free/1/
my code:
import requests
response = requests.get(url, headers=self.headers)
What I should do and could anyone give a code example to access that url?
I can successfully hit the endpoints for the UMLS authentication via Postman, but keep getting 415 errors when moving this code to R and using httr. This only seems to work when using x-www-form-urlencoded (as opposed to json).
My relevant Postman headers are:
Content-Type: application/x-www-form-urlencoded and
Accept: /*/
And trying to recreate this in R:
library(httr)
auth_endpoint <- "https://utslogin.nlm.nih.gov/cas/v1/api-key"
auth_headers <- c("Content-Type" = "application/x-www-form-urlencoded",
"Accept" = "*/*")
getTGT <- function(endpoint, headers) {
request_body <- list(apikey = "API_KEY_HERE")
request <- POST(url = endpoint,
headers = add_headers(.headers = headers),
body = request_body
}
This request is returning the 415 error which I can only tell is related to the Content-Type. I am more used to using JSON but that doesn't work in Postman either. Am I creating the request body correctly for a x-www-form-urlencoded type?
Finally figured this out - for x-www-form-urlencoded content, there needs to be encode = 'form' included in the POST call.
When I try to send a request via python3.6 to some urls, it waits until Timeout exception is raised( ConnectionError: HTTPSConnectionPool(host={host}, port=443): Read timed out) . But when I try the same request via python2.7 it is successfully completed with status code: 200. Can you help me?
Version of Requests Package: 2.23.0
Sample Code:
import requests
url = "https://www.khaneyeshoma.ir/"
requests.get(url=url, timeout=10)
Thanks!
sometimes problem occurs cuz of using timeout parameter try :
requests.get(url=url,)
It think is because of the website you are trying to access. The request is correct, but it may need some extra headers.
If you try the request on other address it will work:
import requests
url = "https://www.google.com"
requests.get(url=url, timeout=10)
Response:
<Response [200]>
You can use urllib.request with postman header, and you won't need timeout anymore:
import urllib.request
url = "https://www.khaneyeshoma.ir/"
req = urllib.request.Request(
url,
data=None,
headers={
'User-Agent':"PostmanRuntime/7.6.0"
}
)
response = urllib.request.urlopen(req)
html = response.read()
print(html)
It is because of a whitespace between the header field-name(access-control-expose-headers) and colon. RFC 7230:
No whitespace is allowed between the header field-name and colon. In the past,
differences in the handling of such whitespace have led to security vulnerabilities
in request routing and response handling. A server MUST reject any received request
message that contains whitespace between a header field-name and colon with a
response code of 400 (Bad Request). A proxy MUST remove any such whitespace from a
response message before forwarding the message downstream.
there's next site - vten.ru
When I try to make GET request with Postman to it, I give in return status code 304 Not Modified.
Code on Phyton:
import requests
url = "http://vten.ru"
payload = ""
headers = {
'cache-control': "no-cache",
'Postman-Token': "29ae741a-1c31-4a52-b10e-4486cb0d6eb7"
}
response = requests.request("GET", url, data=payload, headers=headers)
print(response.text)
how can I get the page?
You presumably already have a version of the request cached, hence the "Not Modified" response indicating that the response hasn't changed since you last requested it.
EDIT:
Viewing that site/inspecting the network activity via Chrome shows that the document returned is actually http://m.vten.ru. You should try making your GET request to that URL instead.
You also need to add the Accept: text/html header to your request. That returns the page you want having just tested it locally.