I am not able to get all the cookies passes in request headers in scrapy ?
How can i get all the cookies of request header?
Set COOKIES_ENABLED = True, COOKIES_DEBUG = True in your settings file and try response.headers.getlist('set-cookie')
Related
I try to emulate the access to the webservice of my company using a dart file.
Here is my request where I try to send the jsessionid cookie to the server :
taches = await http.post(
Uri.parse('https://test1.beotic.net/beop3server62/p3servicejson'),
headers: ({
"content-type": "application/json-rpc",
"set-cookie": setcookie
}),
body: body
);
The recuperate the setcookie variable is from the response header of a previous request that I sent to the server with an authentication token, what's inside setcookie look like this : JSESSIONID=01D68460B589F3B34A9C3208FA6CEA51; Path=/beop3server62; Secure
But it seems that the setcookie variable is never sent to the server as the response to this last request always contains a different jsessionid cookie in its headers and the body is :
{"jsonrpc":"2.0","id":"10","error":{"code":0,"message":"The user is
not
authentified","data":{"exceptionTypeName":"com.beotic.apps.p3.exception.AuthentificationException","message":"The
user is not authentified"}}}
I'm quite lost with all this as I tried many different things and none of them seem to work.
Thanks for helping !
Set-Cookie is a response header, not a request header.
Clients should read the Set-Cookie header, use the information in it to store data in their internal cookie jars, then generate Cookie request headers from the cookie jars when making subsequent requests to the same host.
there's next site - vten.ru
When I try to make GET request with Postman to it, I give in return status code 304 Not Modified.
Code on Phyton:
import requests
url = "http://vten.ru"
payload = ""
headers = {
'cache-control': "no-cache",
'Postman-Token': "29ae741a-1c31-4a52-b10e-4486cb0d6eb7"
}
response = requests.request("GET", url, data=payload, headers=headers)
print(response.text)
how can I get the page?
You presumably already have a version of the request cached, hence the "Not Modified" response indicating that the response hasn't changed since you last requested it.
EDIT:
Viewing that site/inspecting the network activity via Chrome shows that the document returned is actually http://m.vten.ru. You should try making your GET request to that URL instead.
You also need to add the Accept: text/html header to your request. That returns the page you want having just tested it locally.
I have a webpage I made with angular 2 that is a form and at the end of it the form must be sent to my java server. But I'm unable to send, I get the error Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:4200' is therefore not allowed access.
I have already changed my server so it will send the Access-Control-Allow headers. The function is like this:
public static Response buildResponse(int status, Object reponseObject, MediaType mediaType) {
Response.ResponseBuilder rb = Response.status(status).entity(reponseObject);
if (mediaType != null) {
rb = rb.type(mediaType);
}
rb = rb.header("Access-Control-Allow-Origin", "*");
rb = rb.header("Access-Control-Allow-Methods", "GET, POST, PATCH, PUT, DELETE, OPTIONS");
rb = rb.header("Access-Control-Allow-Headers", "Origin, Content-Type, X-Auth-Token, accept");
rb = rb.header("Access-Control-Allow-Credentials", true);
System.out.println("teste");
return rb.build();
}
I have also tried to add the header 'Access-Control-Request-Method': 'POST' to the page request, but I still get the error. The problem is not with the software functionality itself because it works fine if I install and activate the CORS plugin for chrome, but I can't request the user to install the plugin to use my site. Does anyone knows what I'm missing? I have been sarching for sometime and the only solutions I found was to disable this browser security (I can't ask the user to do that) and to add the headers on the request, what I already have.
I was finally able to solve my problem. The problem was in the communication between my page and my server. The CORS block is imposed by the browser and the server needs to send a header authorizing the access (the headers I put in the function in the question). What I was missing was the header in the OPTION method. My JavaScript code automatically send and OPTIONS request (the preflight request) before the actual GET or POST method. What I did was to manually implement the OPTIONS method in the server that respond with the necessary headers.
I am using Angular $HTTP to make a CORS request to a remote API service (SmartyStreets.com). I have set the defaults as is well-documented.
.config(['$httpProvider', function($httpProvider) {
$httpProvider.defaults.useXDomain = true;
delete $httpProvider.defaults.headers.common['X-Requested-With'];
}])
When I make a plain request with no custom headers everything works as expected.
// This works as expected
$http({method: 'get', url: $scope.addr.url, headers: {}})
However, I need to now set some additional custom headers. But setting the custom headers breaks the CORS request.
// this results in the browser 404 error:
// No 'Access-Control-Allow-Origin' header is present on the requested resource
$http({method: 'get', url: $scope.addr.url,
headers: {'x-standardize-only': 'true', 'x- include-invalid': 'true'}})
I've been trying to figure this out for a couple days now...stuck. Anyone know how to solve this problem?
Thank you!!
Your server needs to correctly respond to the OPTIONS request that the browser will make on your behalf to determine if the CORS request is valid. It needs to contain an Access-Control-Allow-Headers header with the right info in it. See: https://developer.mozilla.org/en-US/docs/HTTP/Access_control_CORS#Preflighted_requests
I am trying to login to a site, and then view user details.
The API documentation from the site is:
LogOn : All Calls with JSON and return JSON type
post - https://www.bit2c.co.il/Account/LogOn {UserName:'',Password:'',SecondFactor:[optional]}
return true if OK , error list of strings if not
Balance - GET https://www.bit2c.co.il/Account/Balance
return UserBalance as JSON
I've tried connecting to the site using
import requests
session=requests.session()
session.auth = ("username", "pass")
session.post("https//www.bit2c.co.il/Account/Balance")
but i am getting response 200 and the response content is "you must login".
What am I doing wrong ?
What kind of session? What is on server-side? Can you post the raw request/response pair(s)?
If PHP or ASP runs on server-side, you must capture the Set-Cookie header of the response to the login request, find the session cookie and in consequent requests you must set the Cookie header with the session cookie name and value captured previously.