I'm using a luminati (newly named brightdata) proxy on one of my scrapy spider and I can't find anything to prevent cookies/headers from being discarded when sending a request.
I need to pass cookies and custom headers but when I'm checking what's being sent (via requestbin) I see that my custom headers and cookies have disappeared.
Any idea on how I could prevent this behavior ?
Related
i have a fairly simple testplan with following steps recorded
Thread Group
Recording Controller
HTTP Authorization manager
GET https://xxxx/user/login
GET https://xxxx/accounts/list
GET https://xxxx/user/settings
GET https://xxxx/partners/list
GET https://xxxx/user/logout
When i run this, i see jmeter adding the authorization header in the first user/login request and server responds with 200 OK. This is OK
i extract the session token from the login response and put it as a variable into the subsequent http requests.
jmeter is not adding the authorization header in the subsequent accounts/list request - this is OK
but then jmeter is adding the authorization header into the subsequent user/settings request - this is not needed as it defeats the purpose of sending the session token.
Again jmeter is not sending the auth token in partners/list but is sending it again in user/logout request.
So question is how jmeter determines when to add the auth header and when not?
The desired behaviour i want is for jmeter to send the auth header in the first user/login request but not in any of the subsequent http requests.
Any pointers appreciated.
thanks
HTTP Authorization Manager will add Authorization header to all the HTTP Request samplers it its scope, if you place it at the same level with the HTTP Request samplers - it will add the header to all of them.
If you need to apply the HTTP Authorization Manager to 1st sampler only - move it to be the child of the first sampler.
More information: JMeter Scoping Rules - The Ultimate Guide
Recording any scenario adds up the Header Manager in all the Http requests.
You are doing it in correct way, however still you have to make few changes in your scripts to run seamlessly.
Delete all the subsequent Header Manager under the accounts/list,user/settings,partners/list & logout.
Post the Login Request > extract the session > Add a Header Manager after Login request and pass the session there.
All the further request should be in the same thread so that the session get passed to each request.
Also while recording there are multiple requests that gets captured and for each scenario Jmeter maintains the session under Header Manager, hence this should never be the case that one request is getting the session while others not.
Thanks for the answer that helps.
What i did and worked well in the mean time is add an explicit authorization header to the user/login request with a beanshell preprocessor to base64 encode the username / password.
thanks anyways
I'm using Restangular. I would like to get full response so I set
Restangular.setFullResponse(true);
but then I discovered that my custom headers does not work. Documentation for setFullResponse() method says:
in order for Restangular to access custom HTTP headers, your server must respond having the Access-Control-Expose-Headers: set.
I would like to send my custom headers but I don't want to change server settings. Is it possible?
If I leave default settings i.e.setFullResponse(false) there is no issue with custom headers. Is there another solution except changing server settings?
The sentence you cite from the setFullResponse() docs is about what headers from the response your Restangular app will be able to access. And what that is saying is, it’s not possible to access most of the headers from the response unless the server’s already configured to send the right response-header names in the Access-Control-Expose-Headers header.
Without the server setting any value for that header, the only response headers that browsers will let you access from client-side JavaScript in your web app are the Cache-Control,
Content-Language,
Content-Type,
Expires,
Last-Modified
&
Pragma response headers.
See https://fetch.spec.whatwg.org/#cors-safelisted-response-header-name for the spec on that.
I would like to send my custom headers but I don't want to change server settings. Is it possible?
If you mean you want to send custom headers in a request from your client-side Restangular code to the server, please provide more details about exactly which custom headers you want to send.
There too though, there’s another CORS header your server must send in the response: the Access-Control-Allow-Headers response header. If the server isn’t configured to send that with the right header names listed, then your request will fail. The reason in that case is, you will hit https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS#Preflighted_requests.
I've been stuck on this for a few hours with no luck, so I figured that I would ask here.
I have a service with a bunch of endpoints, most of which accept GET and POST http methods. In that case, my service simply specifies Access-Control-Allow-Origin to be * in the response headers, in the event that one of my apps is on a different domain/port and wants to use the service.
I have one endpoint that uses the DELETE http method, and I can't seem to get it to work. When I call this endpoint from my client app, I get this message in my console:
Response to preflight request doesn't pass access control check:
No 'Access-Control-Allow-Origin' header is present on the requested resource.
I've been reading up on CORS for the past couple hours and have tried a few different things, all of which have not worked. I (think I) at least understand that when I'm using http methods other than GET and POST, the browser sends a pre-flighted request with OPTIONS as the http method.
What is the best way to handle this? Is there a way to disable this pre-flighted request? I specify in my client app that the http method to call this endpoint is DELETE. Should I be putting something specifying headers in my AJAX function that calls this endpoint (I'm using straight JavaScript)?
If I can't figure out a way around this, I'm just going to change my endpoint to use a GET or POST method, but I wanted to find a way to handle this issue before I took the easy way out.
I (think I) at least understand that when I'm using http methods other than GET and POST, the browser sends a pre-flighted request with OPTIONS as the http method.
Yes
Is there a way to disable this pre-flighted request?
No there isn’t. It’s initiated by browsers automatically, with no way to disable it from your JS. As long as you’re sending that DELETE request cross-origin in your JS, browsers will do that preflight.
Should I be putting something specifying headers in my AJAX function that calls this endpoint?
Given the “No 'Access-Control-Allow-Origin' header is present on the requested resource” message you’re getting, no changes you make in your client code will make any difference.
The place where more headers need to be sent to deal with this is instead on the server side.
If I can't figure out a way around this, I'm just going to change my endpoint to use a GET or POST method
You might want to experiment with trying that first. It seems even if you make that change you’re still gonna get that “No 'Access-Control-Allow-Origin' header is present on the requested resource”.
The SO question "No 'Access-Control-Allow-Origin' header is present on the requested resource" is a good place to read up on this to get a better idea of what’s happening.
I have a route that uses an authentication cookie set by another route. I've created it like this:
This method no longer works in the new version. Paw complains that there is no set-cookie header in the response from the Authenticate request.
It seems this is because Paw now takes the cookies and handles them differently from other headers. I like this approach because it should make this sort of authentication easier, but, unfortunately, it's not working like I would expect.
Here's how I have configured a newer request:
So, I've set the cookie header to the Response Cookies dynamic value which, I believe, should pass along the cookies set previously. I would think I should select the Authenticate request from the dropdown (since it's the response from this request that actually sets the cookie, but the cookie value disappears if I do that. Instead, I have left the request value as Current Request since that seems to contain the correct value.
I've also noticed the Automatically send cookies setting which I thought might be an easy solution. I removed the manual cookie header from my request leaving this checked in hopes it might automatically send over any cookies from the cookie jar along with the request, but that doesn't seem to work either. No matter what I try, my request fails to produce the desired results because of authentication.
Can you help me understand how to configure these requests so that I can continue using Paw to test session-authenticated routes?
Here are a couple of things that will make you understand how cookies work in Paw (starting from version 2.1):
1. Cookies are stored in Jars
To allow users to keep multiple synchronous sessions, cookies are stored in jars, so you can switch between sessions (jars) easily.
Cookies stored in jars will be sent only if they match the request (hostname, path, is secure, etc.).
2. Cookies from jars are sent by default, unless Cookie header is overridden
If you set a Cookie header manually, cookies stored in jars wont' be sent.
And obviously unless Automatically Send Cookies is disabled.
3. The previous use of "Response Headers" was hacky. Use Response Cookies.
In fact, Set-Cookie (for responses) and Cookie (for requests) have different syntaxes. So you can't send back the original value of Set-Cookie (even though it seemed to be working in most cases).
The new Response Cookies dynamic value you mentioned has this purpose: send back the cookies set by a specific request.
Now, in your case, I would use a Response Cookies dynamic value all the time. As you have only 1 request doing auth / cookie setting, it might be the easiest to handle. Also, maybe check Ignore Domain, Path, Is Secure and Date to make sure your cookie is always sent even if you switch the host (or something else).
Deleting the cookies in the cookie jar and hitting the authentication endpoint again seems to have fixed the problem. Not sure why, but it seems to be working now either manually sending a cookie or using the Automatically send cookies setting.
I have a web server running on an EC2 node. There is an endpoint which accepts HTTP DELETE requests. It accepts the requests and proceeds with the intended functionality only if some certain headers are found in the DELETE request it send. What would be the easiest way to test this?
Check http://www.hurl.it/
Set the type the DELETE, and click Add Headers to add the headers you want.