I'm writing some http client code to interact with a website, and I need to set some cookies. Simply visiting the website sets 4 cookies (as seen in Chrome Settings).
However, when I look at the HTTP response headers for when those cookies were set (using Live HTTP Headers extension), there is no Set-Cookie header anywhere. How were those cookies set? Is there another way than through Set-Cookie?
Edit: Some of the cookies are HttpOnly.
If you load a site in your browser, it might also load other assets that can also set cookies (given that they are on the same domain).
But there is a second way to set cookies: with Javascript via document.cookies.
As far as I know, if your javascript or python code sets a cookie for that domain, then the response will include the SET-COOKIE field. You can view that from at least the inspect console.
So I see that you're using HTTP live extension, but it doesn't look like it shows that field in the response.
I tried looking for other extensions that could show it, but I wasn't able to find one as far as I know. I suppose we both can always fall back to the chrome inspect console. If you go to the network tab, you should actually see the req-resp.
Related
I have a web server which returns 200 OK with a bunch of set-cookies, and an HTML page which loads a bunch of scripts from the same server.
However, the subsequent loads that was spawned from that HTML page submits a different cookie on their HTTP request headers.
What could be causing that? Surely there's some policy I'm missing out on, but I don't see why it works on some pages and not others?
I'm using chrome as the browser, but this behavior also happened from iOS, so I'm guessing it's not browser specific.
So after a lot more reading and troubleshooting, it turns out that when you don't set a cookie path, it'll default to whatever path the original request set-cookie was sent to. And because my resource paths had a different path, the cookie was not sent.
Adding Path=/ fixed it for my issue. Of course, if you don't want your cookie to be accessible to all pages this is bad, but my web-server requires requests to come with cookies because they are sensitive data (for security reasons).
I am trying to work out how cookies are being set on a website, I have scoured the page source and can see how most of them are being generated.
However, there is one cookie that appears on page load that I can't track down.
Is it possible that a cookie is being set when an image is being requested from a remote server? If so, can I inspect that http request response with a tool to find out if it contains the cookie?
Any HTTP-Request can set a cookie, if the server says so.
Cookies are set using the Set-Cookie HTTP header, sent in an HTTP response from the web server.
https://en.wikipedia.org/wiki/HTTP_cookie#Setting_a_cookie
A request for an image is basically the same as a request for a html page. It uses the same request/response structure. So yes you can set a cookie on an image request.
The request/response can be seen in most modern browsers. In FireFox there is under tools -> Web Developer -> Network a tool that shows the requests/responses from all calls being made on a page. Opera and Chrome have similar functionality.
Is there a way to tell the browser not to share a cached resource among websites?
I want to give websites a link to some JavaScript on my server and I want to make the response be different for each domain using the Referer header as check.
The response which will be cached should be available to the domain that requested it and when the end users visit another site that uses the script link, another request should be made.
I don't know whether I understand your question.
Does your scenario like: stackoverflow.com and yourwebsite.com use the same script called "https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js", but you don't want to share the cached script with stackoverflow.com
This is under the control of googleapis.com's web server.
So if the cached resource's origin server(googleapis.com) want to implement the feature as you said, it may use the Vary response header. Vary Header define the secondary key of cache.
Maybe "Vary: Origin" but only work for CORS
Maybe "Vary: referer" but referer contains url path
It still doesn't solve your problem but I hope it helps.
see MDN HTTP Cache Doc and [RFC 7234 Section 4.1]
As of current, are there still any methods to spoof HTTP referer?
Yes.
The HTTP_REFERER is data passed by the client. Any data passed by the client can be spoofed/forged. This includes HTTP_USER_AGENT.
If you wrote the web browser, you're setting and sending the HTTP Referrer and User-Agent headers on the GET, POST, etc.
You can also use middleware such as a web proxy to alter these. Fiddler lets you control these values.
If you want to redirect a visitor to another website and set their browser's referrer to any value you desire, you'll need to develop a web browser-plugin or some other type of application that runs on their computer. Otherwise, you cannot set the referrer on the visitor's browser. It will show the page from your site that linked to it.
What might be a valid solution in your case would be for you to load the third party page on the visitor's behalf, using whatever referrer is necessary, then display the page to the user from your server.
Yes, the HTTP referer header can be spoofed.
A common way to play with HTTP headers is to use a tool like cURL:
Sending headers using cURL:
How to send a header using a HTTP request through a curl call?
or
The cURL docs:
http://curl.haxx.se/docs/
Yes of course. Browser can avoid to send it, and it can be also "spoofed". There's an addon for firefox (I haven't tried it myself) and likely you can use also something like privoxy (but it is harder to make it dynamically changing). Using other tools like wget, is as easy as setting the proper option.
I remember having seen somwhere that it is possible to tell where a user came from to a website. More specifically, I want to determine the (Google etc.) search that led to my site. How can I do that?
As far as I know the HTTP protocol, nothing like this is sent to the server in a GET request, so I also wonder how it is done technically.
"Referer" field in the HTTP header
You need to check the Referer header. [sic]
When a user clicks a link in a webpage, the browser sets the referer header of the request for the link's target to the page that contained the link.
If a user came to your site from a Google search, this header will be a url in Google.com.
However, for privacy reasons, some browsers do not send this header.
Referer was misspelled in the original HTTP implementation, and the mispelling stuck. (It should be spelled Referrer)
What you're looking for is the Referrer. Look up platform specific info to find out how to use it.
sign up for google analytics for free to get those statistics and a whole lot more. Seeing where someone came from is done by checking the http referer header.
The client may set the Referer field as part of the HTTP header to indicate the referring page. However, as with everything else that the client controls this cannot be viewed as anything but an indication. It is not always set and the client may fake the data, so keep that in mind when using client data.
The Referer HTTP Header contains the URL of the site containing the link clicked on. But it is sent by browsers only and in can be suppressed by security settings!