I'm implementing analytics event and page view tracking on a Roku box (brightscript). That part isn't so important, but know that JS is not an option, nor is cookie setting. I can set variables, but they're not typical cookies. Ok - so I've implemented a great solution (google-analytics-for-roku-developers/) and all is well - EXCEPT!!
Analytics doesn't seem to track uniques, and is listing ZERO. It is tracking events beautifully, however :)
Here's a sample of my URI request (decoded for clarity):
http://www.google-analytics.com/__utm.gif?utmwv=1&utmn=1736644096&utmsr=720p HDTV&utmsc=24-bit&utmul=en-us&utmje=0&utmfl=-&utmdt=RokuPageView&utmhn=Home&utmr=-&utmp=Home&utmac=UA-5035974-13&utmcc=__utma=1394882688.2097034752.1347384621.1347384621.1347384621.2;+__utmb=1394882688;+__utmc=1394882688;+__utmz=1394882688.1347384621.2.2.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none);+__utmv=1394882688.13C1CJ109560;
so, question #1 - I understand from the docs that the __utma is the element that tracks uniques. Do you see anything there that's wrong?
question #2 - The development code I implemented sends a NEW random cookie for EACH EVENT. That seems wrong to me. I'm considering changing it to a session-based cookie that persists through all events. That's when I'm way beyond my understanding. Any guidance on this?
I think the answer to #2 is the answer to #1. Basically, you need to emulate how a browser handles cookies within your code. A "session cookie" is just a cookie, that is, a value passed as an HTTP header along with the request and response -- what makes it a session cookie is that it's expiration date is set to the past, which tells the browser to delete it when the browser instance closes.
Cookies are pretty straightforward -- a mechanism to get around the fact that HTTP is stateless (has no memory). If you pass an HTTP header Set-cookie: <value> in the response, the client is supposed to remember the value and pass it back in a Cookie: <value> header in subsequent requests. (There's a bit more to cookies relating to domains and expiration, and so on, but it's not that much).
So if your client (the Roku) has some mechanism for persistance, then you just need to store any new cookie, then before setting a new random one, check the header and if the cookie has been stored, just send it back as is. You'll probably need to implement some sort of task that cleans up expired cookies periodically, and so on.
Do not fear the cookie. It's just a header (with browser-imparted magical powers).
Related
Everyone claims a domain must have two dots in order for a cookie to be stored, but that cannot be true, because Chrome is storing the cooking named "__RequestVerificationToken" received through the Set-Cookie header from ASP.NET; however, it refuses to store other cookies. I literally am sending a virtually identical cookie with a virtually identical Set-Cookie header, and yet Chrome is refusing to store it. The path is "/", just like the other one. No domain is set. The only difference is the name. Is Chrome giving "__RequestVerificationToken" some kind of special treatment?
Looks like Chrome will save "session" cookies for "localhost", but not "permanent" cookies. No idea why it makes the exception for "session" cookies, but the "permanent" ones aren't stored because cookies are only supposed to be stored for 2nd level domains (domains with at least 2 dots, which localhost is not).
"permanent" cookies are cookies that DO set an expiration date, which is a bit counter-intuitive, but they're "permanent" in the sense that they survive across browser restarts unlike "session" cookies which are supposed to be deleted when the browser closes. In reality, they're all relatively permanent, because Chrome/Firefox typically don't delete the session cookies when the browser closes, thanks to features like "continue where I left off" and "restore my tabs when the browser restarts".
Another takeaway is that we may as well not set an expiration on any kind of session or auth cookie at all, because it offers the following advantages: 1. it works on localhost, 2. it more secure in that they stand a chance of being deleted when the browser closes, and 3. they persist across browser sessions anyway if the user has the browser configured to do so, so setting a long timeout isn't necessary. Setting an expiration on the cookie also doesn't do you any good anyway, because it's just another value to sync with the server-side session, the value isn't sent to the server with the cookie anyway, so it's useless, and utimately the server-side expiration for the token (stored in session or database) is authoritative anyway. So just don't set an expiration on your session or auth-related cookies.
I am working on a new API. With ETags enabled, Chrome always conditionally GETs a resource with a long cache lifetime, that is, Chrome no longer uses the cached version without first confirming the ETag.
Is this how ETags are supposed to work with GET?
If so, and considering that I don't need them for update race-checking, I will remove the ETags and alleviate the extra work my server is doing.
Edit
I would have expected Chrome to use the cached item according to the max-age until it expires and then use the conditional GET to update its cached-item.
As I write this, it has now occurred to me that it would not be able to 'update its cached-item' since the 304 contains no max-age to use to extend the expiry time.
So I guess with ETags enabled, a client will always conditionally GET unless there's a good reason to use the cached version, such as no network.
Thus, it appears that ETags harm server performance when no concurrency control is needed.
Edit
I think I've already guessed the answer (above) but here's the question in a nutshell:
When ETags are enabled and the Max-Age is far in the future, should User Agents always use conditional GET instead of using the previously cached, fresh response?
How you load your page in the browser might be relevant here:
If you press Ctrl and reload the page using the refresh button, that will cause an unconditional reload of your resources with 200s returned.
If you just reload using the refresh button (or equivalent key like F5), conditional requests will be sent and 304s will be returned for static resources.
If you press enter in the address box, add the page to your bookmarks and load from there, or visit the page from a hyperlink, the cache max-age should work as expected.
With ETags enabled, Chrome always conditionally GETs a resource with a long cache lifetime, [...] Is this how ETags are supposed to work with GET?
The answer is in the RFC:
10.3.5 304 Not Modified
If the client has performed a conditional GET request and access is
allowed, but the document has not been modified, the server SHOULD
respond with this status code.
So, yes.
If this is not the answer you expect, you might want to include some actual traffic in your question that shows the order of request-response pairs that you expect and that you actually get.
considering that I don't need them for update race-checking
If you're not going to use conditional requests (If-Match and the likes), you may indeed omit the ETag calculation and processing.
Some small percentage of the time, we see a flow like this, deducing from looking at server logs (I have not been able to reproduce this case with any browser):
At time A, client hits our page:
with no cookies
gets back a response with a Set-Cookie HTTP response header that gives them a session id of B
body has JS to fire an AJAX request /ajax/foo.
At time A + 1 second, client hits us with the AJAX request to /ajax/foo
the referrer is set to the page in step 1 that fired the AJAX, as expected
with no cookies - why?
gets back a response with a Set-Cookie header that gives them a session id of C (expected, since they didn't send us a cookie)
At some time slightly later, all of the client requests are sending either session id B or C - so the problem is not that the browser has cookies turned off.
This seems to be essentially a race condition -- the main page request and the AJAX request come in together very close in time, both with no cookies, and there is a race to set the cookie. One wins and one loses.
What is puzzling to me is how could this happen? My assumption is that by time the browser has read enough of the response to know that it needs to fire an AJAX request, it has already received the HTTP response headers and thus the Set-Cookie response header. So it seems to me that the client would always send back the cookie that we set in the page that fired the AJAX request. I just don't see how this could happen unless the browser is not promptly processing the Set-Cookie response.
Like I said, I can't reproduce this in Firefox, Safari, or Chrome, but we do see it several times a day.
There is a new feature in google chrome that could cause this misbehavior. It is called prerender.
Prerendering is an experimental feature in Chrome (versions 13 and up)
that can take hints from a site’s author to speed up the browsing
experience of users. A site author includes an element in HTML that
instructs Chrome to fetch and render an additional page in advance of
the user actually clicking on it.
Even if you do not proactively trigger prerendering yourself, it is
still possible that another site will instruct Chrome to prerender
your site. If your page is being prerendered, it may or may not ever
be shown to the user (depending on if the user clicks the link). In
the vast majority of cases, you shouldn’t have to do anything special
to handle your page being prerendered—it should just work.
For more information read: http://code.google.com/chrome/whitepapers/prerender.html
Edit:
You could trigger prerender on your page with: http://prerender-test.appspot.com/
a) Does the cookie have an expiration time?
b) If so, have you tried to reproduce it by setting the computer's clock back or forward by more than the TTL of the cookie? (I mean the clock of the computer running the browser, obviously; not the server running the app ... which should be a separate computer whose clock is set accurately.)
I've seen this as well; it seems to be triggered by users with screwed up system clocks. The cookie was sent with an expiration date that, from the browser's perspective, was already past.
I know there are ways to tell if an user has pressed the back button using Javascript, but is there a way to tell without resorting to Javascript? Is there a solution that involves just looking at referer URLs perhaps?
Without javascript no.
The problem is the back button will not guarantee you get a server hit. It can cache the page client side and even if it did hit the server (loading the page), then it would have the request from the initial hit not like it came from the page you were just on. The back button doesn't add 'referral' information to the request. It just goes back to the last thing you did without sending the details of where you just were.
You need to handle this client side.
Yes, it is possible. There are two parts to this -
Every URL should have a unique token in it. On the server side, you keep track of the current and past tokens. When a request comes along, if the token matches a past token, the back button was hit. If it equals the current token, process the request normally. Otherwise fail the request.
Now, the page could have been cached and your server may not see the request. So you have to take steps to defeat the browser cache. Setting the following http headers should force the browser to make a request -
Cache-Control : no-Cache
Pragma : No-Cache
Expires : Thu, 01 Jan 1970 00:00:00 GMT
Because it is possible doesn't mean you should use it though. Backbutton is an essential technique for the web, and breaking it is poor usability.
In pageload, if you do Response.Cookies.Add(..., immediately in the next line, you can access that cookie via Request.Cookies(...
I know that under the covers, the cookie is added to Request.Cookies by .net, but the original request never had that cookie.
If what I'm saying is correct, why is it this way? Shouldn't the cookie be available in the following request? Shouldn't the immediate access to Request.Cookies(... be null?
You're right, ASP.NET does do that behind the scenes:
After you add a cookie by using the
HttpResponse.Cookies collection, the
cookie is immediately available in the
HttpRequest.Cookies collection, even
if the response has not been sent to
the client.
-- HttpRequest.Cookies page on MSDN
Why is it like that? At a guess, it's because that's how developers expect it to work.