I spoofed referrer with Fiddler Web Debugger 4.5 using this code:
static function OnBeforeRequest(oSession: Session) {
oSession.oRequest["Referer"] = "http://ehsan.com";
And when I request, I can see in the miscellaneous part that referrer is my website. but the google analytic says No referrer. what's wrong?
Google Analytics is likely using JavaScript (document.referrer) instead of HTTP to determine what the referrer is. You can try to inject a snippet of JavaScript that sets document.referrer to fool it.
Related
Let's say i created a google sheet to capture user's email addresses.
On my website there is a small form and once the submit button is clicked and an ajax request to a google web app that writes data to a sheet is fired:
// Let's select and cache all the fields
var $inputs = $form.find("input, select, button, textarea");
// Serialize the data in the form
var serializedData = $form.serialize();
// Fire off the request
request = $.ajax({
url: https://script.google.com/macros/s/longURLcode/exec,
type: "post",
data: serializedData
});
In the google script you now use doPost(e) or doGet(e) to handle any incoming http request. To allow this to work as a sign up mechanism permissions for the web app have to be set to (i think !?)
Execute the app as: Me (myemail#gmail.com)
Who has access to the app: Anyone, even anonymous
Given everything on the google script site is set up properly, this works like a charm. So whats wrong?
Problem:
Anyone can either look into the source code of my webpage or use the dev tools to extract the url to the google web app after clicking submit. This url can now in theory be used to flood the sheet with countless (undefined) entries.
Questions:
1) Is there a way to limit accepted http request to certain origins? I tried to do this by accessing the http headers within doPost() but there seems to be no way to do so.
2) Is "Who has access to the app: Anyone, even anonymous" the wrong approach? I thought this is necessary since you can only choose google users here and some url (mywebsite.com) seemed to fall within the anonymous category.
3) I don't think this is possible but maybe i missed an option: Is there a way to NOT expose the google web app url to anyone? I guess not because you can monitor any requests with dev tools.
4) Is using sheets for capturing that kind of data just a terrible idea in general (partly for above reasons) and i should find another solution asap?
I'm currently building an ASP.NET web application to simplify the provisioning of Google Sites, pages, Gadgets on Google Sites and ACLs for Google Sites.
I have encountered the issue which many a developer has already come across: cross-origin resources. According to the Google documentation on CORS requests to Google APIs, you simply use an XMLHttpRequest (or AJAX) request, providing your access token in the header. More information can be found here:
https://developers.google.com/api-client-library/javascript/features/cors
I've been perfectly able to accomplish this when I'm accessing the Google Sites API from within my domain on Google Sites, injecting AJAX requests while my browser window's location is within the domain. An example of a succeeded request to make a new site from within my domain:
$.ajax(
{
////// REQUEST \\\\\\
type: "POST",
url: "https://sites.google.com/feeds/site/[domainName]",
contentType: "application/atom+xml",
headers: {
"Authorization": "Bearer " + [accessToken],
"GData-Version": "1.4"
},
data: ["<entry xmlns='http://www.w3.org/2005/Atom' xmlns:sites='http://schemas.google.com/sites/2008'>",
"<title>What a site</title>",
"<summary>Best description ever.</summary>",
"<sites:theme>ski</sites:theme>",
"</entry>"].join(""),
////// LOGGING \\\\\\
beforeSend: function () {
console.log('-------Making-the-request-------');
},
success: function (result) {
console.log(result);
},
error: function (xhr, ajaxOptions, thrownError) {
console.log(thrownError);
console.log(xhr.status);
}
});
(In several cases below, I'm writing https:// as [https] due to my account still being restricted to 2 links in a post).
At this point everything was going great, I thought I had everything set to use the code into my ASP.NET site. Alas, things don't always go to plan. When I executed the exact same AJAX call from within my application (right now still hosted on [https]localhost:44301), I get the following error:
XMLHttpRequest cannot load
[https]sites.google.com/feeds/site/[censored] Response to preflight
request doesn't pass access control check: No
'Access-Control-Allow-Origin' header is present on the requested
resource. Origin '[https]localhost:44301' is therefore not allowed
access. The response had HTTP status code 405.
The usual CORS error. I was surprised though, as the advised way of making requests to Google APIs is exactly that. I've also found an article about using CORS with the Google Cloud API:
https://cloud.google.com/storage/docs/cross-origin
In the article it states:
Most clients (such as browsers) use the XMLHttpRequest object to make
a cross-domain request. XMLHttpRequest takes care of all the work of
inserting the right headers and handling the CORS interaction with the
server. This means you don't add any new code to take advantage of
CORS support, it will simply work as expected for Google Cloud Storage
buckets configured for CORS.
Of course, this isn't the Google Sites API, but I find it hard to believe that Google hasn't implemented the same functionality in all of their APIs.
Does anyone know whether it's possible to achieve successful requests such as this from within a standalone ASP.NET application? And if so, how?
Many thanks for spending time to read about my hardships.
UPDATE:
I've contacted Google Apps Support regarding my issue, and have gotten the following response:
In addition to the information you provided, I also reviewed your post
at
CORS authenticated requests to Google Sites feed/API blocked.
The note at
https://developers.google.com/google-apps/sites/docs/1.0/developers_guide_java#SiteFeedPOST
only reinforces the statement under 'Can I create a new Google Site?'
and 'How do I copy a site?' at
https://developers.google.com/google-apps/sites/faq#Getting__Started,
which states 'Google Apps users can use the site feed to ...' However,
I don't see why this is relevant to your issue, if you've authorised
against your domain administrator account, as the note is only
indicating that gmail.com users won't be able to use the listed
methods to create, or copy a site.
I haven't used CORS, so can't comment on it's operation, but have been
able to successfully list, and create sites using HTTP GET and POST
requests via a raw HTTP client, so the API is operating as it should
with regard to cross domain requests. I used the sample XML document
at
https://developers.google.com/google-apps/sites/docs/1.0/developers_guide_protocol#SitesFeedPOST
to create my site, configuring the client with credentials for my
Developer console project. The fact that the request only fails in
your ASP.NET site implies that there is something in that environment
which isn't configured correctly. Unfortunately that's outside my
scope of support, so I'm unable to provide any specific advice, other
than to check the relevant documentation, or post a request in the
ASP.NET section of Stack Overflow at
https://stackoverflow.com/questions/tagged/asp.net.
Could you try again after removing "GData-Version": "1.4"?
If you really want to send this value, send it by adding a query parameter such as v=X.0. Resource from here
UPDATED:
"Note: This feature is only available to Google Apps domains." From guides
SOLVED
It seems that a lot of browsers would still block an AJAX request across different domains, even when it's allowed by the API you're trying to reach. Instead of using AJAX, I'm now using the C# WebRequest in a DLL.
Example:
// Create the request
WebRequest request = WebRequest.Create("https://sites.google.com/feeds/site/[domainName]");
request.Method = "POST";
request.contentType = "application/atom+xml";
request.Headers.Set(HttpRequestHeader.Authorization, "Bearer " + [accessToken]);
request.Headers["GData-Version"] = "1.4";
// Fill in the data and encode for the datastream
string data = ["<entry xmlns='http://www.w3.org/2005/Atom' xmlns:sites='http://schemas.google.com/sites/2008'>",
"<title>What a site</title>",
"<summary>Best description ever.</summary>",
"<sites:theme>ski</sites:theme>",
"</entry>"].join("");
byte[] byteArray = Encoding.UTF8.GetBytes (data);
// Add the data to the request
Stream dataStream = request.GetRequestStream ();
dataStream.Write (byteArray, 0, byteArray.Length);
dataStream.Close ();
// Make the request and get the response
WebResponse response = request.GetResponse ();
More info can be found on MSDN:
https://msdn.microsoft.com/en-us/library/debx8sh9(v=vs.110).aspx
I am trying to create a GET request using HttpRequester ( addon in firefox ). And I am analyzing packet using the Http Fox ( addon in firefox ).
I have created a GET packet with following parameters
url :-http://enquiry.indianrail.gov.in/ntes
Headers
Host :- enquiry.indianrail.gov.in
Referer :- http://enquiry.indianrail.gov.in/ntes/
When I submit this request. I get a response code of 200. In the HttpFox add on, When I analyze my packet, I see that there is additional field in header named
cookie with value _ga=GA1.3.150104442.1441509203.
Relevant Information
Before sending the request deleted all the cookies for enquiry.indianrail.gov.in .
Running all this behind a proxy server.
I get the respone 200 in HttpRequester, while 302 in HttpFox
I want to know, If I am not attaching cookie in my header,than Why HttpFox shows cookie in the header ( with response code 302 ) ?
The _ga cookie is a google tracking cookie. It is a client cookie created by google analytics.js running in your browser. The analytics.js is included by common.js, which is included in the /ntes home page.
HttpRequester will not execute the javascript logic which creates the client side _ga cookie. It may not automatically load the analytics.js either. If you are trying to automate a page that needs to execute javascript, one simple way is to use a headless browser, such as phantomjs
I have a special situation where the sites visitors can access the page from a certain domain but no others. So HTML and assets are no problem as long as they are stored on the server. Google Analytics on the other hand requires a download of analytics.js from Googles servers, which is impossible.
So I'm looking for a way to proxy this. The webserver itself has internet access and could relay the trafic. To report to Google about my page view, a single pixel GIF is downloaded from Google, described here: https://developers.google.com/analytics/resources/concepts/gaConceptsTrackingOverview
I think it would be kind of easy to get all the parameters in the GIF and use the measurement protocol to report to Google from the server - but the hard bit is to get all this info to the server. To download analytics.js and modify it to go to my own server seems to me as a hack that ain't future proof at all. To just get the current page from the user to the server is not a big deal, but we would like to get the user id, browser version and everything you get with Analytics.
How would you do it? Do you find a solution for this?
Update: Google has since released server-side GTM, which allows you to proxy requests and scripts through a custom domain. In most use cases I can imagine, this would be the much superior solution to a dyi proxy.
As pointed out in my comment the utm.gif is no longer used. Google Analytics has completely switched to the Measurement Protocol and data is now sent to the Endpoint for the Measurement Protocol at google-analytics.com/collect. Actually this still return a transparent pixel since calling an image with parameters is a probate way of transmitting informations across domain boundaries.
Now, you could just the Measurement Protocol to implement your own Google Analytics tracker.
To quote myself:
Each calls includes at least the ID of the account you want to send
data to, a client id that allows to group interactions into sessions
(so it should be unique per visitor, but it must not identify a user
personally), an interaction type (pageview, event, timing etc., some
interactions types require additional parameters) and the version of
the protocol you are using (at the moment there is only one version).
So the most basic example to record a pageview would look like this:
www.google-analytics.com/collect/v=1&tid=UA-XXXXY&cid=555&t=pageview&dp=%2Fmypage
You probably would want to add the users IP (will be anonymized automatically) and the user agent.
However it sounds like you prefer to use the standard Analytics code to collect the data and relay the tracking call via your own server. While I haven't used the following in production I don't see any reason why it wouldn't work.
First you need the analytics.js file. Self-hosting the file is discouraged, but the given reason is that the code is updated sometimes by Google and if you host it yourself you might miss the updates. This can be remedied by setting up a cron job that downloads the file regularly to your server so you always have a current version.
Next you'd adapt the GA bootstrap function to load the code from your own server:
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.myserver.com/analytics.js','ga');
Now you have the code, but the tracking call will still be sent to the Analytics Server (i.e. in your case it won't be sent at all). So you need to re-route the call via your server.
To make this possible the Google (Universal) Analytics Code has a feature called "tasks". Tasks are functions within the tracking code in which the tracking call is being assembled.
It is possible to modify tasks by using the "set" function of the tracker object, using the taskname as parameter and passing a function that overwrites/overloads the task function.
The following is pretty much the example from the Google documentation (except I omitted the part where data is still being sent to Google - you don't need this at this point):
ga('create', 'UA-XXXXX-Y', 'auto');
ga(function(tracker) {
tracker.set('sendHitTask', function(model) {
var payLoad = model.get('hitPayload');
var gifRequest = new XMLHttpRequest();
var gifPath = "/__ua.gif";
gifRequest.open('get', gifPath + '?' + payLoad, true);
gifRequest.send();
});
});
ga('send', 'pageview');
Now this sends the data to a file called __ua.gif at your own server (if you need to send data cross-domain you can simply do a var ua = new Image; ua.src = gifPath + '?' + payLoad to create an image request).
The model parameter to the sendHitTask-function contains (apart from a lot of overhead) the payload, that is the assembled query string that contains the analytics data. You can then make your _ua.gif a script that proxies the request to the google-analytics.com/collect.
At this point the user agent will be your script and the IP adress will be that of your server, so you need to include &uip (User IP override) and &ua (User agent override) parameters ( https://groups.google.com/forum/#!msg/google-analytics-measurement-protocol/8TAp7_I1uTk/KNjI5IGwT58J) to get geo and technical information.
If you are feeling more adventurous you can override the buildHitTask instead and try and add the additional parameters there (more hassle probably since you'd need to get the IP address from somewhere).
For additional parameter see the reference for analytics.js and the Measurement Protocol.
I want to track clicks with google analytics. Is there any tutorial for this?
I use the following jQuery code to add event tracking, assuming, that all external links begin with http://:
$(document).ready(function() {
$("a[#href^='http://']:not(.internal)").addClass("external").bind('click keypress', function(event) {
var code=event.charCode || event.keyCode;
if(!code || (code && code == 13)) {
if(pageTracker){
pageTracker._trackEvent('outgoing', 'click', this.href);
};
};
});
GA doesn't auto-track external links, so you have manually call one of its functions on all your exit links. You can track it as a virtual page view or as a custom variable or event, passing the relevant information you want to track, to the GA functions (like the exit link url).
Depending on how your links are setup, you can easily setup an click event listener to trigger the GA function call. For example, if all of the exit links have a specific css class attribute associated with them then you can hook the click event to that.
If all of your exit links point to your php script and only pass an ID number to it or something (no actual exit url), then you will not be able to pass to GA the exit link url. If you are fine with just passing the ID then you can use that instead of the url and pass as the virtual url or custom variable value "/exit/[id]" or whatever else makes sense to you. And you can also look for the php script's url as a way to hook the click event to all your exit links, if there is no other unique identifier.
If none of this is an option and you have to do it server-side...then you're kind of out of luck on doing it the "easy" way. GA has an API for interacting with it via server-side code but it's only 1-way. You can use it to get information out of GA but you can't use it to put information into it.
However, what you CAN do is...when a request to GA is made, take a look at that request in Firebug > NET tab or in Charles Proxy or some other request sniffer program, or take a look at the GA's url in its noscript tag. The way it works is a request is made to the GA server and variables and values are appended to the url as a query string.
So what you can do is build a url like that and use cURL to make a request to the GA server...but here's the tricky part: you have to fool GA into thinking the request was made from a browser, not your server. So you need to make sure you send header information with the cURL request to make it look like a browser made the request. Ideally you will want to use whatever header information was sent to your server from the client so that GA can record the hit as if it were the user, so that reports can populate accordingly.
You could send the requests to your server, and your server could send the user a 301 redirect.
Your could then do server-side google analytics, by sending requests to http://www.google-analytics.com/utm.gif to google, like this project does: http://code.google.com/p/php-ga/
You can use this jQuery library to help with sending click events to GA:
https://github.com/spilliton/track_that