I have a simple page that provides a search experience. It allows users to search via a query form, filter results, and perform more in-depth searches based on the results of the first search.
I would like to get some metrics around the user experience and how they are using the page. Most of the user actions translate in a new query string. For example:
how many users perform a search and then follow up with another search / filter
how many times a wildcard is used in the search query
how many results does a user browse before a new search
I am also limited of using google analytics and the sort because of copyright issues (maybe I can make a case if it is really the way to go for open web analytics or smth). Server side I am thinking of using cookies to track users and log4net to log what they do, then dump the info in a db and do analysis from there. Or log to the event viewer and use the Log Viewer to get the info from there.
What do you think is the overall better approach?
I would recommend you use an existing, off-the-shelf solution for this, rather than building your own - it's the kind of project that very rapidly grows in size. You go from the 3 metrics in your question to "oh, and can you break that down by the country from which the user browses?", "what languages affect the questions?", "do they end up buying anything if they click results for bananas?". And then, before you know it, you've built your own web analytics tool...
So, you can either use "web analytics as service" offerings like Google Analytics, or use a more old-fashioned log-parsing solution. Most of the questions you want to answer can be derived from the data in the IIS web logs; there are numerous applications to parse that data, including open source and free solutions.
It's been a long while since I used a log file based analytics tool, but my ISP provides AWStats, which seems pretty good - to do what you want, you'll have to set up specific measurements around your search page; not sure if AWStats does that (Google Analytics definitely does); check the Wikipedia list for log file analysis tools which do that.
Obviously you need to log every submit of the search page.
In particular you need to log:
DateTime.Now
SearchString
SessionID
You could also store a counter in the Session that will be incremeted each time a user loads a page, that is not the search page.
If the users performs a search you could read that value from the session, store it in the database and reset the counter.
Be aware that the metric of "how many results does a user browsw before a new search" should only be taken as an estimate and not as a real metric, due to cookie support, multitabbing, page reloads et cetera.
Related
My question is more of a conceptual one, but in my specific case I am using Google Analytics 4. If the question is unclear, here it is in scenario form: Some guy visits my site x.com after a google search. He closes the tab, does another google search, and arrives at my other site y.com. How do I know it's the same person? I don't think there's anything I can do with User ID's in this situation. How would I solve this?
This isn't without fault, but if you are implementing it via Google Tag Manager, you have more control over the data being sent and on top of that, if you are transporting the data via Google Tag Manager server side container.
You would use a single server (but possibly different containers) or use BigQuery and either use the templateDataStorage API call or the BigQuery API call.
Essentially, the first time you see a google cid or an IP address or combination of user agent and ip address you would store it in the server or in a BigQuery table as a key and create a random associated value next to it.
At each time, across all your sites, you would check to see if the IP address or CID or combination of user agent and ip exists in the server or in the BigQuery table, then output the random value as a custom dimension and if not, it will create one.
Actually you probably wouldn't.
Presumably you could try fingerprinting, but depending on your legislation that might not be quite legal, and it tends to work a lot better in a lab than in real life. Also browsers start to implement anti-fingerprinting measures such as trimming the user agent, and denying access to browser properties such as installed plugins.
I have heard of experimental approaches to recongnize users via usage patterns - e.g. how do they move their mouse etc. I am not aware of any actual product that uses this, and I am not convinced it is a useful (or even legal) approach.
But in general, when it comes to cross-domain detection for unrelated visits (moving from domain to domain works via link decorators, and even that is affected by browser protections) you have the combined power of browser vendor against you, who try to make this harder (either for genuine concerns about privacy, or to establish themselves as the single gatekeeper for user identity. E.g. Google has a huge user base that is almost constantly logged in to Google accounts or Android smartphones, which helps with identifying users all over the web).
We currently have an existing page for our checkout, let's say checkout is the URL.
We would like to create a new version of this page and specifically control the number of users that enter this view/route.
As an example, we would start with 50% of all users will access the original and the other 50% will access the new.
Ideally, we would like to keep the same URL structure and just change out the view that is displayed. No changes are required for the existing view/path setup.
I have looked at a bunch of A/B Testing options but cannot see one that fits these requirements. Ideally, I'd like to keep this server side. Can anyone recommend any A/B testing options that can help with this?
There are multiple ways of accomplishing this. You could use a testing tool server-side SDK like Optimizely or Google Optimize.
Both having cons and pros. This would allow you to make a bucketing decision on the server using a unique user identifier. I typically use a random string in a Cookie. This would allow the controller to switch out the view based on the variant and send goals and objectives to the correct analytics platform. Optimizely's SDK has bucketing built-in and Optimize allows the developer to perform the bucketing themselves. Avoiding the need for an API and therefore slowing the server speed down.
Another way is to add a param to the end of the URL that switches out the view on the server. Then use a redirect test on a front-end testing tool to redirect from the default control to the variation. This typically tends to be better if you are using a caching server as you can avoid the logic involved in cache busting.
Here are a few links to help you out!
https://developers.google.com/optimize/devguides/experiments
https://docs.developers.optimizely.com/full-stack/docs
If I build a website for my new awesome mobile app (or web service or whatever) I might want to do a slow launch, sending email invites to the first x people to register on the site.
Is there a good way to link each registered email to the corresponding data in google analytics (or any similar service), and query them based on location, language, etc.?
Maybe the spanish version isn't quite done yet, so I don't want to invite people who used a spanish browser to sign up. Or maybe my app is location-dependent (like time tables for buses) and just doesn't work at all outside of my home town.
I really want to have a simple email-only "registration".
It is completely possible, although it may breach some of GA's terms of use if done wrong.
You should not store email addresses in any way as part of your GA data because it would be considered personally identifiable data. However, there is nothing saying that you couldn't store a kind of GUID for each user, and then compare that with email addresses offline - although the user should be made aware that any actions they take while using your service/application/whatever are being tracked with the capability of being personally identified.
As far as getting the actual data that you are discussing, language and location are stored by GA by default, so no headache there!
The best way to store the user's GUID would probably be in a custom dimension. How you do this is going to depend on how you build your product. I had to write a tracking library using the measurement protocol for an AS3 project awhile back because there isn't an AS3 library that is supported anymore. If you are using JavaScript, it will be much easier, as Google offers native JS libraries to handle web analytics.
Finally, try taking a look at the documentation. Its pretty easy to understand
Say, I'm developing a Windows (if OS is important) application that will be available to download for free and I would like then to collect some usage statistics. In the easiest case - count of application launches. It looks superfluous to maintain a server (e.g. VDS) just for this.
I've been thinking to use Google Analytics for this (manually send requests to GA server). This will probably work, but it is not GA designed for - the idea looks like a hack.
What are the options here?
I don't think this is a hack. It's all just data about user interaction. There is little logical difference between opening a desktop app and clicking a button vs opening a web page and following a link. Both are measurable user actions you can track, aggregate and put on graphs.
In fact, Google provides a lower level HTTP based "Measurement Protocol" that is intended for exactly that.
https://developers.google.com/analytics/devguides/collection/protocol/v1/
From the overview:
The Google Analytics Measurement Protocol allows developers to make
HTTP requests to send raw user interaction data directly to Google
Analytics servers. This allows developers to measure how users
interact with their business from almost any environment
Just put an HTTP request with the correct parameters in your application launch or button click code and it will collect the data. Any data you want to collect.
In other answers to this question there are suggestions like making web services or storing the data locally but why reinvent the wheel? Google Analytics already provides the collecting and reporting tools and it seems like a good solution.
Does anyone know a script/recipe/library to find most relevant contact information on a website?
Some possible case:
Find contact phone number on a personal web page
Find owner email address on a blog
Find url of the contact page
Check out WSO2's Mashup Server. You can run it on your local machine and follow the tutorial for scraping. You could pass the dynamic parameters you need into the <http> element of the scraper to loop through multiple sites running the same scrape, then push everything to a collection source (AJAX application for capturing the information or store inside WSO2 server). You can write very complex search patterns using XPath and XSLT to capture the information you want.
I don't have enough information about the specific sites you are scraping to help with the script, but any way you go, it's going to take a lot of trial and error until you get the result you are looking for.
Happy scraping!
I'm not aware of any libraries that do this.
Hm, I would use regular expressions to match for phone numbers and email addresses, combined with a web spider that walks the site, and then a method for ranking the contact information.
Typically contact information will also be partnered with one of a few common labels such as "Support", "Support email", "Sales", etc. There's probably a dozen or so versions of this that will cover 95% of all sites in English.
So, basically I would start by building a simple recursive web spider that walks all the publicly accessible pages in a given domain, parsing the HTML for email addresses and phone numbers, and making a list of them, and then ranking them based on whether or not they are listed near to any of the common labels.
It won't be perfect, but then again, that's part of the value of the algorithm - making it smarter, and tweaking it over time until it gets better.