I am trying to automate an scraper which search on Google for newly published websites which contain certain keywords. Resulted websites should also belong to two specific countries.
Currently, I am using Google Custom Search API connected to 2 different Google Programmable Search Engines. First search engine is set to only search in United States, and second one only search on Canada.
For obtaining search results, I am sending an API request to Google API endpoint, using next parameters:
https://customsearch.googleapis.com/customsearch/v1?key=${this.apiKey}&cx=${this.searchEngineId}&q=${question}&dateRestrict=d[1]&num=${numResults}&start=${index}&sort=date
However, I have the impression that I am getting pages which are updated last day, rather newly published websites.
Also, I am fetching many duplicates websites when using both search engines. Both search engines are configured for different countries, so I would expect different results from them
Related
From research I've learnt that Google Maps and Google Places are not the same thing, but can someone confirm this deduction: does that cause the app to not show all the locations that are shown in Google Maps (let's assume because the business owner didn't list it in Google Places)?
You shouldn't expect the Maps API Web Services and the Google Maps website to work in the exact same way at all times. The Google Maps website gathers data from many different sources. Also, the search stack is different. Due to this, you may find differences between the results of these two services, and you should not use Google Maps website results blind eyed as a control group for your Web Services requests. You can find confirmation that Web Services and maps.google.com can show different results in the Google Maps API FAQ at https://developers.google.com/maps/faq#geocoder_differences.
I have a website where people post items and I want to show some basic analytics in their own user panels, such as how many people viewed the posted item.
I am wondering if I can use a single mothership Google Analytics account across the whole site. Then, in my code for the user area, I have it query Google Analytics for that specific users posts and return the number of visits that specific person has had.
So my google analytics account may say "200 views across the whole site" but within the specific users panel I will be able to say "Your posts received 50 views".
This is the closest bit of documentation I could find on the subject:
https://developers.google.com/analytics/devguides/reporting/core/v3/
It sounds like what you would need is multiple Google Analytics IDs / requests per page. This can be done with the newest version of Analytics, though not with the Classic Analytics.
You can certainly have multiple Google accounts access the single Google Analytics for a single domain. You can certainly split reporting capabilities and access to various Analytics features across multiple accounts.
From the docs:
You can install multiple instances of the Google Analytics tracking
code on your web pages to send data to multiple properties in your
account.
Read: more here.
Is it possible to find out in real time what query a visitor searched before landing on your site?
Google switched the URL structure to precede query (q=) with a # instead of ? or & to avoid sending data in the URL. Now document.referrer is https://www.google.com with no parameters to parse.
Before:
How does a website know the Google query I used to find it?
My goal is letting the website search its own database for related content or tools not indexed by Google.
Google Analytics has real time data. The limited beta API may enable joining a session via primary key (user_id or IP) to get their referring keyword.
I'm trying to add recommender systems to an existing website. In particular, I'd like to implement item-item collaborative filtering, to figure out what pages users tend to visit in the same session--much like Amazon's "People who viewed this item also viewed...."
At a minimum, collaborative filtering requires data on each individual viewing session, so that the algorithm can determine which pages get viewed together, rather than just tallying up how many times each page gets viewed in the aggregate.
If I were creating a new website, I could pretty easily add code to collect this data. However, this is an existing website, and has been set up to use Google Universal Analytics.
I have two questions:
Can I get Universal Analytics Data through an API? I need to be able to analyze the data using my own algorithms, not just look at it in a dashboard. I know about the Core Reporting API--but the Core Reporting API doesn't seem to include any extra Universal Analytics variables. I know about the API for sending Universal Analytics data, but that's not what I'm trying to do here.
Assuming I can query an API or otherwise export the Universal Analytics data, will I be able to distinguish individual sessions? The idea here is not to ask questions about individual users (let alone associate their data with some other data), but simply to figure out which pages were viewed in the same sessions.
Thanks for your help.
You can use the Google Analytics Core Reporting API many combinations of the available Dimensions and Metrics. You should check out the Common Queries page to get a sense of how precise you can get in terms of how people might use your application.
Also the Hello Analytics APIs Quickstart guide is a good place to start if you haven't developed an application with Google APIs.
I am new to Google Analytics and found a very interesting help subject on how IP masking works that details that some IP data is dropped by the collector before it is stored in the google analytics database.
I have a question around how the similar behaviour works for 1. Filters and 2. Exclude URL Query Parameters option within view settings.
Basically for each all data will be sent to Google BUT does the data I have excluded using these methods ever get actually databased? with the data excluded being removed from the view served in the GUI / API OR does the data I have excluded get dropped at collection before it is databased. I guess the answer could be different for Filters and Exclude URL Query Parameters.
Many Thanks