I would like to create a daily Kibana table report in a dedicated Slack channel. E.g. Every day, at 10:00 AM I would like a text report on basic information from Kibana visualizations to trigger and say something between the lines of 'Yesterday xxx transactions were made, we sold a XXX's worth of procuts.'
Is this possible to achieve?
Related
I am looking for options to ingest Google Analytics data(historical data as well) into Redshift. Any suggestions regarding tools, API's are welcomed. I searched online and found out Stitch as one of the ETL tools, help me know better about this option and other options if you have.
Google Analytics has an API (Core Reporting API). This is good for getting the occasional KPIs, but due to API limits it's not great for exporting great amounts of historical data.
For big data dumps it's better to use the Link to BigQuery ("Link" because I want to avoid the word "integration" which implies a larger level of control than you actually have).
Setting up the link to BigQuery is fairly easy - you create a project in the Google Cloud Console, enable billing (BigQuery comes with a fee, it's not part of the GA360 contract), add your email address as BigQuery Owner in the "IAM&Admin" section, go to your GA account and enter the BigQuery Project ID in the GA Admin section, "Property Settings/Product Linking/All Products/BigQuery Link". The process is described here: https://support.google.com/analytics/answer/3416092
You can select between standard updates and streaming updated - the latter comes with an extra fee, but gives you near realtime data. The former updates data in BigQuery three times a day every eight hours.
The exported data is not raw data, this is already sessionized (i.e. while you will get one row per hit things like the traffic attribution for that hit will be session based).
You will pay three different kinds of fees - one for the export to BigQuery, one for storage, and one for the actual querying. Pricing is documented here: https://cloud.google.com/bigquery/pricing.
Pricing depends on region, among other things. The region where the data is stored might also important be important when it comes to legal matters - e.g. if you have to comply with the GDPR your data should be stored in the EU. Make sure you get the region right, because moving data between regions is cumbersome (you need to export the tables to Google Cloud storage and re-import them in the proper region) and kind of expensive.
You cannot just delete data and do a new export - on your first export BigQuery will backfill the data for the last 13 months, however it will do this only once per view. So if you need historical data better get this right, because if you delete data in BQ you won't get it back.
I don't actually know much about Redshift, but as per your comment you want to display data in Tableau, and Tableau directly connects to BigQuery.
We use custom SQL queries to get the data into Tableau (Google Analytics data is stored in daily tables, and custom SQL seems the easiest way to query data over many tables). BigQuery has a user-based cache that lasts 24 hours as long as the query does not change, so you won't pay for the query every time the report is opened. It still is a good idea to keep an eye on the cost - cost is not based on the result size, but on the amount of data that has to be searched to produce the wanted result, so if you query over a long timeframe and maybe do a few joins a single query can run into the dozens of euros (multiplied by the number of users who use the query).
scitylana.com has a service that can deliver Google Analytics Free data to S3.
You can get 3 years or more.
The extraction is done through the API. The schema is hit level and has 100+ dimensions/metrics.
Depending on the amount of data in your view, I think this could be done with GA360 too.
Another option is to use Stitch's own specfication singer.io and related open source packages:
https://github.com/singer-io/tap-google-analytics
https://github.com/transferwise/pipelinewise-target-redshift
The way you'd use them is piping data from into the other:
tap-google-analytics -c ga.json | target-redshift -c redshift.json
I like Skyvia tool: https://skyvia.com/data-integration/integrate-google-analytics-redshift. It doesn't require coding. With Skyvia, I can create a copy of Google Analytics report data in Amazon Redshift and keep it up-to-date with little to no configuration efforts. I don't even need to prepare the schema — Skyvia can automatically create a table for report data. You can load 10000 records per month for free — this is enough for me.
I'm setting up a Dashboard on Tableau Server and with a data source extract from Google Analytics.
I need it to refresh hourly.
When publishing the data source I have selected Fixed start and incremental refresh, and i can see that the end date is hardcoded to yesterday and I can only schedule update once a day.
This means that my Dashboard will only have data for the previous day. Is this a known limitation? Is there a way to achieve hourly automated data refreshes?
I thought of three possible solutions:
1) Using command line data extract update, but I don't see any parameter that allows me to control date range, so the end date would always be hardcoded to yesterday (?)
2) Using Tableau API. But again, my extract will only refresh to latest day yesterday.
3) Setting up a separate process to download the data into a database and then connect to it with Tableau. Not sure if in this case I will be able to incrementally update the extract or if I have to reload the entire extract all the time.
Any help would be much appreciated!!
I'm attempting to create a report that is based off the current date. So, for example, creating a line graph that shows total work for all resources for the next 2 months. It would be very similar to the resource overview dashboard, but it wouldn't be pulling in data from the entire project.
The 'Resource Usage' view below has been very helpful, as it would be visual aids based on the hour allocations below.
We can create a graph like the one below in the reporting module, I would like the graph to only look at the next 2 months (instead of the entire project duration).
The goal is to look at capacity and future work allocation to easily look at resource availability to aid in assigning future tasks.
Thoughts? Tips? Advice?
You should be able to use the built-in Report capability in MS Project 2016. Try modifying the Progress Versus Cost chart in the Cost Overview report.
Is there a way to send a daily report with the current date's traffic only (not the whole month)? For example, if the report is sent 2015/01/25, I'd like to see traffic from 2015/01/25. I know I can manually select the data range but I'm looking for something that will automatically send daily PDF email report for current day only.
In the Email dialog of the Custom Reporting section, you can select a frequency of Daily and that will send your report every day for the previous day's data. Usually this is what you want since data for today will not always be current or accurate.
If you really need today's data, you can use the Core Reporting API and set the value for the start-date and end-date parameters to 'today'.
If you want to automate this process without too much effort, I'd recommend checking out the Google Analytics Spreadsheet Add-on, which uses the Core Reporting API behind the scenes. It won't email you reports, but you can do everything else you're asking.
I got some problems with the automatic generated reports option in Google analytics.
So, i made some reports in my dashboard, personalized them and they are ready to be send to the customer.
I set the dates : 1 Januari - 31 Januari 2014 and compare them with 1 Januari - 31 Januari 2013.
I use the E-mail option in google analytics and use the option automaticly send this report every first of the month too the customer.
But when the customer recieves the reports it doesn't display the dates i setted, but it displays the dates i sended the reports. So i double checked, tested, sended them again and kept facing the same problems.
But when i don't use the automaticly generated option and send them directly to the customerm, the dates are right.
How can i fix this problem ?
As far as I know, you can't send automated reports with a time range set beyond what Google Analytics has built in - Once, Daily, Weekly, Monthly, Quarterly.
These settings send the report for the previous specified time period.
If you want to send a monthly report with the previous year of data, you are out of luck. Monthly reports only show the previous month and so on for the frequency settings. In short, no rolling data of a user specified time period.
And, no comparison data either like what you are trying to send.
I have to send the previous 90 days of data from a report every month. Since I am unable to do that in Google Analytics with the scheduled reports, I schedule GA to email me the report once a month as a reminder. I then go in and manually create the 90 day report and use GA to email it using the Once frequency.