I’m having some trouble coming up with a future-proof-ish design for reports for a company. Essentially the requirements are:
Be able to pull whatever data from the database
Generate formatted report from that data by populating a template (HTML, docx)
Export to Word and/or PDF
So initially I made an API endpoint per report (this is a web app), and had PDFs generated and formatted correctly.
But now I need to get the data into .docx/Word format, and I’m trying to figure out how I can design something as D.R.Y. as possible so that I don’t have to put in a TON of work every time the company decides they need another report (they’ve done this two, three times which is how I became aware that I had coded myself into a corner).
Every report I’ve done thus far has been done via a “brute-force” method: code the queries needed for the report, format the data, and then render to PDF (using HTML to PDF via phantomjs).
The complexity occurred when the company came back and said “Hey, we need all of those reports in Word format, also we have 3 other new reports that we need and a report that is a slight variation on the old one but +/- 2 fields”.
I am just having trouble coming up with a solid design/abstraction here, one that doesn’t send me down a week long hacking spree every time a requirement changes.
Related
I am trying to generate a report using Google Analytics Explore tab using Free Form technique. Few weeks ago I could use Message name, stream name and time to see all the notification name, platform and total no of click. I exported the same to excel file.
but today when I tried to generate the same I couldn't find "Message Name" dimension. Is this field removed from pre defined/custom dimensions from GA? or am I doing something wrong?
My main purpose is to get all list of notifications sent via Firebase.
Any help will be deeply appreciated.
Given that you excluded the obvious issues like using the too-fresh data, the proper way to debug it is to export the data into a sample BQ table, then conduct exactly the same analysis that you're trying to conduct in GA4's explorer. From there, if your issue is with explorer's filters, you will quickly see it.
If, however, you're able to see your event properties in BQ, but not able to get the explorer to display them... Well, Google likely saved quite a lot of money on GA4. UA was pretty expensive. GA4 now introduces all these amazing features like data retention limits, properties' values cardinality bugs, odd inconsistencies between explore's reports and default reports and so on.
For now, the best way to really access your data minus all the artificial limitations of GA4 is to ETL your data from there either through the reporting API or exporting it to BQ.
I'm trying to build a stock analysis spreadsheet in Google sheets by using the importXML function in conjunction with XPath (absolute) and importHTML function using tables to scrape financial data from www.morningstar.co.uk key ratios page for the corresponding companies I like to keep an eye on.
Example: https://tools.morningstar.co.uk/uk/stockreport/default.aspx?tab=10&vw=kr&SecurityToken=0P00007O1V%5D3%5D0%5DE0WWE%24%24ALL&Id=0P00007O1V&ClientFund=0&CurrencyId=BAS
=importxml(N9,"/html/body/div[2]/div[2]/form/div[4]/div/div[1]/div/div[3]/div[2]/div[2]/div/div[2]/table/tbody/tr/td[3]")
=INDEX(IMPORTHTML(N9","table",12),3,2)
N9 being the cell containing the URL to the data source
I'm mainly using Morningstar as my source data due to the overwhelming amount of free information but the links keep on breaking, either the URL has slightly changed or the XPath hierarchy altered.
I'm guessing from what I've read so far is that busy websites such as these are dynamic and change often which is why my static links are breaking.
Is anyone able to suggest a solution or confirm if CSS selectors would be a more stable / reliable method of retrieving the data.
Many thanks in advance
Tried short XPath and long XPath links ( copied from dev tool in chrome ) frequently changed URL to repair link to data source but keeps breaking shortly after and unable to retrieve any information
I want to use Firebase Analytics in my website in order to get some statistics for the visitors of each page (I don't want to track user journey in the site). I wanted to define multiple data streams (one for each url) in my google analytics dashboard, but then it warned me with the following message:
In most cases, a single web stream will meet your measurement needs. Using multiple web streams to measure different pages or sites in a single user’s journey may lead to inconsistent results.
in my case-where I want to see the statistics of my site based on its pages (urls)-should I define multiple data streams?
As the message says, it is not necessary to split based on the path in the web site.
You can in the Google Analytics console instead filter based on that path. This gives you the best of both worlds, as you can show stats for a specific path, but also for the site in its entirety.
I ended up using separate data streams in a similar situation where we had a multilingual site with a domain-per-language. The analytics dashboard lets you separate the data by domain, but the tools are bulky and don't seem available everywhere.
In short, creating a separate stream for data that is always going to be viewed separately can be a real convenience, even if it's not "the right way".
The main caveat from the data-streams documentation seems to be that you can miscount data. For instance, a user switching from the English site to the French site will be counted as a visitor on each rather than as a single visit. As long as you're aware of the data implications, you should be okay.
We usually import data from google analytics once a month and use it for some reporting needs internally. The problem is that we have to do this manually and it would be nice if we could automate the process and potentially increase the once a month routine to once a week or even daily. Our ultimate goal would be to have a tool set up to import the data automatically and store it to a csv or excel file. The output file doesn't really matter to us. As long as we can have the data pulled from GA on a regular basis without manual intervention, we'll take care of what to do with the data once it's here. We use some java based executable (found online) for this but we run this manually to extract the data.
I have looked for some solutions, even open source tools(.Net preferably, anything but java based really) but I have not really found anything. most of them require manual intervention to export data, and the best they can do is have reports generated automatically based on that data.
Our last resort would be to write up something ourselves but I would like research this a bit further and save developing/programming time. I am pretty sure someone out there has at least encounter/though of this problem.
Any help, pointers or redirection to better sources would be much appreciated.
Thanks
Have you looked into the Core Reporting API or Google Analytic's Magic Script? These would allow you to pull data into Google Spreadsheets on a regular basis. Specifically, the Magic Script will allow you to setup triggers to run a function on reoccurring time interval E.g. daily, weekly, monthly, etc.
We have an ASP.NET 2 application and at the moment for the reporting, we are utilising DataSet/DataTable taking some storedprocedure/query and display on the screen (grids). For the graphics, we are using Dundas Chart component and again we are utilising DataSet/DataTable to display it. These thing has been running well ... no issue.
Now, we are having a challenge that every single report that we produce above (grid or graph), the user wants to consolidate into 1 single click solution which means if they click the "Consolidated Report" it will produce the page as well (grid/graph) into 1 page. But the issue is that the user wants use the display (grid/graph) and download into 1 file. It could Ms Words/Ms Excell.
How we are going to achieve this?
I am thinking to do as follow:
1. Convert everything to use the Reporting Services (client) .. or
2. Is it possible to do like convert the page/grapf into Word/Excel into a single page? Is there any other solution?
The issue is that the we have simple stored procedures to get the data source out and then we are using that data and do the massive data manipulation in DataSet/DataTable (loop through the records, create a new dataset and manipulate etc etc) and then to construct the final report and display on the screen.
I am appreciated your comment.
Thank you
In my opinion the best solution will be integrating the HTML into a PDF.
You can convert the HTML generated into PDF easily with ABCPdf from WebSuperGoo (I suspect that other solutions can work, but I have only used this product)
See this live demo, and select "Add URL" and point to your URL or check out this example