Day-over-day GA BigQuery Data - google-analytics

I'm working on a few queries to gauge day-over-day performance using Google Analytics data in BigQuery. My project is currently set up with data streamed in from GA. The problem is that yesterday's table, ga.us.ga_sessions_20181105, isn't created until sometime in the late morning.
The prior day's real-time view, ga.us.ga_realtime_sessions_view_20181105, still exists before yesterday's table is created.
Is there a way to query for both yesterday's table and yesterday's real-time view, and handle the table not found error if yesterday's table hasn't been written yet?
Something like this:
SELECT
date,
SUM(totals.visits) AS visits
FROM
-- If this table doesn't exist yet
TABLE_DATE_RANGE([ga:us.ga_sessions_],
DATE_ADD(DATE_ADD(CURRENT_TIMESTAMP(), -6, 'HOUR'), -1, 'DAY'),
DATE_ADD(DATE_ADD(CURRENT_TIMESTAMP(), -6, 'HOUR'), -1, 'DAY')),
-- Query this view instead
TABLE_DATE_RANGE([ga:us.ga_realtime_sessions_view_],
DATE_ADD(DATE_ADD(CURRENT_TIMESTAMP(), -6, 'HOUR'), -1, 'DAY'),
DATE_ADD(DATE_ADD(CURRENT_TIMESTAMP(), -6, 'HOUR'), -1, 'DAY'))
WHERE
TIME(DATE_ADD(SEC_TO_TIMESTAMP(visitStartTime), -6, 'HOUR')) <= TIME(DATE_ADD(CURRENT_TIMESTAMP(), -6, 'HOUR'))
GROUP BY
date;
SOLUTION:
Thanks to #Bobbylank, I'm just selecting both tables in the same query:
SELECT
date,
SUM(totals.visits) AS visits
FROM
TABLE_DATE_RANGE([ga:us.ga_sessions_],
DATE_ADD(DATE_ADD(CURRENT_TIMESTAMP(), -6, 'HOUR'), -1, 'DAY'),
DATE_ADD(DATE_ADD(CURRENT_TIMESTAMP(), -6, 'HOUR'), -1, 'DAY')),
TABLE_DATE_RANGE([ga:us.ga_realtime_sessions_view_],
DATE_ADD(DATE_ADD(CURRENT_TIMESTAMP(), -6, 'HOUR'), -1, 'DAY'),
DATE_ADD(DATE_ADD(CURRENT_TIMESTAMP(), -6, 'HOUR'), -1, 'DAY'))
WHERE
TIME(DATE_ADD(SEC_TO_TIMESTAMP(visitStartTime), -6, 'HOUR')) <= TIME(DATE_ADD(CURRENT_TIMESTAMP(), -6, 'HOUR'))
GROUP BY
date;

I seem to remember having this problem and getting around it by selecting the ga_sessions for 2 days before and the previous day and the realtime table for the previous and current day.
That ensured the query wouldn't error as there should always be at least one table available for each day.
I then just selected the rows for the previous day from the resulting table.

Related

I inserted rows with the UTC date into an int column in my database. How can I get just the rows that were inserted in the last 24 hours?

Here's the insert that I used:
db2.Insert(new QuizHistory()
{
QuizId = quiz,
Cards = 0,
Points = points,
UtcNow = (int)Math.Truncate(DateTime.UtcNow.Subtract(new DateTime(1970, 1, 1)).TotalSeconds),
Viewed = 1,
Deck = deck
});
I tried looking at the different sql functions but now I am more confused than ever.
select * QuizHistory << but just for the last 24 hours.
As you are storing the date as seconds since january 1, 1970, a solution would be to use strftime :
select *
from QuizHistory
where UtcNow > strftime('%s', 'now', '-1 day')
i.e. with %s as format (seconds since 1970-01-01), for the now date with a -1 day modifier

Get days until future event in SQLite

Trying to get several things from a SQLite table with names and dates of birth and am having trouble getting the # of days until a person's next birthday. Dates are stored as SQLite's TEXT data type in format '%Y-%m-%d 00:00:00'.
I can get age:
SELECT ((strftime('%s', 'now') - strftime('%s', dob)) / 31536000) AS age
I like this solution for showing the closest birthdays first:
ORDER BY SUBSTR(date('now'), 6) > SUBSTR(dob, 6), SUBSTR(dob, 6) ASC
But I'm breaking my brain over getting the days until the next birthday. My latest attempt is taking the julianday of the substring of the day and month from dob and concatenate it with the current year to compare against julianday() and put in conditionals to take the year change into account, but I haven't worked that out yet and I'm hoping someone has a more elegant solution.
Have made my hideous solution work, so here it is:
SELECT
CASE WHEN
julianday((SUBSTR(date('now'), 1, 5) || SUBSTR(dob, 6, 5))) > julianday('now')
THEN CAST(ROUND(
julianday((SUBSTR(date('now'), 1, 5) || SUBSTR(dob, 6, 5))) - julianday('now'), 0) AS INTEGER)
ELSE CAST(ROUND((
julianday(SUBSTR(date('now'), 1, 5) || '12-31') - julianday('now')) + (
julianday(SUBSTR(date('now'), 1, 5) || SUBSTR(dob, 6, 5)) - julianday(SUBSTR(date('now'), 1, 5) || '01-01')), 0) AS INTEGER)
END
AS dub FROM person;
Will only have to put in another conditional to improve the rounding.

How to count days between date range with a specifific day?

How to count days between date range with a specific day?
Example:
START_DT = January 1, 2014;
END_DT = January 31, 2014;
Day = :SampleDay
Sample Result:
Monday = 4,
Tuesday = 4,
Wednesday = 5
Please help. :|
Are you looking for something like this,
WITH t(date1, date2) AS
(
SELECT to_date('01/01/2014', 'dd/mm/yyyy'),
to_date('31/01/2014','dd/mm/yyyy')+1 -- Adding 1 to calculate the last day too.
FROM DUAL
)
SELECT count(days) day_count, day
DAY
FROM(
SELECT date1 + LEVEL -1 days,
to_char(date1 + LEVEL -1, 'FmDay') DAY, --Use `FmDay`, this will remove the Embedded spaces.
to_char(date1 + LEVEL -1, 'D') DAY#
FROM t
CONNECT BY LEVEL <= date2 - date1
)
WHERE day = 'Monday' --Filter with day, if you want to get the count for a specific day.
GROUP BY DAY, day#
ORDER BY day#;
You wont have a direct solution to this. In oracle you have this form to know what day of the week is a specific date:
to_char(to_date('01012014', 'ddmmyyyy'), 'Day')
I would recommend to you to make a store procedure with a simple algorithm which receive that three parameters and then display the information you need. Put it in a query and it is done.

converting datetime format to time with only hour and minute

I have values in datetime format. ex: 2012-10-17 00:05:00.000
I casted them into time format ex: 00:05:00.000 but it is still too long for me. I just want hour and minute part of that time. Does anyone know?
Try this:
var dateTime = new DateTime(2012, 10, 17, 0, 5, 0, 0);
var hoursAndMinutes = dateTime.ToShortTimeString();

How to select records from db according to date range?

I have a "LOGTS" columns with datetime data type in a table, i would like to select my records which is within the range of "Last Week", "2 Weeks Ago" and etc! How do i do this?
SELECT * from table WHERE LOGTS > '2009-01-01';
works. Extrapolate. Also you can cast it:
SELECT CAST('1900-01-04 00:00' AS datetime);
I'm assuming SQL Server here with T-SQL. They have functions for the doing stuff like getting the current time and adding negative 7 days.
SELECT * from table WHERE logts > DATEADD(day, -7, CURRENT_TIMESTAMP);
Furthermore, it helps if you index your LOGTS column.
And, when you compare the execution plan of the above query to the one below, I believe you'll find the above one simpler and quicker.
SELECT * FROM aTable WHERE DATEDIFF(week, LOGTS, GETDATE()) <= 2
It appears that you want data from x past weeks until today. If that is the case this should work:
SELECT * FROM aTable WHERE DATEDIFF(week, LOGTS, GETDATE()) <= 2
Here I'm selecting data from 2 weeks ago until now.
The obvious DATEADD and DATEDIFF answers often forget about the time component, which needs to match your definition of when the day or week starts (e.g. midnight Sunday or 8am Monday), otherwise rows from earlier in the first day will be missed (giving different results when run at, say, 9am and 2pm).
These rounding calculations will find various points in time:
SET NOCOUNT ON;
DECLARE #pointInTime datetime;
SET #pointInTime= GETDATE();
SELECT
##DATEFIRST AS FirstDayOfWeek,
DATEADD(hour, DATEDIFF(hour, 0, #pointInTime), 0) AS StartOfHour,
DATEADD(day, DATEDIFF(day, 0, #pointInTime), 0) AS StartOfDay,
DATEADD(day, DATEDIFF(day, 1, #pointInTime), 0) AS Yesterday,
DATEADD(day, DATEDIFF(day, -1, #pointInTime), 0) AS Tomorrow,
DATEADD(day, 1 - DATEPART(weekday, #pointInTime), DATEADD(day,
DATEDIFF(day, 0, #pointInTime), 0)) AS StartOfThisWeek,
DATEADD(day, 1 - DATEPART(weekday, #pointInTime), DATEADD(day,
DATEDIFF(day, 7, #pointInTime), 0)) AS StartOfPreviousWeek,
DATEADD(month, DATEDIFF(month, 0, #pointInTime), 0) AS StartOfMonth,
DATEADD(quarter, DATEDIFF(quarter, 0, #pointInTime), 0) AS StartOfQuarter,
DATEADD(year, DATEDIFF(year, 0, #pointInTime), 0) AS StartOfYear,
DATEADD(hour, 8, DATEADD(day, DATEDIFF(day, 0,
#pointInTime), 0)) AS StartOfBusinessDay_8am,
DATEADD(month, 6, DATEADD(year, DATEDIFF(year, 0,
#pointInTime), 0)) AS StartOfFiscalYear_July1st;
So for your needs your comparison for "This Week", assuming the week starts at midnight Sunday, is:
SELECT * FROM MyTable WHERE
(LOGTS >= DATEADD(day, 1 - DATEPART(weekday, GETDATE()),
DATEADD(day, DATEDIFF(day, 0, GETDATE()), 0))
This also assumes LOGTS uses local time; if using GMT/UTC change GETDATE() to GETUTCDATE().
Personally, I'd assign the date calculations to variables and use them directly in the comparisons; looks a bit cleaner.
Also, any datetime range should be compared using a half-open range to avoid fractional second rounding issues; that is, greater or equal the start/older time and less than (NOT less or equal) the end/newer time. BETWEEN doesn't work here.
So for the records in the range "Last Week", using the named values above:
SELECT ... WHERE (LOGTS >= StartOfPreviouWeek) AND (LOGTS < StartOfThisWeek);
And so on.
dlamblin's answer is a good start. Most databases have a getdate() function and a dateadd() function that you can use to create the offsets you're looking for. Store the result of getdate() in a variable, then use dateadd() to create your range, relative to the current date.
In your query, you can use the "between" operator, or simple greater-than/less-than operators to query the data.
SELECT * FROM myData
where LOGTS > dateadd(day,-7,current_date())
That will do 7 days in the past from today's date.
Change dateadd(day,-7,current_date()) syntax to whatever the sysdate minute 7 days syntax is in your databse flavour.

Resources