I have an Sqlite table as below called balance history:
Table Balance History
Date Amount
2013-11-01 16:26:52 1000
2013-11-15 13:20:52 2000
2013-11-27 12:26:55 3000
I would like to calculate the average for the month.
**
The Expected OutPut will be 1666.67
**
Which will be (1000 * 14 days + 2000 * 12 days + 3000 * 4 days)/30 days
= (14000 + 24000 + 12000)/30 = 1666.67
How can I achieve this in SQlite? any help will be appreciated.
thanks
First, we have to compute the start and the end of each interval.
The simple query (SELECT Date AS "From", Amount FROM BalanceHistory WHERE Date GLOB '2013-11*') gets the start for each interval in the month.
(GLOB is case sensitive and thus allows to use a normal index; LIKE would require a special case-insensitive index.)
If there is no record for the first day of the month, the part after the UNION ALL adds the last record of the previous month and changes the day to the 1st.
The COALESCE computes the end of the interval.
The subquery gets the next date from the table, if there is one in the current month.
If there is no such record, it takes the first day of the next month:
SELECT date("From") AS "From",
COALESCE((SELECT date(MIN(Date))
FROM BalanceHistory
WHERE Date > "From"
AND Date GLOB '2013-11*'),
date('2013-11-01', '+1 month')
) AS "To",
"Amount"
FROM (SELECT Date AS "From",
Amount
FROM BalanceHistory
WHERE Date GLOB '2013-11*'
UNION ALL
SELECT *
FROM (SELECT date(Date, '+1 month', 'start of month'),
Amount
FROM BalanceHistory
WHERE Date < '2013-11'
AND NOT EXISTS (SELECT 1
FROM BalanceHistory
WHERE Date GLOB '2013-11-01*')
ORDER BY Date DESC
LIMIT 1)
);
From To Amount
---------- ---------- ------
2013-11-01 2013-11-15 1000
2013-11-15 2013-11-27 2000
2013-11-27 2013-12-01 3000
We can then wrap this in another query to compute the number of days, and add them up.
The strftime calculates the last day of the month, i.e., the number of days:
SELECT SUM((julianday("To") - julianday("From")) * Amount) /
strftime('%d', '2013-11-01', '+1 month', '-1 day') AS MonthAvg
FROM (SELECT date("From") AS "From",
COALESCE((SELECT date(MIN(Date))
FROM BalanceHistory
WHERE Date > "From"
AND Date GLOB '2013-11*'),
date('2013-11-01', '+1 month')
) AS "To",
"Amount"
FROM (SELECT Date AS "From",
Amount
FROM BalanceHistory
WHERE Date GLOB '2013-11*'
UNION ALL
SELECT *
FROM (SELECT date(Date, '+1 month', 'start of month'),
Amount
FROM BalanceHistory
WHERE Date < '2013-11'
AND NOT EXISTS (SELECT 1
FROM BalanceHistory
WHERE Date GLOB '2013-11-01*')
ORDER BY Date DESC
LIMIT 1)
)
)
And while we're at it, we can wrap this in yet another query to replace all those 2013-11 strings with the month as read from the table.
This allows as to compute this for every month:
SELECT Month,
(SELECT SUM((julianday("To") - julianday("From")) * Amount) /
strftime('%d', Month || '-01', '+1 month', '-1 day')
FROM (SELECT date("From") AS "From",
COALESCE((SELECT date(MIN(Date))
FROM BalanceHistory
WHERE Date > "From"
AND Date GLOB Month || '*'),
date(Month || '-01', '+1 month')
) AS "To",
"Amount"
FROM (SELECT Date AS "From",
Amount
FROM BalanceHistory
WHERE Date GLOB Month || '*'
UNION ALL
SELECT *
FROM (SELECT date(Date, '+1 month', 'start of month'),
Amount
FROM BalanceHistory
WHERE Date < Month
AND NOT EXISTS (SELECT 1
FROM BalanceHistory
WHERE Date GLOB Month || '-01*')
ORDER BY Date DESC
LIMIT 1)
)
)
) AS MonthAvg
FROM (SELECT DISTINCT strftime('%Y-%m', Date) AS Month
FROM BalanceHistory)
ORDER BY 1
Related
My calendar year runs from 07-01-(of one year) to 06-30-(of the next year).
My SQLITE DB has a Timestamp column and it's data type is datetime and stores the timestamp as 2023-09-01 00:00:00.
What I'm trying to do is get the MAX date of the latest snowfall. For example, with my seasonal years beginning July-01 (earliest) and ending June 30 (latest), I want to find only the latest (MAX) date snowfall was recorded, regardless of the year, based on the month.
Say if out of five years (2017 to 2022) worth of data in the database and it snowed Mar 15, 2020. And there was no date greater than than this one in any year, then this would be the latest date regardless which year it fell.
I've been trying many variations of the below query. This query says it runs with no mistakes and returns "null" values. I'm using SQLITE DB Browser to write and test the query.
SELECT Timestamp, MAX(strftime('%m-%d-%Y', Timestamp)) AS lastDate,
snowDepth AS lastDepth FROM DiaryData
WHERE lastDepth <> 0 BETWEEN strftime('%Y-%m-%d', Timestamp,'start of year', '+7 months')
AND strftime('%Y-%m-%d', Timestamp, 'start of year', '+1 year', '+7 months', '- 1 day')
ORDER BY lastDate LIMIT 1
and this is what's in my test database:
Timestamp snowFalling snowLaying snowDepth
2021-11-10 00:00:00 0 0 7.2
2022-09-15 00:00:00 0 0 9.5
2022-12-01 00:00:00 1 0 2.15
2022-10-13 00:00:00 1 0 0.0
2022-05-19 00:00:00 0 0 8.82
2023-01-11 00:00:00 0 0 3.77
If it's running properly I should expect:
Timestamp
lastDate
lastDepth
2022-05-19 00:00:00
05-19-2022
8.82
What am I missing or is this not possible in SQLITE? Any help would be appreciative.
Use aggregation by fiscal year utilizing SQLite's feature of bare columns:
SELECT Timestamp,
strftime('%m-%d-%Y', MAX(Timestamp)) AS lastDate,
snowDepth AS lastDepth
FROM DiaryData
WHERE snowDepth <> 0
GROUP BY strftime('%Y', Timestamp, '+6 months');
See the demo.
I'd get season for each record first, snowfall date relative to record's season start date after this, and largest snowfall date relative to record's season start date finally:
with
data as (
select
*
, case
when cast(strftime('%m', "Timestamp") as int) <= 7
then strftime('%Y-%m-%d', "Timestamp", 'start of year', '-1 year', '+6 months')
else strftime('%Y-%m-%d', "Timestamp", 'start of year', '+6 months')
end as "Season start date"
from DiaryData
where 1==1
and "snowDepth" <> 0.0
)
, data2 as (
select
*
, julianday("Timestamp") - julianday("Season start date")
as "Showfall date relative to season start date"
from data
)
, data3 as (
select
"Timestamp"
, "snowFalling"
, "snowLaying"
, "snowDepth"
from data2
group by null
having max("Showfall date relative to season start date")
)
select
*
from data3
demo
You can use the ROW_NUMBER window function to address this problem, yet need to apply a subtle tweak. In order to account for fiscal years, you can partition on the year for timestamps slided 6 months further. In this way, ranges like [2021-01-01, 2021-12-31] will instead be slided to [2021-06-01, 2022-05-31].
WITH cte AS (
SELECT *, ROW_NUMBER() OVER(
PARTITION BY STRFTIME('%Y', DATE(Timestamp_, '+6 months'))
ORDER BY Timestamp_ DESC ) AS rn
FROM tab
)
SELECT Timestamp_,
STRFTIME('%d-%m-%Y', Timestamp_) AS lastDate,
snowDepth AS lastDepth
FROM cte
WHERE rn = 1
Check the demo here.
I have an API which pulls time series data for different categories. For some categories it only pulls weekday data.
I would like to fill in the Saturday/Sunday data with the preceding Friday. I also need to backfill for public holidays too i.e. a Monday bank holiday which is not in the original time series will also be filled with the preceding Friday. See table below for illustration. I have assumed 8/8/2022 is bank holiday for illustrative purposes.
I have used the answer from Insert records for each calendar date in SQLite table to perform this on a single time series set. I would like to know how I can apply this backfill logic for each respective categories time series. i.e. category 1 uses category 1 friday data, category 2 uses category 2 friday data etc.
I am using sqlite version 3.18.
Before (current data)
After (desired result)
weekends added in green, bank holidays added in grey
Code that works on Single Category
From Insert records for each calendar date in SQLite table. Can the code be adjusted to work on multiple categories to achieve desired result above?
CREATE TABLE BALANCES (Category varchar(25), BalanceDate DATE, Amount INT)
INSERT INTO BALANCES VALUES
('cat 1',date('2022-07-25'),101),
('cat 1',date('2022-07-26'),102),
('cat 1',date('2022-07-27'),103),
('cat 1',date('2022-07-28'),104),
('cat 1',date('2022-07-29'),105),
('cat 1',date('2022-08-01'),106),
('cat 1',date('2022-08-02'),107),
('cat 1',date('2022-08-03'),108),
('cat 1',date('2022-08-04'),109),
('cat 1',date('2022-08-05'),110)
WITH cte AS (
SELECT b1.Category, date(b1.BalanceDate, '+1 day') BalanceDate, b1.Amount FROM BALANCES b1
WHERE NOT EXISTS (SELECT 1 FROM BALANCES b2 WHERE b2.BalanceDate = date(b1.BalanceDate, '+1 day')) AND date(b1.BalanceDate, '+1 day') <= DATE('now')
UNION ALL
SELECT Category, date(c.BalanceDate, '+1 day'), c.Amount FROM cte c
WHERE NOT EXISTS (SELECT 1 FROM BALANCES b WHERE b.BalanceDate = date(c.BalanceDate, '+1 day')) AND date(c.BalanceDate, '+1 day') <= DATE('now')
)
INSERT INTO BALANCES(Category,BalanceDate, Amount)
SELECT Category, BalanceDate, Amount FROM cte;
You must add the condition for Category in the WHERE clauses:
WITH cte AS (
SELECT b1.Category, date(b1.BalanceDate, '+1 day') BalanceDate, b1.Amount FROM BALANCES b1
WHERE NOT EXISTS (SELECT 1 FROM BALANCES b2 WHERE b2.Category = b1.Category AND b2.BalanceDate = date(b1.BalanceDate, '+1 day'))
AND date(b1.BalanceDate, '+1 day') <= DATE('now')
UNION ALL
SELECT Category, date(c.BalanceDate, '+1 day'), c.Amount FROM cte c
WHERE NOT EXISTS (SELECT 1 FROM BALANCES b WHERE b.Category = c.Category AND b.BalanceDate = date(c.BalanceDate, '+1 day'))
AND date(c.BalanceDate, '+1 day') <= DATE('now')
)
INSERT INTO BALANCES(Category,BalanceDate, Amount)
SELECT Category, BalanceDate, Amount FROM cte;
See the demo.
I've got a table in an SQLite3 database containing account balances, but it currently only contains balances for a few specific dates:
Balance Date
Amount
2021-12-15
400
2021-12-18
500
2021-12-22
200
I need to fill in the gaps between these dates with the previous recorded balance, so e.g. 2021-12-16 and 2021-12-17 should have a balance of 400 and 2021-12-19, 2021-12-20 and 2021-12-21 should have a balance of 500.
Is there a way to fill these gaps using SQL? I think I need some logic like
INSERT INTO BALANCES (BalanceDate,BalanceAmount)
VALUES(previous record + 1 day, previous record's amount)
but I don't know how I can point SQL to the previous record.
Thanks
You can use a recursive cte to produce the missing dates:
WITH cte AS (
SELECT date(b1.BalanceDate, '+1 day') BalanceDate, b1.Amount
FROM BALANCES b1
WHERE NOT EXISTS (SELECT 1 FROM BALANCES b2 WHERE b2.BalanceDate = date(b1.BalanceDate, '+1 day'))
AND date(b1.BalanceDate, '+1 day') < (SELECT MAX(BalanceDate) FROM BALANCES)
UNION ALL
SELECT date(c.BalanceDate, '+1 day'), c.Amount
FROM cte c
WHERE NOT EXISTS (SELECT 1 FROM BALANCES b WHERE b.BalanceDate = date(c.BalanceDate, '+1 day'))
AND date(c.BalanceDate, '+1 day') < (SELECT MAX(BalanceDate) FROM BALANCES)
)
INSERT INTO BALANCES(BalanceDate, Amount)
SELECT BalanceDate, Amount FROM cte;
See the demo.
I want to query last 24h in my database SQLite. I have 2 rows time example (18:04:56) and date (2020-12-06 ). When I use this query
SELECT name,cdate,ctime, * FROM {table} WHERE cdate >= datetime('now','-1 day') ORDER BY id DESC
I got only last queries in the same day. (LAST QUERY 00:00:46)
How deal with it ?
You should create a datetime value from cdate and ctime to compare with DATETIME('now','-1 day'):
SELECT name, cdate, ctime
FROM {table}
WHERE (cdate || ' ' || ctime) >= DATETIME('now','-1 day')
ORDER BY id DESC
I want to calculate first date and last date of a week for the current month given current datetime.
For example, 01 December 2017 is first day and 02 December 2017 is last day of week for that week for the month of December. In the same week of year, 26 November 2017 is first day of week and
30 November 2017 is last day of week for previous month.
So if today is 01 December 2017, I should get 01-02 from current datetime , instead of 26-02 range.
PS: I am trying to do this thing in a query of Sqlite. I want to group some values on a weekly basis. The days of the week should belong to current month. Hence this requirement. Till now this is my query.
SELECT SUM(amount) as Y,
strftime('%d', datetime(createdOn/1000, 'unixepoch'), '-'||strftime('%w', datetime(createdOn/1000, 'unixepoch'))||' day' )
||'-'||
strftime('%d', datetime(createdOn/1000, 'unixepoch'), '+'||(6-strftime('%w', datetime(createdOn/1000, 'unixepoch')))||' day' ) AS X
FROM my_table
GROUP BY strftime('%d', datetime(createdOn/1000, 'unixepoch'), '-'||strftime('%w', datetime(createdOn/1000, 'unixepoch'))||' day' )
||'-'||
strftime('%d', datetime(createdOn/1000, 'unixepoch'), '+'||(6-strftime('%w', datetime(createdOn/1000, 'unixepoch')))||' day' )
How to achieve that; given current datetime.
Common table expressions are useful to hold intermediate results:
WITH t1 AS (
SELECT amount, date(createdOn / 1000, 'unixepoch', 'localtime') AS date
FROM my_table
),
t2 AS (
SELECT *,
date(date, 'weekday 6') AS end_of_week,
date(date, 'weekday 6', '-6 days') AS start_of_week
FROM t1
),
t3 AS (
SELECT *,
strftime('%m', date ) AS month,
strftime('%m', start_of_week) AS sowk_month,
strftime('%m', end_of_week ) AS eowk_month,
date(date, 'start of month') AS start_of_month,
date(date, '+1 month', 'start of month', '-1 day') AS end_of_month
FROM t2
),
t4 AS (
SELECT *,
CASE WHEN sowk_month = month THEN start_of_week
ELSE start_of_month
END AS week_in_month_start,
CASE WHEN eowk_month = month THEN end_of_week
ELSE end_of_month
END AS week_in_month_end
FROM t3
),
t5 AS (
SELECT amount,
week_in_month_start || '-' || week_in_month_end AS week_in_month
FROM t4
)
SELECT SUM(amount) AS Y,
week_in_month
FROM t5
GROUP BY week_in_month;