I am trying to run some analysis on sales data using SQLite.
At the moment, my table has several columns including a unique transaction ID, product name, quantity of that product and value of that product. For each transaction, there can be several records, because each distinct type of product in the basket has its own entry.
I would like to add two new columns to the table. The first one would be a total for each transaction ID which summed up the total quantity of all products in that basket.
I realize that there would be duplication in the table, as the repeated transaction IDs would all have the total. The second one would be similar but in value terms.
I unfortunately cannot do this by creating a new table with the values I want calculated in Excel, and then joining it to the original table, because there are too many records for Excel.
Is there a way to get SQL to do the equivalent of a sumif in Excel?
I was thinking something along the lines of:
select sum(qty) where uniqID = ...
But I am stumped by how to express that it needs to sum all quantities where the uniqID is the same as the one in that record.
You wouldn't create a column like that in SQL. You would simply query for the total on the fly. If you really wanted a table-like object, you could create a view that held 2 columns; uniqID and the sum for that ID.
Let's set up some dummy data in a table; column a is your uniqID, b is the values you're summing.
create table tab1 (a int, b int);
insert into tab1 values (1,1);
insert into tab1 values (1,2);
insert into tab1 values (2,10);
insert into tab1 values (2,20);
Now you can do simple queries for individual uniqIDs like this:
select sum(b) from tab1 where a = 2;
30
Or sum for all uniqIDs (the 'group by' clause might be all you're groping for:) :
select a, sum(b) from tab1 group by a;
1|3
2|30
Which could be wrapped as a view:
create view totals as select a, sum(b) from tab1 group by a;
select * from totals;
1|3
2|30
The view will update on the fly:
insert into tab1 values (2,30);
select * from totals;
1|3
2|60
In further queries, for analysis, you can use 'totals' just like you would a table.
Related
I have a table with columns item, store, date, and fcst. For each item store day I need to sum the next x number of days of forecast, however x changes for each item store combination. This following code does not work and I was advised that X FOLLOWING has to be replaced with a static innteger FOLLOWING:
Doesn’t run:
SELECT
ITEMNBR,
STORENBR,
DT,
SUM(FCST)OVER(PARTITION BY STORENBR,ITEMNBR ORDER BY DT BETWEEN CURRENT ROW AND X ROWS FOLLOWING) AS VALUE
FROM TABLENAME
Does run:
SELECT
ITEMNBR,
STORENBR,
DT,
SUM(FCST)OVER(PARTITION BY STORENBR,ITEMNBR ORDER BY DT BETWEEN CURRENT ROW AND 7 ROWS FOLLOWING) AS VALUE
FROM TABLENAME
Any fix or suggested workaround?
I have created a subset of the pg_table_def table with table_name,col_name and data_type. I have also added a column active with 'Y' as value for some of the rows. Let us call this table as config.Table config looks like below:
table_name column_name
interaction_summary name_id
tag_transaction name_id
interaction_summary direct_preference
bulk_sent email_image_click
crm_dm web_le_click
Now I want to be able to map the table names from this table to the actual table and fetch values for the corresponding column. name_id will be the key here which will be available in all tables. My output should look like below:
name_id direct_preference email_image_click web_le_click
1 Y 1 2
2 N 1 2
The solution needs to be dynamic so that even if the table list extends tomorrow, the new table should be able to accommodate. Since I am new to Redshift, any help is appreciated. I am also considering to do the same via R using the dplyr package.
I understood that dynamic queries don't work with Redshift.
My objective was to pull any new table that comes in and use their columns for regression analysis in R.
I made this working by using listagg feature and concat operation. And then wrote the output to a dataframe in R. This dataframe would have 'n' number of select queries as different rows.
Below is the format:
df <- as.data.frame(tbl(conn,sql("select 'select ' || col_names|| ' from ' || table_name as q1 from ( select distinct table_name, listagg(col_name,',') within group (order by col_name)
over (partition by table_name) as col_names
from attribute_config
where active = 'Y'
order by table_name )
group by 1")))
Once done, I assigned every row of this dataframe to a new dataframe and fetched the output using below:
df1 <- tbl(conn,sql(df[1,]))
I know this is a round about solution. But it works !! Fetches about 17M records under 1 second.
Super new to SQLite but I thought it can't hurt to ask.
I have something like the following table (Not allowed to post images yet) pulling data from multiple tables to calculate the TotalScore:
Name TotalScore
Course1 15
Course1 12
Course2 9
Course2 10
How the heck do I SELECT only the max value for each course? I've managed use
ORDER BY TotalScore LIMIT 2
But I may end up with multiple Courses in my final product, so LIMIT 2 etc won't really help me.
Thoughts? Happy to put up the rest of my query if it helps?
You can GROUP the resultset by Name and then use the aggregate function MAX():
SELECT Name, max(TotalScore)
FROM my_table
GROUP BY Name
You will get one row for each distinct course, with the name in column 1 and the maximum TotalScore for this course in column 2.
Further hints
You can only SELECT columns that are either grouped by (Name) or wrapped in aggregate functions (max(TotalScore)). If you need another column (e.g. Description) in the resultset, you can group by more than one column:
...
GROUP BY Name, Description
To filter the resulting rows further, you need to use HAVING instead of WHERE:
SELECT Name, max(TotalScore)
FROM my_table
-- WHERE clause would be here
GROUP BY Name
HAVING max(TotalScore) > 5
WHERE filters the raw table rows, HAVING filters the resulting grouped rows.
Functions like max and sum are "aggregate functions" meaning they aggregate multiple rows together. Normally they aggregate them into one value, like max(totalscore) but you can aggregate them into multiple values with group by. group by says how to group the rows together into aggregates.
select name, max(totalscore)
from scores
group by name;
This groups all the columns together with the same name and then does a max(totalscore) for each name.
sqlite> select name, max(totalscore) from scores group by name;
Course1|15
Course2|12
Lets pretend that we have a supermarket.
We got a table called Sales where every record is one article), so if we scan 3 articles we will have 3 rows with following columns: ArticleId and Amount where amount Always is 1.
And then we have a table called Articles which have columns: ArticleId and AvailableAmount.
When the sale is done we need to Count records that are the same in Sales table and then update AvailableAmount with AvailableAmount subtracted with the sum of each article.
I'm thinking something like this but i dont know if im thinking right:
UPDATE Articles
SET
AvailableAmount = AvailableAmount - (
Select ArticleId,Count(*) From Sales Group by ArticleId HAVING Count(*) > 1
)
WHERE
ArticleId in(Select distinct ArticleId FROM Sales)
This query is almost correct, but
the subquery must return only one column,
HAVING Count(*) > 1 does not make sense, and
the subquery must return only one value, so you need a correlated subquery:
UPDATE Articles
SET AvailableAmount = AvailableAmount -
(SELECT COUNT(*)
FROM Sales
WHERE ArticleId = Articles.ArticleId)
WHERE ArticleId IN (SELECT ArticleId
FROM Sales)
Sqlite doesn't have a row number function. My database however could have several thousands of records. I need to sort a table based upon a date (the date field is actually an INTEGER) and then return a specific range of rows. So if I wanted all the rows from 600 to 800, I need to somehow create a row number and limit the results to fall within my desired range. I cannot use RowID or any auto-incremented ID field because all the data is inserted with random dates. The closest I can get is this:
CREATE TABLE Test (ID INTEGER, Name TEXT, DateRecorded INTEGER);
Insert Into Test (ID, Name, DateRecorded) Values (5,'fox', 400);
Insert Into Test (ID, Name, DateRecorded) Values (1,'rabbit', 100);
Insert Into Test (ID, Name, DateRecorded) Values (10,'ant', 800);
Insert Into Test (ID, Name, DateRecorded) Values (8,'deer', 300);
Insert Into Test (ID, Name, DateRecorded) Values (6,'bear', 200);
SELECT ID,
Name,
DateRecorded,
(SELECT COUNT(*)
FROM Test AS t2
WHERE t2.DateRecorded > t1.DateRecorded) AS RowNum
FROM Test AS t1
where RowNum > 2
ORDER BY DateRecorded Desc;
This will work except it's really ugly. The Select Count(*) will result in carrying out that Select statement for every row encountered. So if I have several thousands of rows, that will be a very poor performance.
This is what the LIMIT/OFFSET clauses are for:
SELECT *
FROM Test
ORDER BY DateRecorded DESC
LIMIT 200 OFFSET 600