So let's say I have an FTS[34] table data, and a regular table info, that are bound together by a info.ID = data.rowid. I have a view master that represents this join, that I use to query overall information.
Here are the CREATE TABLE statements:
CREATE VIRTUAL TABLE data USING fts4(
Name,
Keywords,
Aliases
Description);
CREATE TABLE info (
ID INTEGER PRIMARY KEY,
-- A bunch of other columns...
);
CREATE VIEW master AS SELECT
info.ID AS ID
data.Name AS Name,
data.Keywords AS Keywords,
data.Aliases AS Aliases,
data.Description AS Description,
-- A bunch of other columns...
FROM info JOIN data ON info.ID = data.rowid;
Now what I want to do is create another view that instead of selecting the entire cell from the data table, selects the result of the SQLite snippet function as performed on each column of the data table:
CREATE VIEW search AS SELECT
master.*,
snippet(data '<b>', '</b>', '...', 0) AS sn_Name,
snippet(data '<b>', '</b>', '...', 1) AS sn_Keywords,
snippet(data '<b>', '</b>', '...', 2) AS sn_Aliases,
snippet(data '<b>', '</b>', '...', 3) AS sn_Description
FROM master JOIN data ON master.ID = data.rowid;
However, when I execute that statement from within Python I get the following:
sqlite3.OperationalError: near "'<b>'": syntax error
The comma between the data and '<b>' parameter values is missing.
Related
Im trying to write a recursive query for a use on a old and poorly designed database - and so the queries get quite complex.
Here is the (relevant) table relationships
Because people asked - here is the creation code for these tables:
CREATE TABLE CircuitLayout(
CircuitLayoutID int,
PRIMARY KEY (CircuitLayoutID)
);
CREATE TABLE LitCircuit (
LitCircuitID int,
CircuitLayoutID int,
PRIMARY KEY (LitCircuitID)
FOREIGN KEY (CircuitLayoutID) REFERENCES CircuitLayout(CircuitLayoutID)
);
CREATE TABLE CircuitLayoutItem(
CircuitLayoutItemID int,
CircuitLayoutID int,
TableName varchar(255),
TablePK int,
PRIMARY KEY (CircuitLayoutItemID)
FOREIGN KEY (CircuitLayoutID) REFERENCES CircuitLayout(CircuitLayoutID)
);
TableName refers to another table in the database and thus TablePK is a primary key from the specified table
One of the valid options for TableName is LitCircuit
I'm trying to write a query that will select a circuit and any circuit it is related to
I am having trouble understanding the syntax for recursive ctes
my non-functional attempt is this:
WITH RECURSIVE carries AS (
SELECT LitCircuit.LitCircuitID AS recurseList FROM LitCircuit
JOIN CircuitLayoutItem ON LitCircuit.CircuitLayoutID = CircuitLayoutItem.CircuitLayoutID
WHERE CircuitLayoutItem.TableName = "LitCircuit" AND CircuitLayoutItem.TablePK IN (00340)
UNION
SELECT LitCircuit.LitCircuitID AS CircuitIDs FROM LitCircuit
JOIN CircuitLayout ON LitCircuit.CircuitLayoutID = CircuitLayoutItem.CircuitLayoutID
WHERE CircuitLayoutItem.TableName = "LitCircuit" AND CircuitLayoutItem.TablePK IN (SELECT recurseList FROM carries)
)
SELECT * FROM carries;
the "00340" is a dummy number for testing, and it would get replaced with an actual list in usage
What i'm attempting to do is get a list of LitCircuitIDs based on one or many LitCircuitIDs - that's the anchor member, and that works fine.
What I want to do is take this result and feed it back into itself.
I lack an understanding of how to access data from the anchor member:
I don't know if it is a table with the columns from the select in the anchor or if it is simply a list of resulting values
I dont understand if or where I need to include "carries" in the FROM part of a query
If I were to write this function in python I would do it like this:
def get_circuits(circuit_list):
result_list = []
for layout_item_key, layout_item in CircuitLayoutItem.items():
if layout_item['TableName'] == "LitCircuit" and layout_item['TablePK'] in circuit_list:
layout = layout_item['CircuitLayoutID']
for circuit_key, circuit in LitCircuit.items():
if circuit["CircuitLayoutID"] == layout:
result_list.append(circuit_key)
result_list.extend(get_circuits(result_list))
return result_list
How do I express this in SQL?
danblack's comment made me realize something I was missing:
Here is what I was trying to do:
WITH RECURSIVE carries AS (
SELECT LitCircuit.LitCircuitID FROM LitCircuit
JOIN CircuitLayoutItem ON LitCircuit.CircuitLayoutID = CircuitLayoutItem.CircuitLayoutID
WHERE CircuitLayoutItem.TableName = 'LitCircuit' AND CircuitLayoutItem.TablePK IN (00340)
UNION ALL
SELECT LitCircuit.LitCircuitID FROM carries
JOIN CircuitLayoutItem ON carries.LitCircuitID = CircuitLayoutItem.TablePK
JOIN LitCircuit ON CircuitLayoutItem.CircuitLayoutID = LitCircuit.CircuitLayoutID
WHERE CircuitLayoutItem.TableName = 'LitCircuit'
)
SELECT DISTINCT LitCircuitID FROM carries;
I did not think of the CTE as a table to query against - rather just a result set, so I did not realize you have to SELECT from it - or in general treat it like a table.
I'm trying to add a test user to my website that employers can look at to see my work. I want to use some of the data I have entered into my profile so that it is faster.
I have a workouts table:
CREATE TABLE workouts(
id INTEGER NOT NULL,
userID INTEGER NOT NULL,
DateAndTime smalldatetime NOT NULL,
PRIMARY KEY (id),
FOREIGN KEY (UserID) REFERENCES users(id)
);
I have taken 25 of the first results and put it into a temporary workouts2 table:
CREATE TABLE workouts2 (
userid integer,
dateandtime smalldatetime);
Now I want to take those rows from workouts2 and put them into workouts. I have tried to add them by inserting workouts2 into workouts like this:
insert into workouts (id , userID, DateandTime) values (select * from workouts2);
This gives me an Error: in prepare, near "select": syntax error (1)
I can do it one at a time like this:
insert into workouts (userid, dateandtime) values (2, "2022-01-02T06:00");
Doing it one at a time is not ideal.
What am I missing here? I know I have a syntax error but I don't know how to fix it.
I have looked at this question which inserts one at a time:
How to insert a unique ID into each SQLite row?
The problem is it only inserts one at a time.
You should use SELECT instead of VALUES and not include the column id, which is auto-incremented, in the list of columns of workouts which will receive the values (such a column does not exist in workouts2):
INSERT INTO workouts (userID, DateandTime)
SELECT *
FROM workouts2;
I'm trying to create a trigger that classifies some news in my database by category after every insert on the table news if the attribute title contains "COVID-19" or "coronavirus" (then the category id should be the one with the name "COVID-19") or if the attribute title contains "Ukraine" or "Rusia" (then the category id should be the one with the name "War").
The tables have the following structure:
news (id, title, text, date, user(FK))
category (id, name, description)
new_cat(news(FK), category(FK))
And I tried implementing the following trigger without success:
CREATE TRIGGER news_categorizer AFTER INSERT ON news
FOR EACH ROW
BEGIN
INSERT INTO new_cat (news, category) VALUES (NEW.id,
(SELECT id from category WHERE name = "COVID-19" AND (NEW.title = "%COVID-19%" or NEW.title = "%coronavirus%")));
END;
How could I properly develop this subquery or can I do different INSERTs depending on a condition?
Use properly the operator LIKE in a CASE expression:
CREATE TRIGGER news_categorizer AFTER INSERT ON news
FOR EACH ROW
BEGIN
INSERT INTO new_cat (news, category)
SELECT NEW.id,
id
FROM category
WHERE name = CASE
WHEN (NEW.title LIKE '%COVID-19%') OR (NEW.title LIKE '%coronavirus%') THEN 'COVID-19'
WHEN (NEW.title LIKE '%Ukraine%') OR (NEW.title LIKE '%Rusia%') THEN 'War'
END;
END;
See a simplified demo.
I have a simple table in athena, it has an array of events. I want to write a simple select statement so that each event in array becomes a row.
I tried explode, transform, but no luck. I have successfully done it in Spark and Hive. But this Athena is tricking me. Please advise
DROP TABLE bi_data_lake.royalty_v4;
CREATE external TABLE bi_data_lake.royalty_v4 (
KAFKA_ID string,
KAFKA_TS string,
deviceUser struct< deviceName:string, devicePlatform:string >,
consumeReportingEvents array<
struct<
consumeEvent: string,
consumeEventAction: string,
entryDateTime: string
>
>
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION 's3://XXXXXXXXXXX';
Query which is not working
select kafka_id, kafka_ts,deviceuser,
transform( consumereportingevents, consumereportingevent -> consumereportingevent.consumeevent) as cre
from bi_data_lake.royalty_v4
where kafka_id = 'events-consumption-0-490565';
Not supported
lateral view explode(consumereportingevents) as consumereportingevent
Answer to question it to use unnset
Found the answer for my question
WITH samples AS (
select kafka_id, kafka_ts,deviceuser, consumereportingevent, consumereportingeventPos
from bi_data_lake.royalty_v4
cross join unnest(consumereportingevents) WITH ORDINALITY AS T (consumereportingevent, consumereportingeventPos)
where kafka_id = 'events-consumption-0-490565' or kafka_id = 'events-consumption-0-490566'
)
SELECT * FROM samples
Flatten ('explode') nested arrays in AWS Athena with UNNEST.
WITH dataset AS (
SELECT
'engineering' as department,
ARRAY['Sharon', 'John', 'Bob', 'Sally'] as users
)
SELECT department, names FROM dataset
CROSS JOIN UNNEST(users) as t(names)
Reference: Flattening Nested Arrays
I have a sqlite3 table with a column by the name Title, which stores the names of the some movies.
Table name - table1
Column name - Title
Examples data: "Casablanca" (1983) {The Cashier and the Belly Dancer (#1.4)}
I have another sqlite3 table with a column that stores movie titles.
Table name - table2
Column name - Title
Examples data: casa blanca
Both these tables were created using different datasets, and as such although the movie name is the same (casa blanca vs "Casablanca" (1983) {The Cashier and the Belly Dancer (#1.4)}), both are stored with extra text.
What i would like to do is to SANITIZE the already stored data in both the columns. By sanitization, I would like to strip the cell content of:
1. spaces
2. spl chars like !, ', ", comma, etc..
3. convert all to lower case
I hope with that atleast some level of matching can be had between both the columns.
My question is, how do i perform these sanitizations on data that is already stored in sqlite tables. I do not have an option to sanitize before loading, as i only have access to the loaded database.
I am using sqlite 3.7.13, and i am using sqlite manager as the gui.
Thank You.
This task is too specialized to be done in SQL only.
You should write simple Perl or Python script which will scan your table, read data row by row, scrub it to meet your requirements and write it back.
This is example in Perl:
use DBI;
my $dbh = DBI->connect("dbi:mysql:database=my.db");
# replace rowid with your primary key, but it should work as is:
my $sth = $dbh->prepare(qq{
SELECT rowid,*
FROM table1
});
while (my $row = $sth->fetchrow_hashref()) {
my $rowid = $row->{rowid};
my $title = $row->{title};
# sanitize title:
$title = lc($title); # convert to lowercase
$title =~ s/,//g; # remove commas
# do more sanitization as you wish
# ...
# write it back to database:
$dbh->do(
qq{
UPDATE table1
SET title = ?
WHERE rowid = ?
}, undef,
$title,
$rowid,
);
}
$sth->finish();
$dbh->disconnect();