I'd like to encrypt/decrypt column in BigQuery using AEAD functions, however decryption fails with generic message:
AEAD.DECRYPT_BYTES failed: decryption failed
Here is the sample SQL:
create CustomerData(customer_id, keyset, favorite_animal) table
create EncryptedCustomerData(customer_id, encrypted_animal) and encrypt data
select and decrypt data
WITH CustomerData AS (
SELECT 1 AS customer_id, KEYS.NEW_KEYSET('AEAD_AES_GCM_256') AS keyset, b'jaguar' AS favorite_animal UNION ALL
SELECT 2 AS customer_id, KEYS.NEW_KEYSET('AEAD_AES_GCM_256') AS keyset, b'zebra' AS favorite_animal UNION ALL
SELECT 3 AS customer_id, KEYS.NEW_KEYSET('AEAD_AES_GCM_256') AS keyset, b'nautilus' AS favorite_animal),
EncryptedCustomerData AS (
SELECT customer_id, AEAD.ENCRYPT(keyset, favorite_animal, CAST(CAST(customer_id AS STRING) AS BYTES)) AS encrypted_animal
FROM CustomerData AS cd)
SELECT
ecd.customer_id,
AEAD.DECRYPT_BYTES(
(SELECT cd.keyset FROM CustomerData AS cd WHERE ecd.customer_id = cd.customer_id),
ecd.encrypted_animal,
CAST(CAST(customer_id AS STRING) AS BYTES)
) AS favorite_animal
FROM EncryptedCustomerData AS ecd;
Could you point out the error?
Related
In Sqlite, I defined a view as a union all of two tables. When I run queries using that view, the index is used if the query is simple enough. For certain complex queries, it does not and ends up running full table scans. Are there ways around this so that I can use views performantly?
Table/View Definitions:
CREATE TABLE 'Table1' (Id varchar (18) PRIMARY KEY UNIQUE ON CONFLICT ROLLBACK, Name varchar (255) )
CREATE TABLE 'Table2' (Id varchar (18) PRIMARY KEY UNIQUE ON CONFLICT ROLLBACK, Name varchar (255) )
CREATE TABLE 'Table3' (Id varchar (18) PRIMARY KEY UNIQUE ON CONFLICT ROLLBACK, Name varchar (255) )
CREATE VIEW [UnionView] AS SELECT 'T1' tid, T1.rowid, T1.* FROM [Table1] T1 UNION ALL SELECT 'T2' tid, T2.rowid, T2.* FROM [Table2] T2
Simple Query (indexes are used):
SELECT Id FROM [UnionView] WHERE Id = 'asdf'
Explain Query Plan:
COMPOUND QUERY
LEFT-MOST SUBQUERY
SEARCH TABLE Table1 AS T1 USING INDEX sqlite_autoindex_Table1_1 (Id=?)
UNION ALL
SEARCH TABLE Table2 AS T2 USING INDEX sqlite_autoindex_Table2_1 (Id=?)
LEFT JOIN Query (indexes are not used):
SELECT T3.Id FROM [Table3] T3 LEFT JOIN [UnionView] T ON T3.Id=T.Id WHERE T3.Id = 'asdf'
Explain Query Plan
MATERIALIZE 2
COMPOUND QUERY
LEFT-MOST SUBQUERY
SCAN TABLE Table1 AS T1
UNION ALL
SCAN TABLE Table2 AS T2
SEARCH TABLE Table3 AS T3 USING COVERING INDEX sqlite_autoindex_Table3_1 (Id=?)
SCAN SUBQUERY 2 AS T
Your complex query does full table scans of Table1 and Table2 because you are not doing any filtering on UnionView.
It does use though sqlite_autoindex_Table3_1.
Also, the WHERE clause is applied after the joins.
If you filter UnionView before the join then indexes will be used:
EXPLAIN QUERY PLAN
SELECT T3.Id
FROM [Table3] T3
LEFT JOIN (SELECT Id FROM [UnionView] WHERE Id = 'asdf') T
ON T3.Id=T.Id
WHERE T3.Id = 'asdf'
Result:
MATERIALIZE 3
COMPOUND QUERY
LEFT-MOST SUBQUERY
SEARCH TABLE Table1 AS T1 USING INDEX sqlite_autoindex_Table1_1 (Id=?)
UNION ALL
SEARCH TABLE Table2 AS T2 USING INDEX sqlite_autoindex_Table2_1 (Id=?)
SEARCH TABLE Table3 AS T3 USING COVERING INDEX sqlite_autoindex_Table3_1 (Id=?)
SCAN SUBQUERY 3
I need to create a teradata macro to extract information into a volatile table first, then do CTE to extract data from this volatile table and insert into a teradata table, tried different ways all fail, appreciate help!
CREATE MACRO database.macro_insertion_tablename AS (
CREATE VOLATILE TABLE vt AS
(
SELECT
id, bu,
CONCAT(TO_CHAR(comment_date, 'yyyy-mm-dd HH24:MI:SS'), ' ', action) AS full_action,
ROW_NUMBER() OVER (PARTITION BY id ORDER BY date DESC) AS row_num,
COUNT(*) OVER (PARTITION BY id) as cnt
FROM database.table1
) WITH DATA UNIQUE PRIMARY INDEX(id, row_num) ON COMMIT PRESERVE ROWS;
WITH RECURSIVE cte (id, bu, act, rn) AS
(
SELECT
id, bu
,CAST(full_action AS VARCHAR(5000)) AS full_action
,row_num
FROM vt
WHERE row_num = cnt
UNION ALL
SELECT
vt.id, vt.bu
,cte.act || ' / ' || vt.full_action
,vt.row_num
FROM vt
JOIN cte On vt.id = cte.id AND vt.row_num = cte.rn - 1
)
INSERT INTO database.table (id, bu, full_action)
SELECT id, bu, act
FROM cte
WHERE rn = 1;
DROP TABLE vt;
);
DDL must be the only statement in a Teradata Macro.
As workaround you could switch to a Global Temporary Table which is defined once and then you simply Insert/Select into it instead of CREATE VOLATILE TABLE.
But in your case there's no need for a temp table plus inefficient recursive processing to get a "group concat":
SELECT id, max(bu) -- maybe min(bu)?
XmlAgg(Concat(To_Char(comment_date, 'yyyy-mm-dd HH24:MI:SS'), ' ', action)
ORDER BY comment_date) (VARCHAR(5000)) AS full_action
FROM database.table1
GROUP BY 1
will give you a similar result.
To follow up on my comments, you should be able to define multiple CTEs in the same statement. It may be tricky getting the RECURSIVE CTE to work, but it sounds like it's possible. Maybe something like this:
CREATE MACRO database.macro_insertion_tablename AS (
WITH vt (id, bu, full_action, row_num, cnt) AS
(
SELECT
id, bu,
CONCAT(TO_CHAR(comment_date, 'yyyy-mm-dd HH24:MI:SS'), ' ', action) AS full_action,
ROW_NUMBER() OVER (PARTITION BY id ORDER BY date DESC) AS row_num,
COUNT(*) OVER (PARTITION BY id) as cnt
FROM database.table1
),
RECURSIVE cte (id, bu, act, rn) AS
(
SELECT
id, bu
,CAST(full_action AS VARCHAR(5000)) AS full_action
,row_num
FROM vt
WHERE row_num = cnt
UNION ALL
SELECT
vt.id, vt.bu
,cte.act || ' / ' || vt.full_action
,vt.row_num
FROM vt
JOIN cte On vt.id = cte.id AND vt.row_num = cte.rn - 1
)
INSERT INTO database.table (id, bu, full_action)
SELECT id, bu, act
FROM cte
WHERE rn = 1;
);
I don't have a Teradata system to test with, so not 100% it will work as-is, but give it a try. You may need to change RECURSIVE to WITH RECURSIVE and also the ordering of the CTE queries (i.e. put the RECURSIVE one first). Take a look at these two links:
Teradata Forum - Multiple With Clause
teradata Forum - Common Table Expressions
I am trying to delete all rows in a simple table that have a duplicate value except for the duplicate with the highest id.
Table:
CREATE TABLE IF NOT EXISTS [Expression] (
[ID] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
[Value] VARCHAR(2048) NOT NULL
)
Attempted Queries:
DELETE Expression
WHERE EXISTS (
SELECT 1
FROM Expression Exp2
WHERE Expression.Value=Exp2.Value
AND Expression.Id < Exp2.Id)
fails with
"SQL logic error or missing database near "Expression":syntax error"
DELETE Exp1
FROM Expression Exp1
INNER JOIN Expression Exp2
ON Exp1.Value=Exp2.Value AND Exp1.Id < Exp2.Id
fails with
"SQL logic error or missing database near "Exp1":syntax error"
What syntax do I need to use?
Don't forget the FROM.
DELETE command
DELETE FROM Expression
WHERE EXISTS (
SELECT 1
FROM Expression Exp2
WHERE Expression.Value=Exp2.Value
AND Expression.Id < Exp2.Id
);
You can do this with many ways:
1) USING CTE:
WITH CTE AS (
SELECT ID, VALUE, ROW_NUMBER() OVER(ORDER BY Value) as RowNum
FROM Expression
)
DELETE
FROM CTE
WHERE ROWNum >1
2) USING Temp tables: Same concept
DELETE t1 FROM contacts t1 INNER JOIN contacts t2 WHERE t1.id < t2.id AND t1.email = t2.email;
Below statement generate "ORA-00911: invalid character" issue in ORACLE while migration where as it works fine directly inside AQT/SQL Developer.
INSERT INTO ss.CODETBL(FIELDNAME,FIELDVALUE,IDS)
SELECT FIELDNAME,FIELDVALUE,IDS FROM
(
SELECT 'ACCT_TYPE_CD' FIELDNAME, '$' FIELDVALUE,'SAMP' IDS FROM DUAL UNION ALL
SELECT 'ACCT_TYPE_CD', 'L','SAMP' FROM DUAL UNION ALL
SELECT 'ACCT_TYPE_CD', 'C','SAMP' FROM DUAL
)
D
WHERE NOT EXISTS
(
SELECT 1 FROM SS.CODETBL
WHERE D.FIELDNAME = FIELDNAME
AND D.FIELDVALUE = FIELDVALUE
AND D.IDS = IDS
);
I have following two tables:
CREATE TABLE messages (
id integer UNIQUE NOT NULL,
message text,
recipient integer NOT NULL,
sender integer NOT NULL,
sent_at text NOT NULL,
FOREIGN KEY (recipient) REFERENCES users (id),
FOREIGN KEY (sender) REFERENCES users (id)
);
CREATE TABLE users (
id integer UNIQUE NOT NULL,
username text NOT NULL,
);
I need a very specific query, that looks like the following:
SELECT *
FROM messages
WHERE sender = 123 OR recipient = 123
ORDER BY id desc
LIMIT 1
I need to kind of iterate over the messages table, using every user, and putting him in the WHERE statement.
-- TABLE 'users':
-- 123 = id of user1
-- 456 = id of user2
-- 789 = id of user3
Is it possible to iterate in SQLite?
Goal is, to get the newest "conversation" for every user in the users table. For every user, the newest message involving him should be displayed, no matter if that newest message was sent or recieved by him.
You could use a correlated subquery to get that value for each user ID:
SELECT id,
username,
(SELECT MAX(id)
FROM messages
WHERE sender = users.id
OR recipient = users.id
) AS last_message_id
FROM users
This is also possible with GROUP BY.
First join the two table together, then create a group for each user:
SELECT users.id,
MAX(messages.id)
FROM users
JOIN messages ON users.id = messages.sender OR
users.id = messages.recipient
GROUP BY users.id
SELECT year , COUNT(*) AS Count
FROM Movie
WHERE Movie.MID NOT IN
(SELECT DISTINCT m.MID
FROM Movie m
JOIN M_Cast m_c ON m.MID = m_c.MID
JOIN Person p_1 ON m_c.PID = p_1.PID
AND p_1.Gender='Male')
AND Movie.MID IN
(SELECT DISTINCT m.MID
FROM Movie m
JOIN M_Cast m_c ON m.MID = m_c.MID
JOIN Person p_1 ON m_c.PID = p_1.PID
AND p_1.Gender='Female')
GROUP BY year;