In Tableau Prep I have an output table that I want to import into an Oracle database.
In that output table, there's a column (file_date) with a date value (ex.: '2021-01-01'). The date value is the same for all rows.
Output table:
enter image description here
I need to write a custom SQL query (in Tableau Prep) that checks if my Oracle table already has any rows where the date = '2021-01-01'. If so, all rows need to be deleted before I import the new data.
Table_1:
enter image description here
Something like:
DELETE FROM table_1 WHERE date_column = '2021-01-01';
After checking it will find that the 1st row has the date = '2021-01-01' and deletes that row.
Table_1 after:
enter image description here
As the date changes every time new files come up, manually entering the date in the query is not a possibility. Is there any way to use a value from my table in a custom SQL query?
I'm aware that Tableau Desktop allows for the creation of parameters, but that's not available in Tableau Prep.
If the output is all the same date, then that must also be true of the input data. There is no reason to check for existing data values. Since finding no matching records on delete does not throw an error, checking is not necessary - just delete.
delete
from table_1
where date_column =
(select date_column
from import_table_1
fetch first 1 row only
);
Related
Recently I've moved one Teradata test table data to bigquery and I see the row count difference between TD and BQ. As I checked further, I see one of the row value is in "DATE"format instead of "String" because that column is PI column and the data type is VARCHAR. so this row is returning in BQ when I run select but not in TD whereas I see that row when I'm exporting data to excel. I'm really not sure what could be the reason of not showing when I run select statement. Please help me someone to know the reason and also let me know how can I search those problematic data when table is too big. Thanks.
eg : create multiset table Test(a int,b varchar,c varchar,d timestamp(6)) primary index (b);
Data like below in that table.
enter image description here
I have two databases with the same structure. The first is the main one, while the second get updated periodically (in reality I have multiple "secondary" databases that I want to merge one by one into the main one).
The structure of the main and the secondary databases is identical.
I want to periodically dump all new values from the secondary database in the main one. However, the second time I do it, I want to exclude rows that were already copied the first time (and so on).
The tables in all these database have:
an ID column set as PRIMARY KEY going from 1 to N for each database (I suspect this was a mistake, but at the moment I can't change this)
a DATE column, representing a posix timestamp (float)
some other columns
My code looks like this:
ATTACH DATABASE secondary.db AS temp_db
DROP TABLE IF EXISTS my_table_temp
CREATE TABLE my_table_temp AS SELECT * FROM my_table
INSERT INTO main.my_table_temp SELECT * FROM temp_db.my_table
DELETE FROM my_table
INSERT INTO main.my_table SELECT DISTINCT * FROM main.my_table_temp ORDER BY date
DROP TABLE my_table_temp
the problem is that - I suspect due to the repeated ID column - the DISTINCT clause returns me:
UNIQUE constraint failed: my_table.id
However I don't care at all of the ID field that could also be dropped or reset.
NOTES:
the secondary databases are constantly updated by a code that - at the moment - I can't change
I initialize the "main" database copy-pasting one of the secondary to avoid regenerating the whole structure from scratch. Maybe there is a better way of doing this
Apologies if this is a naive question, but I'm very new with SQLite.
Thanks
Following the advice from #forpas, I solved this with the following code:
Assuming the columns to be id,date,col1 and col2
ATTACH DATABASE secondary.db AS temp_db
DROP TABLE IF EXISTS my_table_temp
CREATE TABLE my_table_temp AS SELECT date,col1,col2 FROM my_table
INSERT INTO main.my_table_temp SELECT date,col1,col2 FROM temp_db.my_table
DROP TABLE my_table /* I need to recreate my_table as I've removed a column*/
CREATE TABLE main.my_table AS SELECT DISTINCT date,col1,col2 FROM main.my_table_temp ORDER BY date
DROP TABLE my_table_temp
also, I automatized the extraction of the column names doing
SELECT name FROM PRAGMA_TABLE_INFO('my_table');
This is then passed to the python code running the script and the column id is removed from the list. Note that the second (and following) time I run this code, the column id won't be present in my_table to start with. However this approach allows the code to be the same in the two cases: either if the column id is there or not.
This procedure is then iterated over each table name to fully merge the two databases.
I need a help in getting the PLSQL procedure to : Insert/Delete the rows of a table , because as I used Update functionality getting duplicates for that particular Sequence ID field.
So for a particular sequence ID row, whenever I insert the data, it should be the latest in that table.
The last sentence you wrote suggests that you have to
delete row(s) whose ID equals that particular "sequence ID" value
then insert a new row
If you expected some code to be written, you should have posted some more info (CREATE TABLE and INSERT INTO sample data, as well as the way you manipulate it by inserting new row, showing what you expect to happen with old one(s)). It is difficult to write code based on unknown data model.
A guess...
INSERT INTO schema_name.table_name(
primary_key_column
, other_column
)
VALUES(
(SELECT max(primary_key_column)+1 FROM schema_name.table_name),
, 'other_value'
);
COMMIT;
This is the procedure I am using:
https://drive.google.com/file/d/1eGbxSppjexpICKh6pzuW0ZzckVxA6BB0/view?usp=sharing
My requirement when we need to insert the new data , the previous data should be deleted for the corresponding ID.
In the above procedure I am updating the data.
In APEX, when performing a Data Load (e.g. upload of a csv file into APEX application), is it possible to validate input data using a transformation rule?
For example, suppose to upload data about cars that have been sold this month.
The target table has the column car_manufacturer and num_car_sold.
The column car_manufacturer must accept only three values, say ('A1', 'A2', 'A3').
In a pseudo PLSQL, just to give an idea:
IF :car_manufacturer IN ('A1, A2, A3') then :car_manufacturer else <error>
How can I check this in the upload phase? Is it possible to use a transformation rule, in order that if it fails, it returns an error message? Other ways?
Thanks in advance.
You could put a constraint on the table definition as per the other answer, or if you only want the error message for when the Data Load is used, you can use a Table Lookup.
Go to Shared Components -> Data Load Definitions
Open the Data Load Definition that you want to edit
Create Table Lookup
Select the column (e.g. car_manufacturer)
Set the Table Lookup attributes to a table that contains the list of valid values (you'll need either a table or a view for this)
Leave Insert New Value set to No (If set to 'No' (the default) then a new record will not be created in the lookup table if the lookup column value(s) entered do not already exist. If set to 'Yes' then a record will be created in the lookup table using the upload column(s) and the Upload Key Column will be retrieved from the newly created record.)
Set Error Message to the message you want to return if a match is not found.
How about having a check constraint on the table for the column "car_manufacturer"?
ALTER TABLE TABLE_NAME
ADD CONSTRAINT CHECK_CAR_MANUFACTURER
CHECK ( CAR_MANUFACTURER in ('A1', 'A2', 'A3'));
I have an sqlite database which currently holds an integer field called Year which currently only stores the year. In future versions I want to store a full date and time.
I updated my table to include a FullDate field using alter table.
> ALTER TABLE Files ADD COLUMN UploadDate DATETIME DEFAULT 0;
Next, I want to migrate all the existing year fields to the new field. So I'm looking for something like:
> UPDATE Files SET UploadDate = (DATETIME('%Y-%m-%d', Year, 1, 1));
Unfortunately this doesn't seem to work as the result is empty. I also tried the date and strftime functions but they either result in incorrect data or empty data.
What's the proper way to update a DATETIME field with existing data in the same table?
The DATE and DATETIME functions don't have a format parameter.
For more: http://sqlite.org/lang_datefunc.html
The main catch is that SQLite does not have any date or time types, so that you might as well populate your field with:
UPDATE Files SET UploadDate = Year || '-01-01';
And that will do the exact same thing. Dates are not stored as typed, but can be evaluated as such against the date and time functions.