Requirement:
Source table contains 5 columns. We are replicating 3 columns on target out of 5.
SEQ_ID is additional column on target.
When update operation is performed on columns which are not in target table,SEQ_ID is increased.
SEQ_ID should increase only when update performed on columns which are present on target.
Enabling unconditional supplemental table level logging on selected columns(ID,AGE,COL1) to be replicated:
Source:
Table name: Test1(ID,AGE,COL1,COL2,COL3)
Target:
Table name: Test1(ID,AGE,COL1,SEQ_ID)
We created a sequence to increse the SEQ_ID when a insert or update happens.
Scenario :
If insert or update happens on source table on these columns (ID,AGE,COL1) SEQ_ID is incresed,
and if update happens on others columns(COL2,COL3) SEQ_ID is also getting incremented.
Our Requirement is when update happens on others columns(COL2,COL3) ,SEQ_ID should not get incrementd.
I want to skip the transaction of updates happening on columns(COL2,COL3) .
Source:
Primary extract test_e1
EXTRACT TEST_e1
USERID DBTUAT_GG,PASSWORD dbt_1234
EXTTRAIL /DB_TRACK_GG/GGS/dirdat/dd
GETUPDATEBEFORES
--IGNOREUPDATES
--IGNOREDELETES
NOCOMPRESSUPDATES
TABLE HARI.TEST1,COLS(ID,AGE,COL1),FILTER (ON UPDATE,IGNORE UPDATE, #STREQ(before.AGE, AGE) = 0);
Datapump test_p1:
EXTRACT TEST_P1
USERID DBTUAT_GG,PASSWORD dbt_1234
RMTHOST 10.24.187.235, MGRPORT 7809,
RMTTRAIL /Trail_files/tt
--PASSTHRU
TABLE DBTUAT_GG.TEST1;
Target:
Target Repicat file:
Edit param test_r
REPLICAT TEST_R
USERID GGPROD,PASSWORD GGPROD_123
SOURCEDEFS ./dirsql/def32.sql
HANDLECOLLISIONS
IGNOREDELETES
INSERTMISSINGUPDATES
MAP HARI.TEST1, TARGET HARI.TEST1, &
SQLEXEC (ID test_num,QUERY "select GGPROD.test_seq.NEXTVAL test_val from dual", NOPARAMS), &
COLMAP(USEDEFAULTS,SEQ_ID=test_num.test_val);
Kindly suggest any possible solutions .
First note: you don't need USERID and PASSWORD in the Pump PRM. The Pump process does not connect to any database.
Actually you have already achieved your goal. You are only replicating the data when AGE is modified. There is a FILTER clause in the Extract PRM file. The ID column should not get changed since this is the PK. The only problem is that your are increasing the SEQ_ID if a delete gets replicated.
Something like that should work:
ALLOWDUPTARGETMAP
IGNOREDELETES
-- just inserts and updates that change COL1
MAP HARI.TEST1, TARGET HARI.TEST1, &
SQLEXEC (ID test_num,QUERY "select GGPROD.test_seq.NEXTVAL test_val from dual", NOPARAMS), &
COLMAP(USEDEFAULTS, SEQ_ID = test_num.test_val);
-- just replicate the delete operations
GETDELETES
IGNOREINSERTS
IGNOREUPDATES
MAP HARI.TEST1, TARGET HARI.TEST1;
GETINSERTS
GETUPDATES
Related
I am trying to delete some duplicate records from mysql table but it was not working.
I am taking help of https://www.javatpoint.com/mysql-delete-duplicate-records
If I will try with example database which is given example it was working fine.
But in my table it was not working.
DELETE S1 FROM employee_attendance AS S1 INNER JOIN employee_attendance AS S2 WHERE S1.DbKey < S2.DbKey AND S1.DivisionDbKey = S2.DivisionDbKey AND S1.Date = S2.Date AND S1.Month = S2.Month AND S1.FinancialYearDbKey = S2.FinancialYearDbKey AND S1.EmployeeDbKey = S2.EmployeeDbKey AND S1.Attendance = S2.Attendance AND S1.InTime = S2.InTime AND S1.OutTime = S2.OutTime AND S1.EmployeeDbKey = 6798 AND S1.Month = '05' AND S1.FinancialYearDbKey = 5;
I am getting error
#1205 - Lock wait timeout exceeded; try restarting transaction
I have tried with another example https://www.geeksforgeeks.org/sql-query-to-delete-duplicate-rows/
DELETE FROM employee_attendance WHERE DbKey NOT IN (SELECT MAX(DbKey) FROM employee_attendance WHERE EmployeeDbKey = 6798 AND Month = '05' AND FinancialYearDbKey = '5' GROUP BY DivisionDbKey,Date,Month,FinancialYearDbKey,EmployeeDbKey,Attendance,InTime,OutTime)
I am getting same error.
#1205 - Lock wait timeout exceeded; try restarting transaction
Any suggestion will be appriciated. Thank you.
I personally think this is a bad practice. You should instead make a (empty) duplicate of the table employee_attendance then define a UNIQUE KEY on that new table that will prevent duplicate entries.
Consider these steps:
Create a duplicate table:
CREATE TABLE employee_attendance_new LIKE employee_attendance;
Add UNIQUE INDEX - now, this is just a simple example. You can add or reduce columns to the unique index but make sure that you drop the existing unique index first then only you re-create:
ALTER TABLE employee_attendance_new
ADD UNIQUE INDEX unq_idx(EmployeeDbKey, date, InTime, OutTime);
Insert the data into the new table using INSERT IGNORE..:
INSERT IGNORE INTO employee_attendance_new
SELECT * FROM employee_attendance;
Check and compare both table. If you're satisfied with the result, rename those tables:
RENAME TABLE employee_attendance TO employee_attendance_old;
RENAME TABLE employee_attendance_new TO employee_attendance;
Now you have the new table with no duplicates and the old table for reference or in case there are some data you need from it.
Fiddle example
I need to merge some randomly uppercased data that has been collected in an SQLite table key_val, such that key is always lowercase and no vals are lost. There is a unique compound index on key,val.
The initial data looks like this:
key|val
abc|1
abc|5
aBc|1
aBc|5
aBc|3
aBc|2
AbC|1
abC|3
The result after the merge would be
key|val
abc|1
abc|2
abc|3
abc|5
In my programmer brain, I would
for each `key` with upper case letters;
if a lower cased `key` is found with the same value
then delete `key`
else update `key` to lower case
Re implementing the loop has a sub query for each row found with upper case letters, to check if the val already exists as a lower case key
If it does, I can delete the cased key.
From there I can UPDATE key = lower(key) as the "duplicates" have been removed.
The first cut of the programming method of finding the dupes is:
SELECT * FROM key_val as parent
WHERE parent.key != lower(parent.key)
AND 0 < (
SELECT count(s.val) FROM key_val as s
WHERE s.key = lower(parent.key) AND s.val = parent.val
)
ORDER BY parent.key DESC;
I'm assuming there's a better way to do this in SQLite? The ON CONFLICT functionality seems to me like it should be able to handle the dupe deletion on UPDATE but I'm not seeing it.
First delete all the duplicates:
DELETE FROM key_val AS k1
WHERE EXISTS (
SELECT 1
FROM key_val AS k2
WHERE LOWER(k2.key) = LOWER(k1.key) AND k2.val = k1.val AND k2.rowid < k1.rowid
);
by keeping only 1 combination of key and val with the min rowid.
It is not important if you kept the key with all lower chars or not, because the 2nd step is to update the table:
UPDATE key_val
SET key = LOWER(key);
See the demo.
Honestly it might just be easier to create a new table and then insert into it. As it seems you really just want a distinct select here, use:
INSERT INTO kev_val_new ("key", val)
SELECT DISTINCT LOWER("key"), val
FROM key_val;
Once you have populated the new table, you may drop the old one, and then rename the new one to the previous name:
DROP TABLE key_val;
ALTER TABLE key_val_new RENAME TO key_val;
I agree with #Tim that it would be easire to re-create table using simple select distict lower().. statement, but that's not always easy if table has dependant objects (indexes, triggers, views). In this case this can be done as sequence of two steps:
insert lowered keys which are not still there:
insert into t
select distinct lower(tr.key) as key, tr.val
from t as tr
left join t as ts on ts.key = lower(tr.key) and ts.val = tr.val
where ts.key is null;
now when we have all lowered keys - remove other keys:
delete from t where key <> lower(key);
See fiddle: http://sqlfiddle.com/#!5/84db50/11
However this method assumes that key is always populated (otherwise it would be a strange key)
If vals can be null then "ts.val = tr.val" should be replaced with more complex stuff like ifnull(ts.val, -1) = ifnull(tr.val, -1) where -1 is some unused value (can be different). If we can't assume any unused value like -1 then it should be more complex check for null / not null cases.
Hello i have an sqlite db with many records like 10540 record they are ordered by creation time , i want to shift like a record in the middle and like to do it automatically
for example :
select * from table1 where id >= 8521;
UPDATE Table1 SET id = id +1 ;
does not work i get Error: Result: UNIQUE constraint failed:
so i want to shift up all records from 8521 to the last record and get place in the 8520 place for example so i can insert my record in that place of table .
even the
id = select max(id)+1
does not work how can i increment the id from last record to the needed record so i can put a place in the records db
A simple update statement would fail, as it would try to create duplicate values in the primary key.
What you can do is this:
First update the column to the negatives of the values they should have:
update table1
set id = -(id + 1)
where id > 8520;
Now there are no duplicates and you just need to update again to the positive values:
update table1
set id = -id
where id < 0;
This will do the trick, but any kind of updating the primary key is not a recommended practice
There are two tables ... the 'master' (tblFile) holds record details of files that have been processed by some java code .. the PK is the file name. A column of interest in this table is the 'status' column (VALID or INVALID).
In the subordinate table (tblAnomaly) there are many records that hold the anomalies from processing each file .. this table has a FK as the file name from tblFile ... and along with other columns of relevant data there is a boolean type column which acts as an acceptance flag of the anomaly. NULL is accept .. # is not.
The user manually works their way through the list of anomalies presented in a swing ListPane and checks off the anomalies as they address the issue in the source file. When all the anomalies have been dealt with i need the status of the file in tblFile to change to VALID so that it can be imported into a database.
Here is the trigger i have settled on having designed the statements individually via an SQL editor .. however, i do not know how to validate/debug the trigger statement after it is loaded to the database, so cannot work out why it does not work ... no action and no feedback!!
CREATE TRIGGER
updateFileStatus
AFTER UPDATE ON tblAnomaly
WHEN 0 = (SELECT COUNT(*) FROM tblAnomaly WHERE tblAnomaly.file_name = tblFile.file_name AND tblAnomaly.accept = '#')
BEGIN
UPDATE tblFile
SET tblFile.file_status = 'VALID'
WHERE tblFile.file_name = tblAnomaly.file_name;
END;
So i worked it out! .. here is the solution that works.
CREATE TRIGGER
updateFileStatus
AFTER UPDATE ON tblAnomaly
WHEN 0 = (SELECT COUNT(*)
FROM tblAnomaly
WHERE file_name = old.file_name
AND accept = '#')
BEGIN
UPDATE tblFile
SET file_status = 'VALID'
WHERE file_name = old.file_name;
END;
I have 3 tables tblpermission, tblgroup, tblassigngrouppermission. Then I have a design there have two comboboxes for selecting group and permission. After select I add it to a listview. Then I save it, at that time it will go to the table tblassigngrouppermission.
That table has columns such as assign id (default increment), groupid, permission id. All are correctly added to the table. After that saving if I select the same group for assign permission. Then I select already assigned permission and click save it added to the table. But I need there not add the already assigned permission to the table.
How can I do this?
When you are saving the data back to tblassigngrouppermission you will have to check the presence of group_id and permission_id in the table.
if they are present you will have to update tblassigngrouppermission else you will have to insert in tblassigngrouppermission
If you are using stored procedure you could do this
IF NOT EXISTS(Select permissionId From tblassigngrouppermission
Where groupId=#GroupID AND permissionId=#permissionId)
Begin
INSERT INTO tblassigngrouppermission(groupId,permissionId) Values(#groupId, #PermissionID)
End
You can also check from your code
==> Write a function that test if the permission already exist
bool GroupPermissionExists(int groupId, int permissionId)
{
//Select Where GroupId=groupId AND PermissionID=permissionId
}
if(!GroupPermissionExists(10, 123))
{
AddPermissionToGroup(10, 123);
}