I have a select query statement which will result 600k rows. When I blindly extract the result using select statement it will impact db performance. Is there an option to use Set rowcount for fetching the data? I tried the below code but it keep on resulting top 50000 rows and ended up in infinite loop.
#!/bin/ksh -x
trap "" 1
updrowcount=50000
while [ $updrowcount -eq 50000 ]
do
QUERY="set rowcount 50000
select subject into tempdb..extract from tablename where fldr_id=8"
runisql <<EOF > db_restenter code here
$QUERY
goenter code here
quit
EOF
updrowcount=`grep "rows affected" db_rest |cut -c2- | cut -f1 -d ' '`
done
exit
If your subject is unique, you could try something like this:
set rowcount 50000
declare #subject varchar(...)
select #subject = max( subject ) from tempdb..extract
insert into tempdb..extract ( subject )
select subject
from tablename
where fldr_id=8
and (subject > #subject OR #subject is null)
order by subject
Otherwise, use a unique column, e.g.
set rowcount 50000
declare #maxId int
select #maxId = max( id ) from tempdb..extract
insert into tempdb..extract (id, subject )
select id, subject into tempdb..extract
from tablename
where fldr_id=8
and (id > #maxId OR #maxId is null)
order by id
(Obviously, you'll want to use an indexed column.)
Related
I have two tables A and B
In table A there are two columns "Sequence Number" and "Content".
Name Null? Type
------- ----- ------------
SEQ_NO NUMBER(6)
CONTENT VARCHAR2(20)
In table B there are multiple statement columns like "Stmt_1", "Stmt_2", "Stmt_3" etc.
Name Null? Type
------ ----- ------------
STMT_1 VARCHAR2(20)
STMT_2 VARCHAR2(20)
STMT_3 VARCHAR2(20)
STMT_4 VARCHAR2(20)
I want to create a trigger on table A such that after every insert on table A, according to the "Sequence Number" value the corresponding column in table B gets updated.
For example: If table A has "Sequence Number" = 1 , then "Stmt_1" of table B gets updated to the value of "Content" column in table A.
If table A is given as
"SEQ_NO" "CONTENT"
1 "This is Content"
Then Table B should look like:
"STMT_1","STMT_2","STMT_3","STMT_4"
"This is Content","","",""
My approach is as follows:
create or replace trigger TestTrig
after insert on A for each row
begin
declare
temp varchar2(6);
begin
temp := concat("Stmt_",:new.seq_no);
update B
set temp = :new.content;
end;
end;
But I am getting an error in the update statement.
Does anyone know how to approach this problem?
You need to use dynamic SQL (and ' is for string literals, " is for identifiers):
create or replace trigger TestTrig
after insert on A
for each row
DECLARE
temp varchar2(11);
begin
temp := 'Stmt_' || TO_CHAR(:new.seq_no, 'FM999990');
EXECUTE IMMEDIATE 'UPDATE B SET ' || temp || ' = :1' USING :NEW.content;
end;
/
You probably want to handle errors when seq_no is input as 5 and there is no STMT_5 column in table B:
create or replace trigger TestTrig
after insert on A
for each row
DECLARE
temp varchar2(11);
INVALID_IDENTIFIER EXCEPTION;
PRAGMA EXCEPTION_INIT(INVALID_IDENTIFIER, -904);
begin
temp := 'Stmt_' || TO_CHAR(:new.seq_no, 'FM999990');
EXECUTE IMMEDIATE 'UPDATE B SET ' || temp || ' = :1' USING :NEW.content;
EXCEPTION
WHEN INVALID_IDENTIFIER THEN
NULL;
end;
/
However
I would suggest that you do not want a table B or a trigger to update it and you want a VIEW instead:
CREATE VIEW B (stmt_1, stmt2, stmt3, stmt4) AS
SELECT *
FROM A
PIVOT (
MAX(content)
FOR seq_no IN (
1 AS stmt_1,
2 AS stmt_2,
3 AS stmt_3,
4 AS stmt_4
)
);
fiddle;
I'm trying to replace a placeholder string inside a selection of 10 random records with a random string (a name) taken from another table, using only sqlite statements.
i've done a subquery in order to replace() of the placeholder with the results of a subquery. I thought that each subquery loaded a random name from the names table, but i've found that it's not the case and each placeholder is replaced with the same string.
select id, (replace (snippet, "%NAME%", (select
name from names
where gender = "male"
) )
) as snippet
from imagedata
where timestamp is not NULL
order by random()
limit 10
I was expecting for each row of the SELECT to have different random replacement every time the subquery is invoked.
hello i'm %NAME% and this is my house
This is the car of %NAME%, let me know what you think
instead each row has the same kind of replacement:
hello i'm david and this is my house
This is the car of david, let me know what you think
and so on...
I'm not sure it can be done inside sqlite or if i have to do it in php over two different database queries.
Thanks in advance!
Seems that random() in the subquery is only evaluated once.
Try this:
select
i.id,
replace(i.snippet, '%NAME%', n.name) snippet
from (
select
id,
snippet,
abs(random()) % (select count(*) from names where gender = 'male') + 1 num
from imagedata
where timestamp is not NULL
order by random() limit 10
) i inner join (
select
n.name,
(select count(*) from names where name < n.name and gender = 'male') + 1 num
from names n
where gender = 'male'
) n on n.num = i.num
My requirement is to create a procedure or SQL query in which where clause should be created in run time depending on the paramters provided by the user.
Example if user provides data for three columns then where clause should have filter conditions for these three columns only to select the data from database table, like wise if user provides data for 4 columns then where caluse should have 4 columns.
I don't see any very specific solution to your question but it can be done using putting OR in where clause with different set of user input. See below:
Created procedure: Here my employee table has emp_id,name and salary columns. Now am assuming user sometimes passes emp_id and emp_name alone.
CREATE OR REPLACE PROCEDURE dynmc_selec (id NUMBER DEFAULT 0,
name VARCHAR2 DEFAULT 'A',
salary NUMBER DEFAULT 0,
emp_output IN OUT SYS_REFCURSOR)
AS
var VARCHAR2 (100);
BEGIN
--You need to make several combinations in where clause like ( emp_id , emp_name , salary ) OR ( emp_id , emp_name ) OR (emp_name , salary ) and use it in where clause with 'OR'.
--Also its needed that all the columns of the table is in where clause. If user doesnot pass anything then defualt value will be passed.
var :=
'select emp_id from employee where ( emp_id ='
|| id
|| ' and emp_name = '''
|| name
|| ''' ) OR emp_sal = '
|| salary;
DBMS_OUTPUT.put_line (var);
EXECUTE IMMEDIATE var;
OPEN emp_output FOR var;
EXCEPTION
WHEN OTHERS
THEN
NULL;
END;
Calling it:
declare
a SYS_REFCURSOR;
v_emp_id employee.emp_id%type;
begin
--passing emp_id and name only
dynmc_selec(id=>1,name=>'KING',emp_output=>a);
loop
FETCH a INTO v_emp_id;
EXIT WHEN a%NOTFOUND;
DBMS_OUTPUT.PUT_LINE(v_emp_id );
end loop;
end;
Output:
select emp_id from employee where ( emp_id =1 and emp_name = 'KING' ) OR emp_sal = 0
1
select * from test_table
where
(1 = nvl2(:l_date, 0, 1) or created_at > :l_date)
and (1 = nvl2(:l_no, 0,1) or no = :l_no);
With Oracle nvl2 function:
When the parameter l_date is null, then 1 = nvl2(l_date, 0, 1) evaluates to true and the filter by created_at is not taking place.
When the parameter l_date is not null, then 1 = nvl2(l_date, 0, 1) evaluates to false and the filter by created_at is taking place.
The same thing happens with the parameter l_no.
I'm attempting to create a derived table of country data from several other tables. Those tables look something like this:
Countries
ID | Name
Country_demographics
ID | date | Population | urban_pop | birth_rate
country_financials
ID | date | GDP | GDP_per_capita
Now, I'm trying to make a new table with
New_Table
ID | Name | date | population | urban_pop | birth_rate | gdp | gdp_per_capita
I have a stored procedure that currently looks something like this:
CREATE OR REPLEACE PROCEDURE SP_COUNTRY (
chunkSize IN INT
) AS
--create tables to hold IDs and stats
TYPE idTable IS TABLE OF COUNTRIES.ID%TYPE;
TYPE dateTable IS TABLE OF COUNTRY_DEMOGRAPHICS.EVALUATION_DATE%TYPE;
TYPE totPopTable IS TABLE OF COUNTRY_DEMOGRAPHICS.POPULATION_TOTAL_COUNT%TYPE;
TYPE urbanPopTable IS TABLE OF COUNTRY_DEMOGRAPHICS.POPULATION_URBAN_COUNT%TYPE;
--constructors
ids idTable;
dates dateTable;
totpop totPopTable;
urbanpop urbanPopTable;
--cursors
CURSOR countryCur IS
SELECT c.ID,cd.EVALUATION_DATE,cd.POPULATION_TOTAL_COUNT,cd.POPULATION_URBAN_COUNT
FROM COUNTRIES c,COUNTRY_DEMOGRAPHICS cd
WHERE c.id=cd.COUNTRY_ID
ORDER BY ID,EVALUATION_DATE;
BEGIN
dbms_output.enable(999999);
--open cursor
OPEN countryCur;
LOOP
--fetch and bulk collect
FETCH countryCur BULK COLLECT INTO ids,dates,totpop,urbanpop
LIMIT chunkSize;
--loop over collections
FOR j in ids.FIRST..ids.LAST
LOOP
--populate record
country.COUNTRY_ID := ids(j);
country.EVALUATION_DATE := dates(j);
country.POPULATION_TOTAL_COUNT := totpop(j);
country.POPULATION_URBAN_COUNT := urbanpop(j);
--update/insert table with record (much confusion here on how to update/insert and check if already exists in derived table..)
UPDATE NEW_TABLE SET ROW = country WHERE COUNTRY_ID = ids(j);
dbms_output.put_line('id: ' || country.COUNTRY_ID || ' date: ' || country.EVALUATION_DATE);
dbms_output.put_line(' pop: ' || country.POPULATION_TOTAL_COUNT || ' urban: ' || country.POPULATION_URBAN_COUNT);
END LOOP;
END LOOP;
--close cursor
CLOSE countryCur;
END;
As you can see, I'm using a different table type for each piece of data. I then plan on making a loop and then just inserting/updating in my new_table. I think there must be a better way to do this with %rowtype, or maybe creating a record and inserting the record? I'm not sure
Unless I'm missing something by simplifying this, and assuming cd.date and cf.date are equal, this should work:
INSERT INTO NEW_TABLE (ID, Name, date, population, urban_pop, birth_rate, gdp, gdp_per_capita)
values
(select c.id, c.name, cd.date,
cd.population, cd.urban_pop, cd.birthrate,
cf.gdp, cf.gdp_per_capita)
from Countries c, country_demographics cd, country_financials cf
where c.id = cd.id
and cd.id = cf.id);
Edit: Use the MERGE statement to update or insert depending on if the primary key exists:
MERGE INTO NEW_TABLE nt
USING ( select c.id, c.name, cd.date,
cd.population, cd.urban_pop, cd.birthrate,
cf.gdp, cf.gdp_per_capita
from Countries c, country_demographics cd, country_financials cf
where c.id = cd.id
and cd.id = cf.id ) a
ON (nt.id = a.id )
WHEN MATCHED THEN
UPDATE SET nt.Name = a.Name,
nt.date = a.date,
nt.population = a.population,
nt.urban_pop = a.urban_pop,
nt.birth_rate = a.birth_rate,
nt.gdp = a.gdp,
nt.gdp_per_capita = a.gdp_per_capita
WHEN NOT MATCHED THEN
INSERT (ID, Name, date, population, urban_pop, birth_rate, gdp, gdp_per_capita)
VALUES (a.id, a.Name, a.date, a.population, a.urban_pop, a.birth_rate, a.gdp, a.gdp_per_capita);
I have a table employee has 30000 records. I need to delete duplicate records based on two columns concatenation. For example name and job, like
martin clerk
martin clerk
Below is my code:
declare
type typ_emp is table of emp%rowtype;
v_emp typ_emp;
cursor cur_emp
is
select *
from emp a
where rowid >
(select min (rowid)
from emp b
where concat (concat (b.ename, '-'), b.job) =
concat (concat (a.ename, '-'), a.job)
)
;
begin
open cur_emp;
loop
fetch cur_emp bulk collect into v_emp;
exit when v_emp.count = 0;
if v_emp.count > 0
then
for i in v_emp.first .. v_emp.last
loop
insert into backup_emp (ename, job)
values (v_emp (i).ename, v_emp (i).job)
;
end loop;
end if;
end loop;
close cur_emp;
delete
from emp s
where s.rowid >
any (select t.rowid
from emp t
where concat (concat (t.ename, '-'), t.job) =
concat (concat (s.ename, '-'), s.job));
commit;
exception
when others then
Raise;
end;
It is taking a long time to delete the records. Can anyone help me in tuning a query for this or suggest me what is the better approach.
Thanks in advance.
Creating function based index might improve your performance
CREATE INDEX concatindex ON emp (ename||'-'||job);
Delete statement would look like this
delete emp a where a.rowid > (select min(rowid) from emp b where b.ename||'-'||b.job=a.ename||'-'||a.job)
unless you need to insert deleted rows into backup table which is not clear from your question. If so I would rather bulkcollect rows into collection. Leave a comment if you need to have detailed this option.
I hope this helps.
SELECT ROWID, ename || '-' || job AS concatenation,
decode(rank() over(PARTITION BY ename || '-' || job ORDER BY ROWID), 1, 'keep', 'delete') AS to_do
FROM emp
ORDER BY ename || '-' || job, ROWID;
Here is my code changes:
cursor cur_emp
is
select *
from
(select b.*
,row_number()over(partition by concat (concat (b.ename, '-'), b.job) order by ename)cnt
from emp b
) where cnt>1;