I am new to Teradata. Can anyone tell me How exactly the AMPs going to helpful in creation of any table in Teradata.
Lets have a scenario.
I have a Teradata database with 4 AMPs. I learned that AMPs will usefull when we inserting the data into a table. Depending on the indexes it will distribute the data with the help of respected AMPs. But while creating the table, the command needs to execute through AMPs only. So i want to know which AMP will be used at that time??
The actual creation of the table in the data dictionary is a RowHash level operation involving a single AMP to store the record in DBC.TVM. Based on the other actions listed in the EXPLAIN there may be other AMPs involved as well but there is not single All-AMP operation. (This doesn't take into consideration the loading of the data and its distribution across the AMPs.)
Sample EXPLAIN:
1) First, we lock FUBAR.ABC for exclusive use.
2) Next, we lock a distinct DBC."pseudo table" for write on a RowHash
for deadlock prevention, we lock a distinct DBC."pseudo table" for
write on a RowHash for deadlock prevention, we lock a distinct
DBC."pseudo table" for read on a RowHash for deadlock prevention,
and we lock a distinct DBC."pseudo table" for write on a RowHash
for deadlock prevention.
3) We lock DBC.DBase for read on a RowHash, we lock DBC.Indexes for
write on a RowHash, we lock DBC.TVFields for write on a RowHash,
we lock DBC.TVM for write on a RowHash, and we lock
DBC.AccessRights for write on a RowHash.
4) We execute the following steps in parallel.
1) We do a single-AMP ABORT test from DBC.DBase by way of the
unique primary index.
2) We do a single-AMP ABORT test from DBC.TVM by way of the
unique primary index.
3) We do an INSERT into DBC.Indexes (no lock required).
4) We do an INSERT into DBC.TVFields (no lock required).
5) We do an INSERT into DBC.TVM (no lock required).
6) We INSERT default rights to DBC.AccessRights for FUBAR.ABC.
5) We create the table header.
6) Finally, we send out an END TRANSACTION step to all AMPs involved
in processing the request.
-> No rows are returned to the user as the result of statement 1.
Related
I’m trying to cause a ‘SELECT’ query to fail if the record it is trying to read is locked.
To simulate it I have added a trigger on UPDATE that sleeps for 20 seconds and then in one thread (Java application) I’m updating a record (oid=53) and in another thread I’m performing the following query:
“SET STATEMENT max_statement_time=1 FOR SELECT * FROM Jobs j WHERE j.oid =53”.
(Note: Since my mariadb server version is 10.2 I cannot use the “SELECT … NOWAIT” option and must use “SET STATEMENT max_statement_time=1 FOR ….” instead).
I would expect that the SELECT will fail since the record is in a middle of UPDATE and should be read/write locked, but the SELECT succeeds.
Only if I add ‘for update’ to the SELECT query the query fails. (But this is not a good option for me).
I checked the INNODB_LOCKS table during the this time and it was empty.
In the INNODB_TRX table I saw the transaction with isolation level – REPEATABLE READ, but I don’t know if it is relevant here.
Any thoughts, how can I make the SELECT fail without making it 'for update'?
Normally consistent (and dirty) reads are non-locking, they just read some sort of snapshot, depending on what your transaction isolation level is. If you want to make the read wait for concurrent transaction to finish, you need to set isolation level to SERIALIZABLE and turn off autocommit in the connection that performs the read. Something like
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
SET autocommit = 0;
SET STATEMENT max_statement_time=1 FOR ...
should do it.
Relevant page in MariaDB KB
Side note: my personal preference would be to use innodb_lock_wait_timeout=1 instead of max_statement_time=1. Both will make the statement fail, but innodb_lock_wait_timeout will cause an error code more suitable for the situation.
I am looking to implement the equivalent of snapshot isolation with a Teradata transaction. Oracle supports this type of isolation, but Teradata does not (at least not in versions 14 or prior that I am aware of). The goal is to create a procedure that deletes a table's contents and then repopulates it all while preventing other users from reading from/writing to the table.
I came across the begin request statement which, according to my understanding, allows the optimizer to know about all the various table locks within the request.
I wrote the procedure below, but don't know how to reliably debug it as easy as I would if I were testing thread locking in a .NET application (easy to set breakpoints and monitor other threads). In Teradata, not sure if what I wrote here will properly lock mydb.destinationtable exclusively for the duration of the procedure. Is this correct?
Edit: I'll add that the procedure does work. It's just being able to properly time a SELECT while it's doing its DELETE/INSERT.
replace procedure mydb.myproc()
begin
begin request
locking mydb.destinationtable for exclusive
delete mydb.destinationtable;
locking mydb.destinationtable for exclusive
insert into mydb.destinationtable
select * from mydb.sourcetable;
end request;
end;
BEGIN REQUEST/END REQUEST creates a so-called Multi Statement Request (MSR) which is the same a submitting both requests in SQL Assistant using F9.
To see the plan run this with F9:
EXPLAIN
locking mydb.destinationtable for exclusive
delete mydb.destinationtable;
insert into mydb.destinationtable
select * from mydb.sourcetable;
or in BTEQ:
EXPLAIN
locking mydb.destinationtable for exclusive
delete mydb.destinationtable
;insert into mydb.destinationtable
select * from mydb.sourcetable;
Btw, the 2nd lock is redundant.
But. When you run Delete & InsSel as a single transaction both will be Transient Journalled. Which is much slower than seperate requests.
A more common way to do this is to use two copies of the target table and base access on Views not Tables:
-- no BEGIN/END REQUEST
insert into mydb.destinationtable_2
select * from mydb.sourcetable;
-- there's just a short dictionary lock
-- all requests against the view submitted before the replace use the old data
-- and all submitted after the new data
replace view myview as
select * from mydb.destinationtable_2;
delete from mydb.destinationtable_1;
Now your SP only needs the logic to switch between 1 & 2 (based on table [not] empty)
I am developing an application which fetches some data from a Teradata DWH. DWH developers told me to use LOCK ROW FOR ACCESS before all SELECT queries to avoid delaying writes to that table(s).
Being very familiar with MS SQL Servers's WITH(NOLOCK) hint, I see LOCK ROW FOR ACCESS as its equivalent. However, INSERT or UPDATE statements do not allow using LOCK ROW FOR ACCESS (it is not clear for me why this fails, since it should apply for table(s) the statement selects from, not to the one I insert into):
-- this works
LOCK ROW FOR ACCESS
SELECT Cols
FROM Table
-- this does not work
LOCK ROW FOR ACCESS
INSERT INTO SomeVolatile
SELECT Cols
FROM PersistentTable
I have seen that LOCKING TABLE ... FOR ACCESS can be used, but it is unclear if it fits my need (NOLOCK equivalent - do not block writes).
Question: What hint should I use to minimize writes delaying when selecting within an INSERT statement?
You can't use LOCK ROW FOR ACCESS on an INSERT-SELECT statement. The INSERT statement will put a WRITE lock on the table to which it's writing and a READ lock on the tables from which it's selecting.
If it's absolutely imperative that you get LOCK ROW FOR ACCESS on the INSERT-SELECT, then consider creating a view like:
CREATE VIEW tmpView_PersistentTable AS
LOCK ROW FOR ACCESS
SELECT Cols FROM PersistentTable;
And then perform your INSERT-SELECT from the view:
INSERT INTO SomeVolatile
SELECT Cols FROM tmpView_PersistentTable;
Not a direct answer, but it's always been my understanding that this is one of the reasons your users/applications/etc should access data through views. Views lock for access, which does not prevent inserts/updates. Selecting from a table uses read locks, which will prevent inserts/updates.
The downside is with access locks, the possibility for dirty reads exists.
Change your query as below and you should be good.
LOCKING TABLE PersistentTable FOR ACCESS
INSERT INTO SomeVolatile
SELECT Cols
FROM PersistentTable ;
Given a table:
CREATE TABLE Foo(
Id INTEGER PRIMARY KEY AUTOINCREMENT,
Name TEXT
);
How can I return the ids of the multiple rows inserted at the same time using:
INSERT INTO Foo (Name) VALUES
('A'),
('B'),
('C');
I am aware of last_insert_rowid() but I have not found any examples of using it for multiple rows.
What I am trying to achieve can bee seen in this SQL Server example:
DECLARE #InsertedRows AS TABLE (Id BIGINT);
INSERT INTO [Foo] (Name) OUTPUT Inserted.Id INTO #InsertedRows VALUES
('A'),
('B'),
('C');
SELECT Id FROM #InsertedRows;
Any help is very much appreciated.
This is not possible. If you want to get three values, you have to execute three INSERT statements.
Given SQLite3 locking:
An EXCLUSIVE lock is needed in order to write to the database file. Only one EXCLUSIVE lock is allowed on the file and no other locks of any kind are allowed to coexist with an EXCLUSIVE lock. In order to maximize concurrency, SQLite works to minimize the amount of time that EXCLUSIVE locks are held.
And how Last Insert Rowid works:
...returns the rowid of the most recent successful INSERT into a rowid table or virtual table on database connection D.
It should be safe to assume that while a writer executes its batch INSERT to a ROWID-table there can be no other writer to make the generated primary keys non-consequent. Thus the insert primary keys are [lastrowid - rowcount + 1, lastrowid]. Or in Python SQLite3 API:
cursor.execute(...) # multi-VALUE INSERT
assert cursor.rowcount == len(values)
lastrowids = range(cursor.lastrowid - cursor.rowcount + 1, cursor.lastrowid + 1)
In normal circumstances when you don't mix provided and expected-to-be-generated keys or as AUTOINCREMENT-mode documentation states:
The normal ROWID selection algorithm described above will generate monotonically increasing unique ROWIDs as long as you never use the maximum ROWID value and you never delete the entry in the table with the largest ROWID.
The above should work as expected.
This Python script can be used to test correctness of the above for multi-threaded and multi-process setup.
Other databases
For instance, MySQL InnoDB (at least in default innodb_autoinc_lock_mode = 1 "consecutive" lock mode) works in similar way (though obviously in much more concurrent conditions) and guarantees that inserted PKs can be inferred from lastrowid:
"Simple inserts" (for which the number of rows to be inserted is known in advance) avoid table-level AUTO-INC locks by obtaining the required number of auto-increment values under the control of a mutex (a light-weight lock) that is only held for the duration of the allocation process, not until the statement completes
I am loading data into my common table DB_TBLS.ACCOUNT in parralel for three process. To avoid blocking i have created view as below on top of this base table.
But again i am getting blocking on the base table.
Replace View DB_VWS.S_ACCOUNT as locking row for access
Select * From DB_TBLS.ACCOUNT where id = 1 ;
Replace View DB_VWS.T_ACCOUNT as locking row for access
Select * From DB_TBLS.ACCOUNT where id = 2 ;
Replace View DB_VWS.R_ACCOUNT as locking row for access
Select * From DB_TBLS.ACCOUNT where id = 3 ;
Can anyone please help me in loading data in parallel in common table ?
Stream Operator in the Teradata Parallel Transporter (TPT) utility provides the functionality which you require. According to TPT Reference Manual (14.10):
The Stream operator, a consumer operator, emulates the Teradata TPump utility to perform high-speed parallel Inserts, Updates, Deletes, and Upserts in a near-real-time to one or more empty or preexisting Teradata Database tables without locking target tables.