I'm trying to debug a deadlock on Redshift:
SQL Execution failed ... deadlock detected
DETAIL: Process 7679 waits for AccessExclusiveLock on relation 307602 of database 108260; blocked by process 7706.
Process 7706 waits for AccessShareLock on relation 307569 of database 108260; blocked by process 7679.
Is there a sql query to get a description for process ids 7679 and 7706?
select * from stl_query where pid=XXX
This will give you the query txt which will help you identify your query.
You can also query stv_locks to check is there are any current updates in the database, and str_tr_conflict will display all the lock conflict on the table.
Related
I am trying to get some useful information from airflow backend. I need to get following details
How many times a particular job failed.
which task has failed over and over.
The problem is all our task has dependency on their upstream, and so when it fails, we fix the issue and mark it as success. This changes status in database as well. Is there a place I can get historical records?
following query shows which task failed. However if I mark it as success from UI, status is updated in database as well. And I have no way to show if this was failed.
select * from dag_run where dag_id='test_spark'
Currently, there is no native way but you can check log table -- it adds a record whenever there is action via CLI or UI.
On the linux server, where our web app runs, we also have a small app that uses
sqlite (it is written in c).
For performing database actions we use the following commands:
sqlite3_prepare_v2
sqlite3_bind_text or sqlite3_bind_int
sqlite3_step
sqlite3_finalize
Every now and then there was a concurrency situation and I got the following error:
database is locked
So I thought: "This happens when one process writes a certain record and the
other one is trying to read exactly the same record."
So after every step-command, where this collision could be, I checked for this error. When it happended, I waited a few milliseconds and the tried again.
But the sqlite error "database is locked" still occurred.
So I changed every step command and the code lines after it. Somehow I thought that this "database is locked" error could only occur with the step command.
But the error kept coming.
My question is now:
Do I have to check after any sqlite3 command for "error_code ==5" (database is locked)?
Thanks alot in advance
If you're receiving error code 5 (busy) you can limit this by using an immediate transaction. If you're able to begin an immediate transaction, SQLite guarantees that you won't receive a busy error until you commit.
Also note that SQLite doesn't have row-level locking. The entire database is locked. Using a WAL journal, you can one writer and multiple readers. With other journaling methods, you can have either one writer, or multiple readers, but not both simultaneously.
SQLite Documentation on 'SQLITE_BUSY'
For a particular requirement, I will have to iterate through a list of 50000 records and insert them into database. The requirement is that if any one of the 50000 records fail, all the other records should be rollback. And hence we did not involve any commit in the processing. But this resulted in the following error:
[2/1/16 14:01:47:939 CST] 000000be SystemOut O ERROR
org.springframework.jdbc.UncategorizedSQLException:
PreparedStatementCallback; uncategorized SQLException for SQL [INSERT
INTO ...) VALUES (...)]; SQL state [null]; error code [0]; Current
thread has not commited in more than [120] seconds and may incur
unwanted blocking locks. Please refactor code to commit more
frequently.
Now, when we implemented batching - We are using PreparedStatement.executeBatch() method to insert data in batches of 100, the above error doesn't arise. The autoCommit is set to true by default for the batching, so a commit happens after every batch execution.
Could anyone suggest how we can handle the rollback mechanism in the above case. If 50th batch execution fails, then we want all the previous 49 batch executions to be reverted. We are using Spring Data/JDBC, Oracle 11g database, WebSphere application server. I have read somewhere that the above 120 seconds timeout for commit can also be configured in the JTA settings of WebSphere. Is it so? Please suggest any alternatives or other possible solutions.
Thank you in advance.
You must set autocommit to false and only commit at the end if all your batches executed successfully.
Problem Description:
1.There is Biztalk Application receiving formatted/zipped data file containing > 2 million data records.
2.Created pipeline component that process file and 'de batching' these 2 million records of data into smaller slice-messages with ~2000 records each .
3.Slice-messages are being sent to SQL port and processed by stored procedure.Slice-messages contains filename and batch id.
Questions:
A.What would be the best way to know that all slice-messages received and processing of whole file completed on SQL side ?
B.Is there any way in biztalk port to say "do not send message of type B, until all messages of Type A have been send" (messages priority)?
Here are possible solutions I've tried :
S1.Add specific 'end of file' tags to end of last slice-message saying that file is being processed and stored procedure will receive this part of message mark file as completed.
But because messages are being delivered asynchronously last message can be received on sql earlier that other messages and I will have false-competed event.
So this solution is only possible for "Ordered delivery ports" - but this type of ports have poor performance because sending only one message at a time.
S2.Transfer total records count into every slice-message and run count() sql statement after every slice-message received.
Because table where data is stored is super huge, even running count with filename as parameter takes time.
I'm wondering if there is better solution to know that all messages are being received ?
Have your pipeline component emit a "batch" message that contains the count of the records in the batch and some unique identifier that can link it back to the slice-messages records.
Have both the stored procedure that processes the slice-message and the batch message check to see if the batch total (if it exists yet for the slice-message process) matches the processed total, if they match, then you've finished processing them all.
Here's how I would approach this.
Load the 2MM records into a SQL Server table or tables by SSIS.
Drain the table at whatever rate give you an acceptable performance profile.
Delete records as they are processed (completed).
When no more records for "FILE001.txt" exist, SQL Server will return a flag
saying "FILE001.txt complete".
Do further processing.
When the staging table is empty, the Polling SP can either return nothing (the Adapter will silently ignore the response) or return a flag that says "nothing to do" and you handle that yourself.
Im getting a time out expired.
The timeout period elapsed prior to completion of the operation.
If I execute the same query in SQL server management its getting executed but trying to execute the query through the program does not succeed.
Where could I have gone wrong?
The query is very simple and its throwing exception.
The time out set is 90 sec.
Select isnull(max(voucherid),0)+1 from XXX
dbmanager.executequery(con,"uspXXX",parameter);
Rebuild your statistics and/or indexes on that table.
See this answer.