I use an app which creates this SQLite DB with this table:
CREATE TABLE expense_report (_id INTEGER PRIMARY KEY, ...)
And for some reason that _id (which is the ROWID) became invalid in that DB.
When I scan the table I see that the last rows got an _id which was already being used long ago:
1,2,3...,1137,1138,...,1147,1149,...,12263,12264,1138,...,1148
I highlighted above the ranges in which I see that I have the same _id for completely different rows (the rest of the values do not match at all).
And querying this DB usually gets me inaccurate results due to that. For instance:
SELECT
(SELECT MAX(_ID) FROM expense_report) DirectMax
, (SELECT MAX(_ID) FROM (SELECT _ID FROM expense_report ORDER BY _ID DESC)) RealMax;
| DirectMax | RealMax |
| 1148 | 12264 |
And inserting a new row into this table via DB Browser for SQLite also generates an _id of 1149 (instead of 12265), so the problem becomes worse if I keep using this DB.
Running PRAGMA QUICK_CHECK or PRAGMA INTEGRITY_CHECK show this error response:
*** in database main ***
On page 1598 at right child: Rowid 12268 out of order
And running VACUUM also detects the problem but doesn't seem to be able to fix it:
Execution finished with errors.
Result: UNIQUE constraint failed: expense_report._id
Anyone knows a way to fix these duplicate ROWID values?
Related
For example, let say DB has foreign key A.b_id -> B.id with SET NULL on delete.
If record with some B.id get deleted, all b_id references will be set to NULL.
But if A already contains record where A.b_id has value that is not in B.id (it was inserted without foreign keys support), is there a way to force SQLite DB check foreign keys and set to NULL such data?
In fact, in first place I'm solving an DB upgrading task.
On start app checks if internal DB (resource) has higher version than user DB.
If so it backups user DB, copies internal empty DB to user storage. Than turns off foreign keys support and fills new DB with data from backup, inserting automatically in loop table by table for all columns with same name. Turns on foreign keys support back.
Everything works fine, but if in some table in old DB there is no foreign key constrain previously, while new DB has one, the data will be inserted as is and link can point nowhere (possibly wrong links is unavoidable and not related to question).
Yes, I understand a way to insert without turning off foreign keys support, but it would need knowledge of tables dependencies order that I would like to avoid.
Thanks for any help in advance!
Although I don't know of a way that automatically will set to NULL all orphaned values of a column in a table that (should) reference another column in another table, there is a way to get a report of all these cases and then act accordingly.
This is the PRAGMA statement foreign_key_check:
PRAGMA schema.foreign_key_check;
or for a single table check:
PRAGMA schema.foreign_key_check(table-name);
From the documenation:
The foreign_key_check pragma checks the database, or the table called
"table-name", for foreign key constraints that are violated. The
foreign_key_check pragma returns one row output for each foreign key
violation. There are four columns in each result row. The first column
is the name of the table that contains the REFERENCES clause. The
second column is the rowid of the row that contains the invalid
REFERENCES clause, or NULL if the child table is a WITHOUT ROWID
table. The third column is the name of the table that is referred to.
The fourth column is the index of the specific foreign key constraint
that failed. The fourth column in the output of the foreign_key_check
pragma is the same integer as the first column in the output of the
foreign_key_list pragma. When a "table-name" is specified, the only
foreign key constraints checked are those created by REFERENCES
clauses in the CREATE TABLE statement for table-name.
Check a simplified demo of the way to use this PRAGMA statement, or its function counterpart pragma_foreign_key_check().
You can get a list of the rowids of all the problematic rows of each table.
In your case, you can execute an UPDATE statement that will set to NULL all the orphaned b_ids:
UPDATE A
SET b_id = NULL
WHERE rowid IN (SELECT rowid FROM pragma_foreign_key_check() WHERE "table" = 'A')
This also works in later versions of SQLite:
UPDATE A
SET b_id = NULL
WHERE rowid IN (SELECT rowid FROM pragma_foreign_key_check('A'))
but it does not seem to work up to SQLite 3.27.0
We have an old batch file which does only such statement:
Insert into table_a select * from table_b;
table a is bulk table with no index and constraint
after few years, with increments in record counts, this batch became slow
but suddenly for few days, we got this error every time we try to run the batch:
Ora-00604 error occured at recursive sql level 1 ora-02449 unique/primary keys in table referenced by foreign keys
our only option is to make chunks of data, and insert them part by part, which fixes the batch output, but the problem still exists
we are not dropping and table or object here
can you help us find the cause of problem?
I've checked database level triggers but there is no trigger for insert at database level
I'm facing a strange error. I have a 5.5.5-10.1.20-MariaDB install on my local mac (brew) and a 5.5.52-MariaDB on my prod server (centos7). My local DB content is a copy from my server DB. I've executed this query on local:
## CREATE DIRECT RELATION BETWEEN JOURNAL AND PUBLICATION
INSERT INTO journal_publication (journal_id, `publication_id`) (
select issues.journal_id as journal_id, publications.id as publication_id from issues
join publications on issues.id = publications.`issue_id`
where publications.id Not In (select distinct publication_id from journal_publication)
);
It works fine and takes only less than a second to execute.
Now when I try the exact same query on my prod server, the query is never ending and takes all CPUs. Moreover, I've tried to EXPLAIN the query, it works fine on my local:
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY issues index PRIMARY issues_journal_id_foreign 5 NULL 70993 Using index; Using temporary
1 PRIMARY publications ref publications_issue_id_foreign publications_issue_id_foreign 5 pubpeer.issues.id 1 Using where; Using index
2 MATERIALIZED journal_publication index NULL PRIMARY 8 NULL 143926 Using index
Whereas the same query on my Prod returns an error:
You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'INSERT INTO journal_publication (journal_id, `publication_id`)
(select issues.j' at line 2
Again, the content of the two DBs are identical, primary keys and indexes are set equally. For the record, when I try and execute this query:
select issues.journal_id as journal_id, publications.id as publication_id from issues
join publications on issues.id = publications.`issue_id`
where publications.id Not In (select distinct publication_id from journal_publication;
either on local or prod takes only a second.
Have you got any clue or process I could follow to help me understand these differences?
Thanks.
Xavier
MariaDB server versions < 10.0 only support EXPLAIN SELECT
MariaDB server versions >= 10.0 support additionally EXPLAIN
UPDATE, EXPLAIN INSERT and EXPLAIN DELETE
Please note that the version string 5.5.5-10.1.20-MariaDB means MariaDB 10.1.20, the 5.5.5 prefix is required since MySQL replication would break, since it supports only 1 digit numbers for the major version.
See also EXPLAIN UDATE/INSERT/DELETE in MySQL and MariaDB
A strange thing, that I don't know the cause, is happenning when trying to collect results from a db2 database.
The query is the following:
SELECT
COUNT(*)
FROM
MYSCHEMA.TABLE1 T1
WHERE
NOT EXISTS (
SELECT
*
FROM
MYSCHEMA.TABLE2 T2
WHERE
T2.PRIMARY_KEY_PART_1 = T1.PRIMARY_KEY_PART_2
AND T2.PRIMARY_KEY_PART_2 = T1.PRIMARY_KEY_PART_2
)
It is a very simple one.
The strange thing is, this same query, if I change COUNT(*) to * I will get 8 results and using COUNT(*) I will get only 2. The process was repeated some more times and the strange result is still continuing.
At this example, TABLE2 is a parent table of the TABLE1 where the primary key of the TABLE1 is PRIMARY_KEY_PART_1 and PRIMARY_KEY_PART_2, and the primary key of the TABLE2 is PRIMARY_KEY_PART_1, PRIMARY_KEY_PART_2 and PRIMARY_KEY_PART_3.
There's no foreign key between them (because they were legacy ones) and they have a huge amount of data.
The DB2 query SELECT VERSIONNUMBER FROM SYSIBM.SYSVERSIONS returns:
7020400
8020400
9010600
And the client used is SquirrelSQL 3.6 (without the rows limit marked).
So, what is the explanation to this strange result?
Without the details (including, at least, the exact Db2 version and DDL for both tables and their indexes) it can be just anything, and even with that details only IBM support will be really able to say, what is the actual reason.
Generally this looks like damaged data (e.g. differences in index vs table data).
Worth to open the support case with IBM.
I am trying to get list of all tables/views (in other words all objects) where a particular field is referenced using the system or catalog tables. I am using the following query.
select *
from dba_col_comments
where column_name like('SXX_AXXX_%')
order by 1;
However, the output is volatile. When I repeatedly run the same query without any changes the output is varies. For instance, it produced 9300 records and then 9350 after a couple of minutes and then 9347 after a couple of minutes.
I am observing the same behaviour in Teradata as well.
My theory would be - in a production enironment temporary objects that are created are probably getting an entry in the system/catalog tables.
Any thoughts/directions?
In Teradata you will find that as global temporary tables are instantiated (referenced by an SQL statement) records should be added to the data dictionary table TVM. These records are then dropped after the session logs off leaving just the base table record associated with the original CREATE GLOBAL TEMPORARY TABLE statement that was submitted.
You can find these instances using the view DBC.AllTempTables.
In Teradata, volatile tables are not maintained within the data dictionary.
EDIT - Your mileage may vary but this should get you started on Teradata
SELECT D1.DatabaseNameI AS DatabaseName_
, T1.TVMNameI AS TableName_
, F1.FieldName AS ColumnName_
FROM "DBC".TVM T1
INNER JOIN
"DBC".Dbase D1
ON D1.DatabaseId = T1.DatabaseId
INNER JOIN
"DBC".TVFields F1
ON F1.DatabaseId = T1.DatabaseId
AND F1.TableId = T1.TVMId
WHERE F1.FieldName = 'MyColumn'
--AND D1.DatabaseNameI IN ('{Database1}', ... '{Database99}') -- Filter on databases
AND F1.FieldType in ('i', 'i1', 'i2', 'i8') -- Integer, ByteInt, SmallInt, BigInt
--AND T1.TableKind IN ('T') -- Optional Filter to just tables.
AND NOT EXISTS
(SELECT 'x'
FROM "DBC".TempTables TT1
WHERE Tt1.TableId = T1.TVMId
)
;