I've got a problem while creating (altering) foreign key.
I have two tables in my DB (created via flyway migrations):
connector (migration)
create table if not exists connector
(
id char(36) not null,
# other fields omitted
primary key (id)
) DEFAULT CHARACTER SET 'utf8mb4'
COLLATE 'utf8mb4_unicode_520_ci';
connector_preset (migration)
CREATE TABLE IF NOT EXISTS connector_preset
(
id CHAR(36) NOT NULL,
# other fields omitted
PRIMARY KEY (id)
);
I want to create a link between connector and connector_preset, so I created another migration like this:
ALTER IGNORE TABLE `connector`
# `connector_preset`.`id`
ADD COLUMN IF NOT EXISTS `preset_id` CHAR(36),
ADD CONSTRAINT `fk_connector_preset_id` FOREIGN KEY (`preset_id`) REFERENCES `connector_preset` (`id`);
but it fails with the following error:
SQL State : HY000
Error Code : 1005
Message : (conn=4) Can't create table `test`.`connector` (errno: 150 "Foreign key constraint is incorrectly formed")
Location : db/migration/...
Line : 34
Statement : ALTER IGNORE TABLE `connector`
# `connector_preset`.`id`
ADD COLUMN IF NOT EXISTS `preset_id` CHAR(36),
ADD CONSTRAINT `fk_connector_preset_id` FOREIGN KEY (`preset_id`) REFERENCES `connector_preset` (`id`),
The columns seem to be of the same type. Also, for some reason it works in local k8s cluster (10.3.29-MariaDB), but fails in integration tests (testcontainers, MariaDB 10.6.11). Also fails in GH Actions which use 10.3.29, which is strange since it's working locally.
UPD: If I set mariadb version in testcontainers to 10.3.29 - it still fails.
UPD: Tried altering connector_preset table to use the same charset and collation:
ALTER IGNORE TABLE `connector_preset` DEFAULT CHARACTER SET 'utf8mb4' COLLATE 'utf8mb4_unicode_520_ci';
Still doesn't work
Table connector was created with character set utf8mb4 and collation utf8mb4_unicode_520_ci, while connector_preset was created with servers default character set and collation, which obviously differs.
So you have to convert your table to the right character set.
Your attempt to change it with ALTER TABLE ... DEFAULT CHARACTER SET 'utf8mb4' sets the default character set for the table, but not for the columns. Instead you have to convert it with:
ALTER TABLE connector_preset CONVERT TO CHARSET utf8mb4 COLLATE utf8mb4_unicode_520_ci
Related
I have a huge table in a MariaDb (10.4.10-MariaDB-1:10.4.10+maria~bionic) and I am adding a new column using
alter table Appointment add column responsible_organization varchar(256);
The existing table is this:
CREATE TABLE `Appointment` (
`id` VARCHAR(256) NOT NULL,
`version` VARCHAR(24) NOT NULL,
`repetition_ref` VARCHAR(256) NULL DEFAULT NULL,
`type` VARCHAR(256) NULL DEFAULT NULL,
`comment` VARCHAR(2048) NULL DEFAULT NULL,
`description` VARCHAR(2048) NULL DEFAULT NULL,
`end` DATETIME NULL DEFAULT NULL,
`start` DATETIME NULL DEFAULT NULL,
`status` VARCHAR(256) NULL DEFAULT NULL,
`statuschangedate` DATETIME NULL DEFAULT NULL,
`deliverystatus` VARCHAR(256) NULL DEFAULT NULL,
`reasoncancelled` VARCHAR(256) NULL DEFAULT NULL,
`visit_type` VARCHAR(256) NULL DEFAULT NULL,
`modified_db_time` TIMESTAMP(3) NOT NULL DEFAULT current_timestamp(3) ON UPDATE current_timestamp(3),
`markedasdeleted` DATETIME NULL DEFAULT NULL,
PRIMARY KEY (`id`, `version`),
INDEX `FKc4f6e4y3ftaya162pwf7v4uj4` (`deliverystatus`),
INDEX `FKeuhxsh83rlweegn404penommb` (`reasoncancelled`),
INDEX `FK5xmurewn61wf4n3of5yx2nsmg` (`visit_type`),
INDEX `modified_db_time` (`modified_db_time`),
CONSTRAINT `FK5xmurewn61wf4n3of5yx2nsmg` FOREIGN KEY (`visittype`) REFERENCES `Coding` (`id`),
CONSTRAINT `FKc4f6e4y3ftaya162pwf7v4uj4` FOREIGN KEY (`deliverystatus`) REFERENCES `CodeableConcept` (`id`),
CONSTRAINT `FKeuhxsh83rlweegn404penommb` FOREIGN KEY (`reasoncancelled`) REFERENCES `CodingDt` (`id`)
)
;
As far as I read the MariaDb documentation it should choose the most efficient algorithm if I don't specify any. I would expect it to use INPLACE at the minimum. But when I run it i can see in the process list that it is running with the state "Copy to tmp table". So this is the COPY algorithm right?
I then tried to force it to use INSTANT as sugested by #o-jones. That gave me this output:
MariaDB [mydb]> alter table Appointment add column responsibleorganisation varchar(256), ALGORITHM=INSTANT;
ERROR 1846 (0A000): ALGORITHM=INSTANT is not supported. Reason: Cannot change column type. Try ALGORITHM=COPY
Weird, since I am adding a column.
I am wondering if it has something to do with the table being created on an older version of MariaDb and not having been rebuilt recently. I have found references to this being an issue for tables with old style temporal columns.
The variables old_alter_table and alter_algorithm both have the value DEFAULT
My table has 110+ million rows, so I would have liked to find a way to optimize this.
Any ideas?
MariaDB Server 10.4.10 was released over 2 years ago, in November 2019. Is this repeatable with a more recent version? There have been some fixes to ALTER TABLE since then.
If the problem is repeatable with the latest version in the 10.4 series, I would suggest that you file a bug report at https://jira.mariadb.org with a minimal reproducible test case (CREATE TABLE and ALTER TABLE statements).
With a more recent version, the answer could have been that MDEV-20590 introduced a way to disable instant operations that involve changing the data file format. SET GLOBAL innodb_instant_alter_column_allowed=never; would make ADD COLUMN always rebuild the table, like it did before MariaDB Server 10.3.
I have seen this in cases where a database was created in an earlier version of MariaDB. Sometimes the storage format for some columns has changed, but the upgrade did not rewrite all of the data. Instead it waits until certain schema changes are made and then it modifies that underlying data format. In my cases it was because some of the columns were date/datetime and the temporal storage format had changed in 10.3, but many of my tables were still in the old format. Any schema change I made in 10.4 wanted to update the format for all of those columns, which was a pain for tables with many millions of rows.
If you rebuild the table first using "ALTER TABLE tab_name ENGINE = InnoDB;" then the column add would be able to be added "instantly". Although the alter will take as long as the non-instant column add would. However, there are circumstances where doing the processes separately might be advantageous.
In my case I had to do a manual table rebuild because I could not allow the live table to be locked for as long as the rebuild would take. So I wrote a stored procedure to take the table and create a copy of it gradually, and keep the copy up-to-date. The copy process was throttled to avoid putting to much strain on the db and then once the copy was ready I was able to swap the old and new tables quickly with renames so that the down time was measured in seconds instead of hours. Once the new table was in place I was able to do instant ddl operations on it like normal.
I recently duplicated an existing MariaDB table (for use on Amazon RDS). The two tables are basically identical, data included, but I noticed the new version was missing all the unique constraints (kept the primary keys just fine). When I tried to add back the uniques, I got an error:
BLOB/TEXT column 'url' used in key specification without a key length
Others have had this problem, and people have claimed that MySQL/MariaDB simply doesn't let you make text columns unique. But my original MariaDB database (on my local machine) has plenty of text columns with unique keys, so that can't be it. I tried their solution anyway (switching to varchar) but MariaDB wouldn't let me do that either because my data has too many characters. Any ideas? Many thanks.
MariaDB 10.4 supports a UNIQUE KEY on a TEXT column because it automatically converts the index to a HASH index:
mysql> create table t (id serial primary key, t text, unique key(t));
Query OK, 0 rows affected (0.01 sec)
mysql> show create table t\G
*************************** 1. row ***************************
Table: t
Create Table: CREATE TABLE `t` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`t` text DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `t` (`t`) USING HASH
) ENGINE=InnoDB DEFAULT CHARSET=latin1
Unfortunately, Amazon RDS for MariaDB does not support MariaDB 10.4 yet (as of 2020-03-19), according to https://aws.amazon.com/rds/mariadb/:
Amazon RDS supports MariaDB Server versions 10.0, 10.1, 10.2, and 10.3 which means that the code, applications, and tools you already use today can be used with Amazon RDS.
The HASH index feature is not supported in MariaDB 10.3 or earlier:
mysql> create table t (id serial primary key, t text, unique key(t));
ERROR 1170 (42000): BLOB/TEXT column 't' used in key specification without a key length
You can create a unique key on a prefix of the text column:
mysql> create table t (id serial primary key, t text, unique key(t(1000)));
Query OK, 0 rows affected (0.01 sec)
mysql> show create table t\G
*************************** 1. row ***************************
Table: t
Create Table: CREATE TABLE `t` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`t` text DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `t` (`t`(1000))
) ENGINE=InnoDB DEFAULT CHARSET=latin1
This means the unique key will enforce uniqueness in the first 1000 characters. You won't be allowed to store two rows with string that have the same first 1000 characters, but differ in the 1001st character or later.
Does SQLite indeed have a limitation that it is not possible to retrieve the name of a foreign key? I am asking because I couldn't find this limitation mentioned anywhere in their documentation.
For example, I run the following script:
CREATE TABLE
users (
id INTEGER NOT NULL PRIMARY KEY,
first_name TEXT NOT NULL,
last_name TEXT NOT NULL
) ;
CREATE TABLE
orders (
id INTEGER NOT NULL PRIMARY KEY,
user_id INTEGER NOT NULL,
CONSTRAINT fk_users FOREIGN KEY (user_id) REFERENCES users(id)
) ;
Now I would like to check that the key "fk_users" was created indeed, so I run the following PRAGMA:
PRAGMA foreign_key_list(orders);
I would expect to see the name of my foreign key in the first column, but I am seeing some "0" value instead. Moreover, if I create multiple foreign keys with custom names, they are all called either "0" or "1".
Is this indeed a limitation of SQLite, or am I missing something?
There is no mechanism to extract the constraint name.
The table sqlite_master stores a CREATE command in the column "sql". You could query that command and do some parsing to extract the name of the foreign key. An example for a combined foreign key that works for me:
SELECT sql FROM sqlite_master WHERE name = 'song'
yields
CREATE TABLE "song" (
"songid" INTEGER,
"songartist" TEXT,
"songalbum" TEXT,
"songname" TEXT,
CONSTRAINT "fk__song_album" FOREIGN KEY ("songartist", "songalbum") REFERENCES "album" ("albumartist", "albumname")
)
and contains the name "fk__song_album" of the foreign key.
If one alters the foreign key with a query, the content of the sql column is modified/updated:
The text in the sqlite_master.sql column is a copy of the original CREATE statement text that created the object, except normalized as described above and as modified by subsequent ALTER TABLE statements. The sqlite_master.sql is NULL for the internal indexes that are automatically created by UNIQUE or PRIMARY KEY constraints.
https://www.sqlite.org/fileformat2.html
Extra tip:
In order to see the foreign key information in Navicat (Lite) ... right click on a table and choose "Design table". Then select the foreign keys tab.
I am trying to use the Federated engine of MariaDB 10.1.12 to create tables that are based on tables in a remote database. Following the MariaDB instructions about how to use the FederatedX implementation, in database db1 I create a table as
CREATE TABLE test_table (
id int(20) NOT NULL auto_increment,
name varchar(32) NOT NULL default '',
other int(20) NOT NULL default '0',
PRIMARY KEY (id),
KEY name (name),
KEY other_key (other))
DEFAULT CHARSET=latin1;
Now when I want to see this table in a second database db2 using the Federated engine, I can issue
CREATE TABLE test_table (
id int(20) NOT NULL auto_increment,
name varchar(32) NOT NULL default '',
other int(20) NOT NULL default '0',
PRIMARY KEY (id),
KEY name (name),
KEY other_key (other)
) ENGINE=FEDERATED
DEFAULT CHARSET=latin1
CONNECTION='mysql://user_x:pass_y#localhost/db1/test_table';
All this is copied from the MariaDB documentation and works well. However, if I try to create the table without explicitly duplicating the definition of the table structure - an example given in the same documentation
CREATE TABLE test_table ENGINE=FEDERATED DEFAULT CHARSET=latin1
CONNECTION='mysql://user_x:pass_y#localhost/db1/test_table';
MariaDB responds with an error
ERROR 1113 (42000): A table must have at least 1 column
Am I missing something or is it not possible to use federated tables without specifying the individual columns?
It seems that with the standard installation of MariaDB on Ubuntu 14.04, the old Federated engine is active, not the new FederatedX variant. Only the latter supports auto-discovery of columns. In order to correct this, I took the following steps:
uninstall soname 'ha_federated';
to remove the federated plugin. After a server restart, the correct one is loaded with
install plugin federated soname 'ha_federatedx';
This loads the new FederatedX implementation supporting auto-discovery of columns. To view the installed engines, one can list them with
show engines;
If all is correct, there should be this line in the output:
| FEDERATED | YES | FederatedX pluggable storage engine |
what'S wrong with the following statement?
ALTER TABLE submittedForecast
ADD CONSTRAINT FOREIGN KEY (data) REFERENCES blobs (id);
The error message I am getting is
Can't create table `fcdemo`.`#sql-664_b` (errno: 150 "Foreign key constraint is incorrectly formed")
This works for me on MariaDB 10.1.8:
CREATE TABLE `submittedforecast` (
`id` INT(11) NOT NULL,
`data` INT(11) NOT NULL,
PRIMARY KEY (`id`),
INDEX `data` (`data`)
) ENGINE=InnoDB;
CREATE TABLE `blobs` (
`id` INT(11) NOT NULL,
`content` BLOB NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB;
ALTER TABLE submittedForecast
ADD CONSTRAINT FOREIGN KEY (data) REFERENCES blobs (id);
Can you give your MariaDB version number and a complete example including the CREATE TABLE statements for submittedForecast and blobs?
No idea if you already solved this but make sure both engines and collations match between tables (e.g: latin1 to latin1 and InnoDB to InnoDB).
I was having the same issue and by switching these I managed to get it working.
I was getting the same issue and after looking at my table structure, found out that my child table's foreign key clause was not referencing the primary key of parent table. Once i changed it to reference to primary key of parent table, the error was gone.
I had the same error and is actually pretty easy to solve, you have name the constraint, something like this should do:
ALTER TABLE submittedForecast ADD CONSTRAINT `fk_submittedForecast`
FOREIGN KEY (data) REFERENCES blobs (id)
If you would like more cohesion also add at the end of the query
ON DELETE CASCADE ON UPDATE RESTRICT
This could also be a different fields error so check if the table key and the foreign key are of the same type and have the same atributes.
I had the same problem, too. When checking the definition of the fields, I noticed that one field was defined as INT and the other as BIGINT. After changing the BIGINT type to INT, I was able to create my foreign key.