MariaDB won't let me make column unique - mariadb

I recently duplicated an existing MariaDB table (for use on Amazon RDS). The two tables are basically identical, data included, but I noticed the new version was missing all the unique constraints (kept the primary keys just fine). When I tried to add back the uniques, I got an error:
BLOB/TEXT column 'url' used in key specification without a key length
Others have had this problem, and people have claimed that MySQL/MariaDB simply doesn't let you make text columns unique. But my original MariaDB database (on my local machine) has plenty of text columns with unique keys, so that can't be it. I tried their solution anyway (switching to varchar) but MariaDB wouldn't let me do that either because my data has too many characters. Any ideas? Many thanks.

MariaDB 10.4 supports a UNIQUE KEY on a TEXT column because it automatically converts the index to a HASH index:
mysql> create table t (id serial primary key, t text, unique key(t));
Query OK, 0 rows affected (0.01 sec)
mysql> show create table t\G
*************************** 1. row ***************************
Table: t
Create Table: CREATE TABLE `t` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`t` text DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `t` (`t`) USING HASH
) ENGINE=InnoDB DEFAULT CHARSET=latin1
Unfortunately, Amazon RDS for MariaDB does not support MariaDB 10.4 yet (as of 2020-03-19), according to https://aws.amazon.com/rds/mariadb/:
Amazon RDS supports MariaDB Server versions 10.0, 10.1, 10.2, and 10.3 which means that the code, applications, and tools you already use today can be used with Amazon RDS.
The HASH index feature is not supported in MariaDB 10.3 or earlier:
mysql> create table t (id serial primary key, t text, unique key(t));
ERROR 1170 (42000): BLOB/TEXT column 't' used in key specification without a key length
You can create a unique key on a prefix of the text column:
mysql> create table t (id serial primary key, t text, unique key(t(1000)));
Query OK, 0 rows affected (0.01 sec)
mysql> show create table t\G
*************************** 1. row ***************************
Table: t
Create Table: CREATE TABLE `t` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`t` text DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `t` (`t`(1000))
) ENGINE=InnoDB DEFAULT CHARSET=latin1
This means the unique key will enforce uniqueness in the first 1000 characters. You won't be allowed to store two rows with string that have the same first 1000 characters, but differ in the 1001st character or later.

Related

MariaDB constraint is incorrectly formed although columns are of the same type

I've got a problem while creating (altering) foreign key.
I have two tables in my DB (created via flyway migrations):
connector (migration)
create table if not exists connector
(
id char(36) not null,
# other fields omitted
primary key (id)
) DEFAULT CHARACTER SET 'utf8mb4'
COLLATE 'utf8mb4_unicode_520_ci';
connector_preset (migration)
CREATE TABLE IF NOT EXISTS connector_preset
(
id CHAR(36) NOT NULL,
# other fields omitted
PRIMARY KEY (id)
);
I want to create a link between connector and connector_preset, so I created another migration like this:
ALTER IGNORE TABLE `connector`
# `connector_preset`.`id`
ADD COLUMN IF NOT EXISTS `preset_id` CHAR(36),
ADD CONSTRAINT `fk_connector_preset_id` FOREIGN KEY (`preset_id`) REFERENCES `connector_preset` (`id`);
but it fails with the following error:
SQL State : HY000
Error Code : 1005
Message : (conn=4) Can't create table `test`.`connector` (errno: 150 "Foreign key constraint is incorrectly formed")
Location : db/migration/...
Line : 34
Statement : ALTER IGNORE TABLE `connector`
# `connector_preset`.`id`
ADD COLUMN IF NOT EXISTS `preset_id` CHAR(36),
ADD CONSTRAINT `fk_connector_preset_id` FOREIGN KEY (`preset_id`) REFERENCES `connector_preset` (`id`),
The columns seem to be of the same type. Also, for some reason it works in local k8s cluster (10.3.29-MariaDB), but fails in integration tests (testcontainers, MariaDB 10.6.11). Also fails in GH Actions which use 10.3.29, which is strange since it's working locally.
UPD: If I set mariadb version in testcontainers to 10.3.29 - it still fails.
UPD: Tried altering connector_preset table to use the same charset and collation:
ALTER IGNORE TABLE `connector_preset` DEFAULT CHARACTER SET 'utf8mb4' COLLATE 'utf8mb4_unicode_520_ci';
Still doesn't work
Table connector was created with character set utf8mb4 and collation utf8mb4_unicode_520_ci, while connector_preset was created with servers default character set and collation, which obviously differs.
So you have to convert your table to the right character set.
Your attempt to change it with ALTER TABLE ... DEFAULT CHARACTER SET 'utf8mb4' sets the default character set for the table, but not for the columns. Instead you have to convert it with:
ALTER TABLE connector_preset CONVERT TO CHARSET utf8mb4 COLLATE utf8mb4_unicode_520_ci

10.4.24-MariaDB - Foreign key constraint is incorrectly formed

In person_addresses table i am getting error Foreign key constraint is incorrectly formed. My Version is 10.4.24-MariaDB
CREATE TABLE persons (
person_id int(11) NOT NULL AUTO_INCREMENT,
name varchar(255) NOT NULL,
email varchar(50) DEFAULT NULL,
phone int(11) DEFAULT NULL,
bio text DEFAULT NULL,
dob date DEFAULT NULL,
gender enum('Male','Female','Other') NOT NULL,
status tinyint(4) NOT NULL,
created_at timestamp NOT NULL DEFAULT current_timestamp(),
PRIMARY KEY (person_id),
UNIQUE KEY email (email),
UNIQUE KEY phone (phone)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
CREATE TABLE person_addresses (
person_id int(11) NOT NULL ,
address text NOT NULL,
created_at timestamp NOT NULL DEFAULT current_timestamp(),
UNIQUE KEY uc_person_address (person_id,address),
CONSTRAINT fk_person FOREIGN KEY (person_id) REFERENCES persons(person_id) ON UPDATE CASCADE ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
More Detailed Error
------------------------
LATEST FOREIGN KEY ERROR
------------------------
2023-01-11 12:41:52 0x421c Error in foreign key constraint of table `test`.`person_addresses`:
There is no index in table ```test``.``person_addresses``` where the columns appear
as the first columns. Constraint:
FOREIGN KEY (person_id) REFERENCES persons (person_id) ON UPDATE CASCADE ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
Please refer to https://mariadb.com/kb/en/library/foreign-keys/ for correct foreign key definition.Create table `test`.`person_addresses` with foreign key constraint failed. There is no index in the referenced table where the referenced columns appear as the first columns near ' FOREIGN KEY (person_id) REFERENCES persons (person_id) ON UPDATE CASCADE ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4'.
I think I found a matching bug report in MariaDB: https://jira.mariadb.org/browse/MDEV-29717
The issue is that if the index required for the foreign key is very large, the index is created automatically. But this fails if that index is larger than the index size limit for the storage engine (this should be 3072 bytes in the version of MariaDB you are using, because it defaults to DYNAMIC row format with innodb_large_prefix=on). The text type in your index is too large to be indexed without defining an index prefix. But foreign keys can't use indexes defined with index prefixes.
Unfortunately, because foreign keys are implemented deep in the storage engine, there is little opportunity for a more informative error message to be revealed. This is a problem with the implementation of pluggable storage engines.
The solution in your case should be to change the address column type. It cannot be text, it can only be a varchar that is not too long for the index prefix length limit.
For example, I tested with MariaDB 10.4. https://dbfiddle.uk/5jTX8iFt It fails if address is text, and it works fine if address is varchar(255). I did not test other lengths, I'll leave that to you.
I doubt you need text for an address anyway. Does anyone have an address that requires 64KB?

increase the max character limit for the index keys in mariaDB

On Nextcloud/social we have this problem:
Primary index name on "oc_social_a2_cache_documts" is too long.
see: https://github.com/nextcloud/social/issues/850
Is there a way to increase the limit for the character length of the index?
Update:
The table structure is
REATE TABLE `oc_social_a2_cache_documts` (
`id_prim` varchar(128) COLLATE utf8_bin NOT NULL,
`id` varchar(1000) COLLATE utf8_bin DEFAULT NULL
PRIMARY KEY (`id_prim`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
It works, If I rename the table to a shorter name with one character less: just oc_social_a2_cache_docums
But that is not a solution here, since it seems to have worked before in older versions I am not sure, how the problem could only occur now with the latest update???

JBPM issue with Maria DB Galera were Primary key mandatory

JBPM issue with Maria DB Galera were Primary key mandatory.
Some tables in JBPM db schema have no primary key.
If I add a primary key column along with them what will be the impact?
Is there nay other way I can get ride of this problem?
Currently we have Mariadb as the only database option to use.
create table EventTypes (
InstanceId bigint not null,
element varchar(255)
) ENGINE=InnoDB;
create table PeopleAssignments_PotOwners (
task_id bigint not null,
entity_id varchar(255) not null
) ENGINE=InnoDB;
Source for MariaDB primary Key mandatory:
mariadb-galera-cluster-known-limitations
Please help.
PeopleAssignments_PotOwners looks like a many:many mapping table between tasks and entities?? If so, then the 'natural' PRIMARY KEY would be
PRIMARY KEY(task_id, entity_id)
(in either order).
Perhaps ditto for the other table?
More discussion of efficiency in many:many tables: http://mysql.rjweb.org/doc.php/index_cookbook_mysql#many_to_many_mapping_table
If you don't have a 'natural' primary key composed of one (or more) columns, add
id INT UNSIGNED AUTO_INCREMENT NOT NULL PRIMARY KEY

How do FK:PK relations work in Sqlite?

I am using the DB Browser for SQLite to try and figure this out. I've opened Northwind.sqlite and in it it shows me the following for a table:
CREATE TABLE `Order Details` (
`OrderID` int,
`ProductID` int,
`UnitPrice` float ( 26 ),
`Quantity` int,
`Discount` float ( 13 ),
PRIMARY KEY(`OrderID`,`ProductID`)
);
However, in the Sql Server Northwind OrderID and ProductID are foreign keys, not primary keys. Does this work differently in SQLite? And if so, how do the relationships work?
thanks - dave
The above will create a table that has no FOREIGN keys but 2 indexes.
One a largely hidden index according to rowid.
The other, PRIMARY KEY(OrderID,ProductID) will be an index according to the combination of OrderId and ProductID.
some things about rowid (aka id)
rowid is an automatically created column called rowid (it can also be referenced using oid or rowid (case independent)) and if present is really the primary key.
rowid will be a unique signed integer using up to 64 bits. The lowest value and also the first value will be 1, the highest value being 9223372036854775807.
In later versions of SQLite 3.8.2 on the WITHOUT ROWID keyword was added to allow suppression of the rowid column/index (your Order Details table may benefit being a without rowid table).
if a column is defined with the type INTEGER PRIMARY KEY or INTEGER PRIMARY KEY AUTOINCREMENT then that column (there can only be 1 such column per table) is an alias of for the rowid column.
AUTOINCREMENT introduces a rule that when inserting a row the rowid must be greater than any that exist or existed.
It DOES NOT guarantee that the rowid will monotonically increase, although generally the id will (even without AUTOINCREMENT (perhaps the most misused/misunderstood keyword in SQLite)).
Without AUTOINCREMENT SQlite may find a lower rowid and use that, but not until a rowid of 9223372036854775807 has been reached.
AUTOINCREMENT, if a rowid of 9223372036854775807 has been reached will is an SQLITE_FULL exception.
AUTOINCREMENT results in overheads (e.q. a table named sqlite_sequence is then maintained recording the highest given sequence number). The documentation recommends that it not be used unless required, which is rarely the case.
Some limited testing I did resulted in an 8-12% greater processing time for AUTOINCREMENT. What are the overheads of using AUTOINCREMENT for SQLite on Android?
For more about rowid see SQLite Autoincrement and also Clustered Indexes and the WITHOUT ROWID Optimization
Coding PRIMARY KEY (if not on an INTEGER column i.e. not an alias of rowid) implies a UNIQUE constraint. It is not saying/checking that the value or any of the values in a clustered index exists in any other table.
Note null is not considered to be the same value, so in your Order Details table it is possible to have any combination of the values as null.
Coding a FOREIGN KEY introduces a constraint that the referenced value(s) must exist in the respective table/column. Additionally :-
Usually, the parent key of a foreign key constraint is the primary key
of the parent table. If they are not the primary key, then the parent
key columns must be collectively subject to a UNIQUE constraint or
have a UNIQUE index. If the parent key columns have a UNIQUE index,
then that index must use the collation sequences that are specified in
the CREATE TABLE statement for the parent table.
SQLite Foreign Key Support
Considering all of this you may want to do make some changes to the Order Details table :-
You could make it a WITHOUT ROWID table.
You could make both the OrderID and the ProductID columns NOT NULL.
You could add FOREIGN KEY's to both the OrderID and the ProductID columns.
So perhaps you could have :-
CREATE TABLE `Order Details` (
`OrderID` int NOT NULL REFERENCES `Orders` (`OrderId`), -- ADDED NOT NULL and FKEY
`ProductID` int NOT NULL REFERENCES `Products`(`ProductId`) , -- ADDED NOT NULL and FKEY
`UnitPrice` float ( 26 ),
`Quantity` int,
`Discount` float ( 13 ),
PRIMARY KEY(`OrderID`,`ProductID`)
)
WITHOUT ROWID -- ADDED WITHOUT ROWID
;
The above uses column constraints
Alternately, utilising TABLE constraints, you could do :-
CREATE TABLE `Order Details` (
`OrderID` int NOT NULL, -- ADDED NOT NULL
`ProductID` int NOT NULL, -- ADDED NOT NULL
`UnitPrice` float ( 26 ),
`Quantity` int,
`Discount` float ( 13 ),
PRIMARY KEY(`OrderID`,`ProductID`),
FOREIGN KEY (`OrderId`) REFERENCES `Orders`(`OrderId`), -- ADDED FKEY AS TABLE CONSTRAINT
FOREIGN KEY (`ProductID`) REFERENCES `Products`(`ProductID`) -- ADDED FKEY AS TABLE CONSTRAINT
)
WITHOUT ROWID -- ADDED WITHOUT ROWID
;
Both have the same outcome, the only difference being where the FOREIGN KEY constraints are defined.
Both the above assumes that the referenced tables are Orders and Products.

Resources