I have about 100 million texts(text length 4500~5000). I'm going to put them in mariadb, and the query I use is as follows.
SELECT TT_TEXT FROM TEXT_TABLE WHERE TT_HASH = UNHEX('MD5 HASH');
Which perfomance would be better for TEXT_TABLE1 or TEXT_TABLE2? The capacity of the database doesn't matter. Just the select perfomance.
# TABLE1
CREATE TABLE `TEXT_TABLE1` (
`TT_INDEX` INT UNSIGNED NOT NULL AUTO_INCREMENT COMMENT 'Index',
`TT_HASH` BINARY(16) NOT NULL COMMENT 'MD5 hash for `TT_TEXT`',
`TT_TEXT` TEXT(5000) NOT NULL COMMENT 'Text for 4500~5000 length',
PRIMARY KEY (`TT_INDEX`),
UNIQUE KEY (`TT_HASH`)
) ENGINE=InnoDB COMMENT 'Text table';
vs
# TABLE2
CREATE TABLE `TEXT_TABLE2` (
`TT_INDEX` INT UNSIGNED NOT NULL AUTO_INCREMENT COMMENT 'Index',
`TT_HASH` BINARY(16) NOT NULL COMMENT 'MD5 hash for `TT_TEXT`',
`TT_TEXT1` CHAR(250) NOT NULL COMMENT 'Text for 4500~5000 length',
...
`TT_TEXT20` CHAR(250) NOT NULL COMMENT 'Text for 4500~5000 length',
PRIMARY KEY (`TT_INDEX`),
UNIQUE KEY (`TT_HASH`)
) ENGINE=InnoDB COMMENT 'Text table';
Related
I have the following MARIADB code. It's supposed to demonstrate:
Constructing tables using sequences for incrementing the ID.
Using a temporary table+join to INSERT data into a table, while incrementing the ID.
Procedure:
Sequence S1 and table T1 are created. T1_ID is incremented with S1
Sequence S2 and table T2 are created. T2_ID is incremented with S2
Table T1 is filled with data. All is fine.
Temporary table TEMP_T2 is created and filled with data. No ID in this table. Column T1_NAME is a cross reference to SHORT_NAME in table T1.
The T1_ID is introduced into table TEMP_T2 with a join. The result of this SELECT is inserted into T2. Here, the sequence S2 should auto-increment T2_ID.
For some reason, at the end, T2 looks like this:
T2_ID|T1_ID|NAME|
-----+-----+----+
2| 1|y |
4| 2|x |
6| 2|z |
Why was T2_ID double-incremented?
Thanks!
USE DB1;
SET FOREIGN_KEY_CHECKS = 0;
DROP SEQUENCE IF EXISTS `S2`;
DROP SEQUENCE IF EXISTS `S1`;
DROP TABLE IF EXISTS `T2`;
DROP TABLE IF EXISTS `T1`;
-- Create sequence S1 and able T1
CREATE SEQUENCE `S1` start with 1 minvalue 1 maxvalue 9223372036854775806 increment by 1 cache 1000 nocycle ENGINE=InnoDB;
SELECT SETVAL(`S1`, 1, 0);
CREATE TABLE `T1` (
`T1_ID` tinyint(4) NOT NULL DEFAULT nextval(`S1`),
`SHORT_NAME` varchar(10) NOT NULL,
PRIMARY KEY (`T1_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
-- Create sequence T2 and table T2
CREATE SEQUENCE `S2` start with 1 minvalue 1 maxvalue 9223372036854775806 increment by 1 cache 1000 nocycle ENGINE=InnoDB;
SELECT SETVAL(`S2`, 1, 0);
CREATE TABLE `T2` (
`T2_ID` int(11) NOT NULL DEFAULT nextval(`S2`),
`T1_ID` int(11) DEFAULT NULL,
`NAME` varchar(100) DEFAULT NULL COLLATE 'utf8mb3_bin',
PRIMARY KEY (`T2_ID`),
UNIQUE KEY `T2_NAME_UN` (`NAME`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
-- Load data into T1
DELETE FROM T1;
INSERT INTO T1(SHORT_NAME) VALUES
('a'),
('b'),
('c');
SELECT * FROM T1;
-- Create temporary file for joining with T1
DROP TABLE IF EXISTS `TEMP_T2`;
CREATE TEMPORARY TABLE `TEMP_T2` (
`T1_NAME` varchar(10) DEFAULT NULL,
`NAME` varchar(100) DEFAULT NULL,
UNIQUE KEY `T2_NAME_UN` (`NAME`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
DELETE FROM TEMP_T2 ;
-- Insert data into the temporary table
INSERT INTO TEMP_T2(T1_NAME,NAME) VALUES
('b','x'),
('a','y'),
('b','z');
SELECT * FROM TEMP_T2;
# Do a join with TEMP_T2 x T1 and insert into T2
INSERT INTO T2(T1_ID,NAME)
SELECT
t1.T1_ID ,
t2.NAME
FROM TEMP_T2 AS t2
INNER JOIN T1 AS t1
ON t2.T1_NAME =t1.SHORT_NAME ;
SELECT * FROM T2;
Thanks for the responses.
I'm using SEQUENCE rather than AUTO_INCREMENT because I was told that it is the more modern way. It also enables retrieving the last ID of any specific table.
It's strange that this should be a bug. It seems like really basic functionality. But so it is...
I've found this as a reported existing bug MDEV-29540 in INSERT ... SELECT as it pertains to sequences in default values of columns.
Because this bug is reported and fix, this problem is/will not occur in the 10.3.37, 10.4.27, 10.5.18, 10.6.11, 10.7.7, 10.8.6, 10.9.4, 10.10.2, 10.11.1 and later versions.
I need to retrieve events from table, with skills requirements and user engagements and skills.
For a project, I have :
an users table
a skills table with inheritance from others skills
an users_skills table with skills of users
an events table
an events_skills table with the quantity of a required skills for the events
an events_users table with users engagements on events
CREATE TABLE `users` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY,
`name` char(32) NOT NULL
);
CREATE TABLE `skills` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY,
`name` char(64) NOT NULL,
`parent_id` int(11) unsigned DEFAULT NULL,
FOREIGN KEY (`parent_id`) REFERENCES `skills` (`id`)
);
CREATE TABLE `users_skills` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY,
`user_id` int(11) unsigned NOT NULL,
`skill_id` int(11) unsigned NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users` (`id`),
FOREIGN KEY (`skill_id`) REFERENCES `skills` (`id`)
);
CREATE TABLE `events` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY,
`name` char(64) NOT NULL
);
CREATE TABLE `events_skills` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY,
`event_id` int(11) unsigned NOT NULL,
`skill_id` int(11) unsigned NOT NULL,
`quantity` smallint unsigned NOT NULL,
FOREIGN KEY (`event_id`) REFERENCES `events` (`id`),
FOREIGN KEY (`skill_id`) REFERENCES `skills` (`id`)
);
CREATE TABLE `events_users` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY,
`event_id` int(11) unsigned NOT NULL,
`user_id` int(11) unsigned NOT NULL,
FOREIGN KEY (`event_id`) REFERENCES `events` (`id`),
FOREIGN KEY (`user_id`) REFERENCES `users` (`id`)
);
I want to retrieve all events, with skills requirments and users participation.
My first idea was to use LEFT JOIN :
SELECT e.id, e.name, es.skill_id, s.name skill_name, es.quantity,
eu.user_id, u.name user_name, us.skill_id user_skill_id,
uss.name user_skill_name
FROM events e
LEFT JOIN events_skills es ON es.event_id = e.id
LEFT JOIN skills s ON s.id = es.skill_id
LEFT JOIN events_users eu ON eu.event_id = e.id
LEFT JOIN users u ON u.id = eu.user_id
LEFT JOIN users_skills us ON us.user_id = u.id
LEFT JOIN skills uss ON uss.id = us.skill_id;
id name skill_id skill_name quantity user_id user_name user_skill_id user_skill_name
1 eve1 2 ski2 3 1 use1 2 ski2
1 eve1 3 ski2-2 1 1 use1 2 ski2
1 eve1 2 ski2 3 2 use2 NULL NULL
1 eve1 3 ski2-2 1 2 use2 NULL NULL
2 eve2 NULL NULL NULL NULL NULL NULL NULL
2 differents events, second without users and skills. First, with 2 users, first with one skills, seconds without skills, and finally, two skills requirements.
It work "fine", but it's verry ugly, and I think it can be very slow if an event or an user has too many skills.
Also, each event has a lot of lines, and it's difficult to sort them (if I want to display only 20 events...).
To improve, I think I can load all skills separately and remove the two LEFT JOIN for table skills.
But, can I merge all skills and quantity in one column, and users in others ? Did you have betters solutions/improvements ?
id name skills users
1 eve1 2,3;3,1 1,2;2,NULL
2 eve2 NULL NULL
Thanks for your help.
Another way to do what Thorsten suggested:
select e.*,
( SELECT group_concat(skill_id order by skill_id separator ',')
from events_skills
WHERE event_id = e.id
) AS skills,
( SELECT group_concat(user_id order by user_id separator ',' )
from events_users
WHERE event_id = e.id
) AS users
from events e
order by e.id;
Each subquery needs an extra join to get the "name" of the skill and user instead of the "id".
May I suggest you use the name of the skill as its id.
When you have a many-many mapping table, be sure to use the pair of ids as the PRIMARY KEY and have an INDEX in the opposite order. And don't have an AUTO_INCREMENT. Details: http://mysql.rjweb.org/doc.php/index_cookbook_mysql#many_to_many_mapping_table
A result with skills and users as comma-separated strings is exactly how I would do this, too. Use GROUP_CONCAT for this.
select e.*, s.skills, u.users
from events e
left join
(
select event_id, group_concat(skill_id order by skill_id separator ',') as skills
from events_skills
group by event_id
) s on s.event_id = e.id
left join
(
select event_id, group_concat(user_id order by user_id separator ',') as users
from events_users
group by event_id
) u on u.event_id = e.id
order by e.id;
Create SOF.SQL
CREATE TABLE "android_metadata" ("locale" TEXT DEFAULT 'en_US');
INSERT INTO "android_metadata" VALUES ('en_US');
CREATE TABLE main.t_def (
_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
word TEXT(20) not null,
word_def TEXT(20) not null
);
insert into t_def (word, word_def) values ('ball','spherical object');
insert into t_def (word, word_def) values ('cat','feline');
insert into t_def (word, word_def) values ('dog','common housekept');
CREATE TABLE main.t_a (
_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
corr_answer TEXT(20) not null,
user_answer TEXT(20) not null,
is_correct INTEGER not null
);
insert into t_a (user_answer, corr_answer, is_correct) values ('ball','cat',0);
insert into t_a (user_answer, corr_answer, is_correct) values ('dog','dog',1);
.exit
Then run:
sqlite3 foo.db < SOF.SQL
I want a result set that is:
ball|spherical object|cat|feline|0
This is the closest I have gotten:
select t_def.word, t_def.word_def from t_def, t_a where t_a.is_correct=0 and t_a.corr_answer=t_def.word;
To get values from two rows, you need two instances of the table:
SELECT t_a.user_answer,
user_def.word_def AS user_word_def,
t_a.corr_answer,
corr_def.word_def AS corr_word_def,
t_a.is_correct
FROM t_a
JOIN t_def AS user_def ON t_a.user_answer = user_def.word
JOIN t_def AS corr_def ON t_a.corr_answer = corr_def.word
WHERE NOT t_a.is_correct
I have following tables in my DB
CREATE TABLE [author_details] (
[_id] INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
[name] TEXT NOT NULL,
[surname] TEXT NOT NULL,
[middle_name] TEXT NULL
);
CREATE TABLE [authors] (
[_id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
[book_id] INTEGER NOT NULL,
[author_id] INTEGER NOT NULL
);
CREATE TABLE [books] (
[_id] INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
[title] TEXT NOT NULL,
[publisher_id] INTEGER NOT NULL,
[isbn] VARCHAR(10) UNIQUE NULL,
[ean] VARCHAR(13) UNIQUE NULL,
[pages] INTEGER DEFAULT '0' NULL,
[year] INTEGER NOT NULL,
[edition] TEXT NULL
);
CREATE TABLE [publishers] (
[_id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
[name] TEXT NOT NULL
);
I want a list of all books with details, I've used following query:
SELECT b.title,b.isbn,b.ean,b.year,b.pages,b.edition,
CASE
WHEN ad.middle_name IS NULL
THEN ad.name||" "||ad.surname
ELSE ad.name||" "||ad.middle_name||" "||ad.surname
END AS author, p.name
FROM books AS b, authors AS a, author_details AS ad, publishers AS p
INNER JOIN authors, author_details, publishers ON b._id=a.book_id AND ad._id=a.author_id AND b.publisher_id=p._id
GROUP BY b._id
It returns All books but only one author for books with multiple authors. How to write the query to get all authors per book?
To get the values from all records in a group, you have to use the group_concat function:
SELECT b.title,b.isbn,b.ean,b.year,b.pages,b.edition,
group_concat(CASE
...
END) AS author, p.name
FROM ...
Additionally, you need to use the correct join syntax.
In your query, you are joining every table twice, which results in lots up duplicate records.
There are two equivalent syntaxes for joins.
Either use a plain list of tables, and WHERE:
...
FROM books AS b,
authors AS a,
author_details AS ad,
publishers AS p
WHERE b._id = a.book_id
AND a.author_id = ad._id
AND b.publisher_id = p._id
...
or use the JOIN operator for each join, with a join condition for each join:
...
FROM books AS b
JOIN authors AS a ON b._id = a.book_id
JOIN author_details AS ad ON a.author_id = ad._id
JOIN publishers AS p ON b.publisher_id = p._id
...
Try to use group_concat():
SELECT b.title,b.isbn,b.ean,b.year,b.pages,b.edition,
GROUP_CONCAT(CASE
WHEN ad.middle_name IS NULL
THEN ad.name||" "||ad.surname
ELSE ad.name||" "||ad.middle_name||" "||ad.surname
END) AS author,
p.name
FROM
.........
I building a blog for school i would like to display the count of comments for each thread made. However I'm a bit lost to how to achieve this goal any help would be great thank you!
i have 2 tables
CREATE TABLE `blog_message` (
`MessageID` int(30) NOT NULL AUTO_INCREMENT,
`Username` varchar(45) NOT NULL,
`Message` text,
`AddedDate` datetime DEFAULT NULL,
`Title` varchar(45) DEFAULT NULL,
PRIMARY KEY (`MessageID`)
)
CREATE TABLE `blog_comments` (
`CommentID` int(30) NOT NULL AUTO_INCREMENT,
`MessageID` int(30) DEFAULT NULL,
`Author` varchar(45) DEFAULT NULL,
`CommentMessage` text,
`AddedDate` datetime DEFAULT NULL,
PRIMARY KEY (`CommentID`),
KEY `blog_comments_ibfk_1` (`MessageID`),
CONSTRAINT `blog_comments_ibfk_1` FOREIGN KEY (`MessageID`) REFERENCES `blog_message` (`MessageID`)
)
my goal is
to display in a gridview
int a table format
Comment count | title | Username | Date of creation
The following is MySQL syntax, not sure what you're using. But this will return a list MessageIds and the number of comments they have.
----------------------------
| MessageId | comment_count|
----------------------------
| 1234 | 34 |
----------------------------
SELECT bm.MessageId, count(bc.CommentId) as comment_count
FROM blog_comments bc, blog_message bm
WHERE bm.MessageId = bc.MessageId
GROUP BY bm.MessageId
If you want the Author and AddedDate just add it to the SELECT statement (i.e. SELECT bm.MessageId, count(bc.CommentId) as comment_count, bm.AddedDate, bm.Author).
Try the following
SELECT
bm.MessageID, Count(bc.CommentID) as Cnt, Title, Username, AddedDate
FROM blog_message bm
LEFT OUTER JOIN blog_comments bc
ON bm.MessageID = bc.MessageID
GROUP BY bm.MessageID,Title, Username, AddedDate