I'm not a database expert and I'm simply building a prototype app, so nothing really important.
Anyway, the app is about a subway: this subway has many lines and sometimes some stops are shared between lines (so, for example, stops 3 and 4 are stops of lines 2, 7 and 9).
So, I made up a SQLite stops table:
+---------+-------------+------+
| Field | Type | Auto |
+---------+-------------+------+
| id | integer | YES |
| name | varchar(20) | NO |
| lines | ? | NO |
+---------+-------------+------+
What's the best way to deal with shared stops? My idea was to create a lines table and then in the lines field of the stops table put a comma separated list of lines.id. I don't know why, but I feel there could be a better way.
Any suggestion is appreciated, and sorry for the really noob question.
I would keep it simple and use a table lines which has an ID (primary key) along with other metadata for a line (such as name):
lines
(id, name)
Then, create a table for the stops:
stops
(id, name)
Finally, you can create a bridge table which will connect lines with stops:
bridge
(lineId, stopId)
Each record in the bridge table represents one line having a given stop.
Note that using CSV to represent a line having multiple stops is totally not the way to go here, as it renders the powers of your relational database useless.
Update:
If you want to record the position of a stop in a given line (and assuming that positions would differ across lines), you could use the following table:
stopNumbers
(lineId, stopId, stopPosition)
The stop position can be obtained knowing the line's ID and the stop's ID.
You need a many-to-many relation, which is stored in a separate table like this:
table lines_to_stops
line_fk
stop_fk
That's the relational world ...
Note that records in the database are not in any specific order. If you need to put the stops into any specific order (which you most probably do), you have to store this order to the database as well:
table lines_to_stops
line_fk
stop_fk
order_in_line
Related
I am a software engineer, but I am very new to databases and I am trying to hack up a tool to show some demo.
I have an Apache server which serves a simple web page full of tables. Each row in the table has a proposal id and a link to a web page where the proposal is explained. So just two columns.
----------------------
| id | proposal |
|--------------------
| 1 | foo.html |
| 2 | bar.html |
----------------------
Now, I want to add a third column titled Comments where a user can leave comments.
------------------------------------------------
| id | proposal | Comments |
|-----------------------------------------------
| 1 | foo.html | x: great idea ! |
| | | y: +1 |
| 2 | bar.html | z: not for this release |
------------------------------------------------
I just want to quickly hack up something to show this as a demo and get feedback. I am planning to use SQLite to create a table per id and store the userid, comments in the table. People can add comment at the same time. I am planning to use lock to perform operations on the SQLite database. I am not worried about scaling just want to show and get feedback. Are there any major flaw in this implementation?
There are similar questions. But I am looking for a simplest possible implementation.
Table per ID; why would you want to do that? If you get a large number of proposals, the number of tables can get out of hand very quickly. You just need to keep an id column in the table to keep track of things and keep the number of tables in a sane figure.
The other drawback of using a table for each proposal is that you will not be able to use prepared statements for those, because table names cannot be bound as a parameter.
SQLite assumes the table name is 'a'
Add column
alter table a add column Comments text;
Insert comment
insert into a values (4,"hello.html","New Comment");
You need to provide values for the other two columns along with the new comment.
My application uses a database to save its data. I have table Objects that looks like
localID | title | content
1 Test "1,embed","3,embed","5,append"
and another table Contents that looks like
localID | content
1 Alpha
2 Beta
3 Gamma
4 Delta
5 Epsilon
The main applications runs in the main thread, the whole database stuff in a second thread. So if my application loads, I want to pass each record (QSqlRecord) to the main thread where it gets further processed (loaded into real objects). I pass that record via signals. But my data is split up into 2 tables. I want to return a record containing both, perhaps similar to a join:
localID | title | content
1 Test "Alpha,embed","Gamma,embed","Epsilon,append"
So this way, I would have all the needed information at once after only one thread return value. Without combining, I would have to call the database for each single referenced content.
I expect the database to contain less than 100.000 records, yet some content may be big (files saved as blob, e.g. a book of size of 300 mb or so).
I have two questions:
(How) Can I join the tables this way inside a query (efficiently)?
Am I too concerned about threading and should make it single threaded?
That way I would not need to bother with multiple read requests.
As a sidenode, this is my first post on Database Admins, I was not too sure about this site or Stackoverflow being the right place to ask this.
For any actual problem, use the way recommended by #Vérace in the comments,
i.e. a "linking" table. That is the way.
However, if you are either forced to keep the database structure
or for fun
or for learning (which is indicated by the migration header),
learning dirty tricks however, instead of good design...
have a look at this:
select
localID, title,
(
with recursive cnt(x) as
( select ','||a.content
union all
select replace(x, '"'||b.localID||',', '_"'||b.content||',')
from cnt, toy2 as b
)
select replace('_"'||replace(x, ',_"', ',"'), '_","', '"') from cnt
where not x like '%,"%' LIMIT 1
) as 'content' from toy as a;
using a recursive method to flexibly
(no assumptions on number of entries in AlphaBeta table, or number of their uses)
replace the numbers by greek
applying a naming scheme with "_" to create an end condition
prepend a "_" to content, to make it be processed
and cooperate with end condition
cleanup the end-condition "_"s for desired output
cleanup special case at start of output line
select the result of recursive together with other desired outputs
Note the assumption that your table does not naturally contain '__"' or '_"'. If that happens choose more "weird" strings there. If you have all kinds of strings in your table, then you look at a very meek example of what Verace describes as "a desaster to happen". Actually this non-trivial solution is in itself probably a desaster which happened.
Output (.headers on and .mode column):
localid title content
---------- ---------- --------------------------------------------
1 Test "Alpha,embed","Gamma,embed","Epsilon,append"
2 mal "Beta,append","Delta,embed"
Here is my mcve (.dump), with an additional line "mal" for testing purposes:
BEGIN TRANSACTION;
CREATE TABLE toy (localid int, title varchar(20), content varchar(100));
INSERT INTO toy VALUES(1,'Test','"1,embed","3,embed","5,append"');
INSERT INTO toy VALUES(2,'mal','"2,append","4,embed"');
CREATE TABLE toy2 (localID int, content varchar(10));
INSERT INTO toy2 VALUES(1,'Alpha');
INSERT INTO toy2 VALUES(2,'Beta');
INSERT INTO toy2 VALUES(3,'Gamma');
INSERT INTO toy2 VALUES(4,'Delta');
INSERT INTO toy2 VALUES(5,'Epsilon');
COMMIT;
SQLite 3.18.0 2017-03-28 18:48:43
i want to design something like a dynamic form in which admin define each form fields.
i design 3 table: mainform table for shared properties, then formfield tables which have mainformID as a foreign key and define each form fields
e.g:
AutoID | FormID | FieldName
_____________________________
100 | Form1 | weight
101 | Form1 | height
102 | Form1 | color
103 | Form2 | Size
104 | Form2 | Type
....
at leas a formvalues table like bellow:
FormFieldID | Value | UniqueResponseID
___________________________________________
100 | 50px | 200
101 | 60px | 200
102 | Red | 200
100 | 30px | 201
101 | 20px | 201
102 | Black | 201
103 | 20x10 | 201
104 | Y | 201
....
for each form i have to join these 3 tables to catch all fields and values. i wonder if its the only way to design such a scenario? does it decrease sql performance? or is there any fast and better way?
This is a form of EAV, and I'm gonna assume you absolutely have to do it instead of the "static" design.
does it decrease sql performance?
Yes, getting a bunch of rows (under EAV) is always going to be slower than getting just one (under the static design).
or is there any fast and better way?
Not from the logical standpoint, but there are significant optimizations (for query performance at least) that can be done at the physical level. Specifically, you can carefully design your keys to minimize the I/O (by putting related data close together) and even eliminate the JOIN itself.
For example:
This model migrates keys through FOREIGN KEY hierarchy all the way down to the ATTRIBUTE_VALUE table. The resulting natural composite key in ATTRIBUTE_VALUE table enables us to:
Get all attributes1 of a given form by a single index range scan + table heap access on ATTRIBUTE_VALUE table, and without doing any JOINs at all. In addition to that, you can cluster2 it, eliminating the table heap access and leaving you with only the index range scan3.
If you need to only get the data for a specific response, change the order of the fields in the composite key, so the RESPONSE_ID is at the leading edge.
If you need both "by form" and "by response" queries, you'll need both indexes, at which point, I'd recommend secondary index to also cover4 the VALUE field.
For example:
-- Since we haven't used NONCLUSTERED clause, this is a B-tree
-- that covers all fields. Table heap doesn't exist.
CREATE TABLE ATTRIBUTE_VALUE (
FORM_ID INT,
ATTRIBUTE_NAME VARCHAR(50),
RESPONSE_ID INT,
VALUE VARCHAR(50),
PRIMARY KEY (FORM_ID, ATTRIBUTE_NAME, RESPONSE_ID)
-- FOREIGN KEYs omitted for brevity.
);
-- We have included VALUE, so this B-tree covers all fields as well.
CREATE UNIQUE INDEX ATTRIBUTE_VALUE_IE1 ON
ATTRIBUTE_VALUE (RESPONSE_ID, FORM_ID, ATTRIBUTE_NAME)
INCLUDE (VALUE);
1 Or a specific attribute, or a specific response for a specific attribute.
2 MS SQL Server clusters all tables by default, unless you specify NONCLUSTERED clause.
3 Friendliness to clustering and elimination of JOINs are some of the main strengths of natural keys (as opposed to surrogate keys). But they also make tables "fatter" and don't isolate from ON UPDATE CASCADE. I believe pros outweigh cons in this particular case. For more info on natural vs. surrogate keys, look here.
4 Fortunately, MS SQL Server supports including fields in index solely for covering purposes (as opposed to actually searching through the index). This makes the index leaner than a "normal" index on the same fields.
I like Branko's approach, and it is quite similar to metadata models i have created in the past, so this post is by way of extension to his. you may want to add a datatype table, which can work both for native types (int,varchar,bit,datetime etc.) and your own definitions (although i don't see the necessity off the cuff).
thence, Branko's "value" column becomes:
value_tinyint tinyint
value_int int
value_varchar varchar(xx)
etc.
with a datatype_id (probably tinyint) as a foreign key into the "mydatatype" table.
[excuse the lack of pretty ER diagrams like BD's]
mydatatype
datatype_id tinyint
code varchar(16)
description varchar(64) -- for reference purposes
This extension should:
a. save you a good deal of casting when reading or writing your data
b. allow both reads and writes with some easily constructed dynamic SQL
Furthermore (and maybe this is out of scope), you may want to store the order in which these objects are created/saved, as well as conditional display based on button push/checkbox/radio button selection etc.
I won't go into detail here, since i'm not sure you need these things, but if you do i'll check this every so often and respond with stuff.
The architecture for this scenario is as follows:
I have a table of items and several tables of forms. Rather than having the forms own the items, the items own the forms. This is because one item can be on several forms (although only one of each type, but not necessarily on any). The forms and items are all tied together by a common OrderId. This can be represented like so:
OrderItems | Form A | Form B etc....
---------- |--------- |
ItemId |FormAId |
OrderId |OrderId |
FormAId |SomeField |
FormBId |OtherVar |
FormCId |etc...
This works just fine for these forms. However, there is another form, (say, FormX) which cannot have an OrderId because it consists of items from multiple orders. OrderItems does contain a column for FormXId as well, but I'm confused about the best way to get a list of the "FormX"s related to a single OrderId. I'm using MySQL and was thinking maybe a stored proc was the best way to go on this, but I've never used a stored proc on MySQL and don't really know the best way to go about it. My other (kludgy) option was to hit the DB twice, first to get all the items that are for the given OrderId that also have a FormXId, and then get all their FormXIds and do a dynamic SELECT statement where I do something like (pseudocode)
SELECT whatever FROM sometable WHERE FormXId=x OR FormXId=y....
Obviously this is less than ideal, but I can't really think of any other way... anything better I could do either programmatically or architecturally? My back-end code is ASP.NET.
Thanks so much!
UPDATE
In response to the request for more info:
Sample input:
OrderId = 1000
Sample output
FormXs:
-----------------
FormXId | FieldA | FieldB | etc
-------------------------------
1003 | value | value | ...
1020 | ... .. ..
1234 | .. . .. . . ...
You see the problem is that FormX doesn't have one single OrderId but is rather a collection of OrderIds. Sometimes multiple items from the same order are on FormX, sometimes it's just one, most orders don't have any items on FormX. But when someone pulls up their order, I need for all the FormXs their items belong on to show up so they can be modified/viewed.
I was thinking of maybe creating a stored proc that does what I said above, run one query to pull down all the related OrderIds and then another to return the appropriate FormXs. But there has to be a better way...
I understand you need to get a list of the "FormX"s related to a single OrderId. You say, that OrderItems does contain a column for FormXId.
You can issue the following query:
select
FormX.*
From
OrderItems
join
Formx
on
OrderItems.FormXId = FormX.FormXId
where
OrderItems.OrderId = #orderId
You need to pass #orderId to your query and you will get a record set with FormX records related to this order.
You can either package this query up as a stored procedure using #orderId paramter, or you can use dynamic sql and substitute #orderId with real order number you executing your query for.
I'm new to ASP.NET but not to programming. I am migrating our current site from PHP/MySQL to ASP.NET(3.5)/SqlServer. I've been lurking here since the site's launch, so I'm confident that one (or more) of you can help me out. Here's the scenario:
This is a training department site and the dept. has a course catalog stored in the table course. Each course may have many prerequisite courses, For example, A and B are prerequisites for C. I would normally store this either as a comma-delimited column in course or in a separate table course_prereq or course_course as a recursive relationship. This part I can do.
However, the business rules require that each course can have multiple sets of prerequisites. Fore example, N requires A, B and C, or N requires X and Y. This is where I'm stuck.
Previously, I stored this information in a column for row N as A,B,C|X,Y, parsed the ids into a PHP 2D-array, submitted a second query for all the rows whose id was in that array, then used PHP to separate those rows into their respective groups. Once all this processing is done, the groups of prerequisites are displayed as separate tables on the web page, like so:
| A | A's information goes here |
| B | B's information goes here |
| C | C's information goes here |
- - - - - - - OR - - - - - - - -
| X | X's information goes here |
| Y | Y's information goes here |
How would I accomplish this using ASP.NET?
Add a table to hold Prerequisite Sets. This table holds a set ID and key back to the courses table for each course in the set. The table may have several rows for a given set ID, so your primary key will be the set ID plus the course ID. Then in your course_prereq table you relate courses to the different prerequisite sets. An OR relationship can be assumed there because any ANDs are enforced in the sets themsevles.
Have a table called PrerequisiteSet that FKs to each prereq. Then have a Course_PrerequisiteSet many to many table that FKs to Course and PrerequisiteSet. Most of the time there will only be one entry in Course_PrerequistieSet, but if there are more than one, then it will be an OR relationship.
Both the answers above were very helpful. I ended up using just one database table instead of the suggested two. The table contains a course_id, prereq_id, and set_id, which all together form the primary key.
In the ASP.NET page, I use a repeater to loop over the sqldatasource stored procedure that returns a course's prerequisite sets, and a gridview inside that repeater that reads the individual prerequisite information from a second sqldatasource stored procedure. Like this:
RepeaterSqlDataSource (returns set ids)
Repeater
. . . GridViewSqlDataSource (returns course info for each prereq_id in set
. . . GridView
Hope this is helpful to anyone else looking at a similar scenario.