This is continuation of question SQLIte and recursive data .
Say I want to save trees in SQLite. Each node/leave has exactly the same data structure, but any 2 trees can share 0 or more nodes or leaves. Number of trees is undefined.
So what can be proper schema for that?
The best solution I can propose so far is to create table with data:
create table main (dataid int, data text);
and then for each tree dynamically create new table:
create table tree (dataid int, parent references (tree), foreign key dataid references main(dataid) on delete cascade );
If anybody can do better - welcome.
Related
I am new to dynamodb & was having some trouble in finding a way to randomly getting items without a full table scan ,most of the algorithms that i found consist of full table scans
I am also taking the case where we don’t have additional information of the table(Like columns and column Type such info is unknown)
Is there a way exist to do so
You can randomly sample by using a randomly generated exclusive start key for the scan or query operation. The exclusive start key does not have to match a record in the table. It just needs to follow the key structure of the table/index.
As with most questions about queries in DynamoDB, how you structure your data depends on how you want to query it.
For something like a random sampling, you have to make it confirm to the following core constraint of DynamoDB:
You have to provide a partition key
You can provide a sort key
So with a "single table" type design, you could structure your data something like this:
PK
SK
myVal
my_dict
6caaf1e3-eb8d-404a-a2ae-97d6682b0224
foo
my_dict
1c5496e8-c660-4b4e-980f-4abfb1942863
bar
my_dict
56551340-fff8-4824-a5be-70fcaece2e1a
baz
my_other_dict
520a7b37-233c-49dd-87da-77d871d98c92
test1
my_other_dict
65ccd54e-72c3-499d-a3a7-0cd989252607
test2
The PK is the identifier for your collection of random things to look up. The SK is a random UUID. And myVal contains the value you want to be returned.
You can query this db the following way:
SELECT * FROM "my-table" WHERE PK = 'my_dict' AND SK < '06a04e20-b239-48f2-a205-552eb61fef35'
By querying with an UUID as the SK, you'll get the first item in the table with an UUID close to the one you query for. By using a random uuid each time you query, you'll get a random result back.
The particular query above actually returns nothing, so you need to retry until you get a result.
Also, I haven't done the math (who has?), but I'd imagine that periodic queries like this won't generate perfectly random distributions, especially for small data sets.
Is it possible to manage a cascading relationship in icCube? For example, if I have the following 3 tables:
(1) NameID, FuelID
(2) FuelID, ScheduleID
(3) ScheduleID, DateTime, Value
Can I tie the NameID from table (1) to Value in table (3)
You can do this using a Javascript View. You can check the documentation here how to do this, the idea is that the 'main' table is the big one and the small ones, that should behave as lookup tables, are the 'cached' tables.
hope it helps
I have created a table with sqlite for my corona/lua app. It's a hashtable with ~=700 000 values.The table has two columns, which are the hashcode (a string), and the value (another string). During the program I need to get data several times by providing the hashcode.
I'm using something like this code to get the data:
for p in db:nrows([[SELECT * FROM test WHERE id=']].."hashcode"..[[';]]) do
print(p)
-- p = returned value --
end
This statement is though taking insanely too much time to perform
thanks,
Edit:
Success!
the mistake was with the primare key thing.I set the hashcode as the primary key like below and the retrieve time whent to normal:
CREATE TABLE IF NOT EXISTS test (id STRING PRIMARY KEY , array);
I also prepared the statements in advance as you said:
stmt = db:prepare("SELECT * FROM test WHERE id = ?;")
[...]
stmt:bind(1,s)
for p in stmt:nrows() do
The only problem was that the db file size,that was around 18 MB, went to 29,5 MB
You should create the table with id as a unique primary key; this will automatically make an index.
create table if not exists test
(
id text primary key,
val text
);
You should not construct statements using string concatenation; this is a security issue so avoid getting in this habit. Also, you should prepare statements in advance, at program initialization, and run the prepared statements.
Something like this... initially:
hashcode_query_stmt = db:prepare("SELECT * FROM test WHERE id = ?;")
then for each use:
hashcode_query_stmt:bind_values(hashcode)
for p in hashcode_query_stmt:urows() do ... end
Ensure that there is an index on the id/hashcode column? Without one such queries will be slow, slow, slow. This index should probably be unique.
If only selecting the value/hashcode (SELECT value FROM ..), it may be beneficial to have a covering index over (id, value) as that can avoid additional seeking to the row data (see SQLite Query Planning). Try it with and without such a covering index.
Also, it may be worthwhile to employ caching if the same hashcodes are queried multiple times.
As already stated, get sure you have an index on ID.
If you can't change table schema now, you can add a index ad hoc:
CREATE INDEX test_id ON test (id);
About hashes: if you are computing hashes in your software to speed up searches, don't!
SQLite will use your supplied hashes as any regular string/blob. Also, RDBMS are optimized for efficient searching, which may be greatly improved with indexes.
Unless your hashing to save space, you are wasting processor time computing hashes in your application.
I have a table in a MS Access 2010 Database and it can easily be split up into multiple tables. However I don't know how to do that and still keep all the data linked together. Does anyone know an easy way to do this?
I ended up just writing a bunch of Update and Append queries to create smaller tables and keep all the data synced.
You must migrate to other database system, like MSSQL, mySQL. You can't do in MsAccess replication...
Not sure what do you mean by split up into multiple tables.
Are the two tables have same structure? you want to divide the table into two pats ... means if original table has fields A,B,C,D ... then you want to split it to Table1: A,B and
Table2: C,D.
Anyways, I googled it a bit and the below links might of what you are looking for. Check them.
Split a table into related tables (MDB)
How hard is it to split a table in Access into two smaller tables?
Where do you run into trouble with the table analyzer wizard? Maybe you can work around the issue you are running into.
However, if the table analyzer wizard isn't working out, you might also consider the tactics described in http://office.microsoft.com/en-us/access-help/resolve-and-help-prevent-duplicate-data-HA010341696.aspx.
Under Microsoft Access 2012, Database Tools, Analyze table.. I use the wizard to split a large table into multiple normalized tables. Hope that helps.
Hmmm, can't you just make a copy of the table, then delete opposite items in each table leaving the data the way you want except, make sure that both tables have the same exact auto number field, and use that field to reference the other.
It may not be the most proficient way to do it, but I solved a similar issue the following way:
a) Procedure that creates a new table via SQL:
CREATE TABLE t002 (ID002 INTEGER PRIMARY KEY, CONSTRAINT SomeName FOREIGN KEY (ID002) REFERENCES t001(ID001));
The two tables are related to each other through the foreign key.
b) Procedure that adds the neccessary fields to the new table (t002). In the following sample code let's use just one field, and let's call it [MyFieldName].
c) Procedure to append all values of field ID001 from Table t001 to field ID002 in Table t002, via SQL:
INSERT INTO ID002 (t002) SELECT t001.ID001 FROM t001;
d) Procedure to transfer values from fields in t001 to fields in t001, via SQL:
UPDATE t001 INNER JOIN t002 ON t001.ID001 = t002.ID002 SET t002.MyFieldName = t001.MyFieldName;
e) Procedure to remove (drop) the fields in question in Table t001, via SQL:
ALTER TABLE t001 DROP COLUMN MyFieldName;
f) Procedure that calls them all one after the other. Fieldnames are fed into the process as parameters in the call to Procedure f.
It is quite a bunch of coding, but it did the job for me.
I have recently stumbled upon a problem with selecting relationship details from a 1 table and inserting into another table, i hope someone can help.
I have a table structure as follows:
ID (PK) Name ParentID<br>
1 Myname 0<br>
2 nametwo 1<br>
3 namethree 2
e.g
This is the table i need to select from and get all the relationship data. As there could be unlimited number of sub links (is there a function i can create for this to create the loop ?)
Then once i have all the data i need to insert into another table and the ID's will now have to change as the id's must go in order (e.g. i cannot have id "2" be a sub of 3 for example), i am hoping i can use the same function for selecting to do the inserting.
If you are using SQL Server 2005 or above, you may use recursive queries to get your information. Here is an example:
With tree (id, Name, ParentID, [level])
As (
Select id, Name, ParentID, 1
From [myTable]
Where ParentID = 0
Union All
Select child.id
,child.Name
,child.ParentID
,parent.[level] + 1 As [level]
From [myTable] As [child]
Inner Join [tree] As [parent]
On [child].ParentID = [parent].id)
Select * From [tree];
This query will return the row requested by the first portion (Where ParentID = 0) and all sub-rows recursively. Does this help you?
I'm not sure I understand what you want to have happen with your insert. Can you provide more information in terms of the expected result when you are done?
Good luck!
For the retrieval part, you can take a look at Common Table Expression. This feature can provide recursive operation using SQL.
For the insertion part, you can use the CTE above to regenerate the ID, and insert accordingly.
I hope this URL helps Self-Joins in SQL
This is the problem of finding the transitive closure of a graph in sql. SQL does not support this directly, which leaves you with three common strategies:
use a vendor specific SQL extension
store the Materialized Path from the root to the given node in each row
store the Nested Sets, that is the interval covered by the subtree rooted at a given node when nodes are labeled depth first
The first option is straightforward, and if you don't need database portability is probably the best. The second and third options have the advantage of being plain SQL, but require maintaining some de-normalized state. Updating a table that uses materialized paths is simple, but for fast queries your database must support indexes for prefix queries on string values. Nested sets avoid needing any string indexing features, but can require updating a lot of rows as you insert or remove nodes.
If you're fine with always using MSSQL, I'd use the vendor specific option Adrian mentioned.