I need to manipulate some data in SQLite, it should be simple but trying to figure it out how to do exactly this has frustrated me.
It's just a join, one table called "routes" has a column "stop_id". I need to take another table called "stops" which also has a "stop_id" column and everywhere that they match, add all the additional columns from "stops" to the "routes" table (added columns are "stop_name" "stop_lat" "stop_lon" and "master_station"). "stop_id" is the primary key in the stops table. I need to join the tables and not keep them relational because after I do that I will be changing the rows by hand with new information. I am using Firefox SQLite Manager if that matters.
A join can be done with JOIN:
SELECT * FROM routes JOIN stops USING (stop_id)
However, the result of a join cannot be changed directly; the UPDATE statement works only on actual tables.
To change values that come from the routes or stops tables, you have to update those tables by using their respective primary keys to look up the records.
Related
I have two databases with the same structure. The first is the main one, while the second get updated periodically (in reality I have multiple "secondary" databases that I want to merge one by one into the main one).
The structure of the main and the secondary databases is identical.
I want to periodically dump all new values from the secondary database in the main one. However, the second time I do it, I want to exclude rows that were already copied the first time (and so on).
The tables in all these database have:
an ID column set as PRIMARY KEY going from 1 to N for each database (I suspect this was a mistake, but at the moment I can't change this)
a DATE column, representing a posix timestamp (float)
some other columns
My code looks like this:
ATTACH DATABASE secondary.db AS temp_db
DROP TABLE IF EXISTS my_table_temp
CREATE TABLE my_table_temp AS SELECT * FROM my_table
INSERT INTO main.my_table_temp SELECT * FROM temp_db.my_table
DELETE FROM my_table
INSERT INTO main.my_table SELECT DISTINCT * FROM main.my_table_temp ORDER BY date
DROP TABLE my_table_temp
the problem is that - I suspect due to the repeated ID column - the DISTINCT clause returns me:
UNIQUE constraint failed: my_table.id
However I don't care at all of the ID field that could also be dropped or reset.
NOTES:
the secondary databases are constantly updated by a code that - at the moment - I can't change
I initialize the "main" database copy-pasting one of the secondary to avoid regenerating the whole structure from scratch. Maybe there is a better way of doing this
Apologies if this is a naive question, but I'm very new with SQLite.
Thanks
Following the advice from #forpas, I solved this with the following code:
Assuming the columns to be id,date,col1 and col2
ATTACH DATABASE secondary.db AS temp_db
DROP TABLE IF EXISTS my_table_temp
CREATE TABLE my_table_temp AS SELECT date,col1,col2 FROM my_table
INSERT INTO main.my_table_temp SELECT date,col1,col2 FROM temp_db.my_table
DROP TABLE my_table /* I need to recreate my_table as I've removed a column*/
CREATE TABLE main.my_table AS SELECT DISTINCT date,col1,col2 FROM main.my_table_temp ORDER BY date
DROP TABLE my_table_temp
also, I automatized the extraction of the column names doing
SELECT name FROM PRAGMA_TABLE_INFO('my_table');
This is then passed to the python code running the script and the column id is removed from the list. Note that the second (and following) time I run this code, the column id won't be present in my_table to start with. However this approach allows the code to be the same in the two cases: either if the column id is there or not.
This procedure is then iterated over each table name to fully merge the two databases.
I have a Customers table which contains the salesRepEmployeeNumber which is in the Employees table.
How do I do something like
SELECT *
FROM Customers
JOIN Employees
ON Customers.salesRepEmployeeNumber = Employees.employeeNumber
with icCube ETL ?
As pointed in another answer, you can add a table based in an SQL statement that would do the job. In case your original datasource is not able to do a join :
We've not yet an join transformation, added this in our todo list. On the meantime, what you can do is.
Create an Union Table with your two tables. This will create a new table with the columns of both tables. Put the small one, first as we're going to cache it later on.
Create a Javascript view, you might need to activate Javascript in your icCube.xml configuration. In this one you can cache the first table and use a bit of js to do the join. You can trigger the table change on a field being empty. Don't forget to put 'Table Row Ordering' to Keep Table Order.
hope it helps
No need to use the ETL.
With the designer, add a table with the + sign in the menu above DataSource. The next panel gives you the choice between reading data from an existing table or an sql query.
I'm working on a sqlite database and try to make a special request between two tables.
In the first table (table1 for example), i have two columns named "reference" and "ID". I want to search an ID in it, get it value in "reference" and display all informations from the table which have this value as name.
I try to find something on the internet but I didn't find an answer.
This is the request I made:
select * from (select Reference from table1 where Name='Value1')
It only give me the result of
select Reference from table1 where Name='Value1'
EDIT:
I want
select Reference from table1 where Name='Value1' => name of table
select * from name of table => show all elements
I'm new in sqlite but I hope you can help me.
Thank you by advance
Matt
If I understand your question correctly, I don't think there's a way to do it in sql completely (or at least not in a portable way). I'd recommend one of 3 solutions:
Do exactly what you want, but do some processing in Python. That means query your master table, then construct new query based on each of the rows returned.
If you have many tables, possibly changing dynamically - it may be a good idea to rethink your database design. Maybe you can move some of the changing table names into a new column and put your data in one table?
If you have only a few tables available as the Reference and they never change, you could join all the possible tables, like:
SELECT ... FROM table1
LEFT JOIN table2
ON table1.id = table2.id AND table1.Reference = "table2"
LEFT JOIN table3 ...
But you may need to explain it all a bit better...
In drupal website we are getting sql query data exporting very slow (taking longtime) issue how to solve the issue
The query as follows
SELECT DISTINCT(a.*), c.nid, b.uac_inst_campus_cricos
FROM uac_export_coursetable_latest AS a
LEFT JOIN uac_institutiondata AS c
ON c.uac_institutiondata_institution = a.uac_course_institution
LEFT JOIN uac_inst_campus_latest AS b
ON b.nid = c.nid AND b.uac_inst_furtherinfobox_heading = a.campusname
WHERE a.uac_course_institution = '6628'
AND intyear12 = 'Yes'
ORDER BY uaccoursecode
Because we don't know the exact schema of your custom tables, we can't give you an exact solution but in general when query execution is slow, you need to verify the columns you are using for the JOINS and within the WHERE clausule.
Be sure that you are joining on foreign key columns
Be sure that indexes are set on the columns used within conditions
In your case, I would add index on following columns: uac_institutiondata_institution (uac_institutiondata table), intyear12 (uac_export_coursetable_latest), nid (uac_inst_campus_latest table)
If the uac_course_institution column in uac_export_coursetable_latest table is not a primary key, also on a index on this column.
More info about indexes on a MySql database: http://dev.mysql.com/doc/refman/5.0/en/create-index.html
Can you reorder SQLite table columns via a query?
I would prefer a query method, but if that is not possible is there any other way?
Yes, you control the order of columns by the order you name them in the query. Both these queries return the same rows, but the columns are in a different order.
select first_column_name, second_column_name
from mytable;
select second_column_name, first_column_name
from mytable;
If you want it to appear that the columns in the base table have been permanently changed, you can use a view.
create view mytable_reordered
select second_column_name, first_column_name
from mytable;
If you wanted to make that change transparent to application programs, you'd first rename the table, then create the view, giving it the old name of the table. Finally, you'd jump through whatever hoops your dbms requires in order to make that view updatable. In SQLite, I think that means writing code to implement some INSTEAD OF triggers.