I'm trying to read a .sql file into SQLite, but I'm getting syntax errors because the file was dumped from MySQL, which can add multiple entries at once, but I'm using SQLite v3.7.7, which can't read more than one entry to a table at a time with the VALUES command.
My understanding is that I either need to upgrade SQLite, or somehow modify the file to read in one entry at a time into the tables. Please note I'm dealing with tens of thousands of entries, so inserting the UNION SELECT command probably won't be very easy.
You need at least SQLite 3.7.11 to use the VALUES syntax you're interested in. But mysqldump has about 100 command-line options. And one of them, --skip-extended-insert, can disable extended inserts. (So you get one INSERT statement per row.) Read the mysqldump documentation, and run the dump again with options that better fit your target.
Or better yet, look at the list of SQLite converter tools.
Related
I've a clustered partitioned table exported from GA 360. Attached is the image. I would like to create exact replica of the same. Using Web UI it's not possible. I created backup table using bq command line tool, still no luck.
Also, whenever we check preview it has a day filter. It looks like this:
Whenever data is appended to the backup table, I don't find this filter there even though this option is set to true while creating a table.
If you can give more context about handling this kind of table it would be beneficial.
Those are indeed sharded tables. As explained by #N. L they follow a time-based naming approach: [PREFIX]_YYYYMMDD. They then get grouped together. The explained procedure to backup them seems correct. Anyhow, I would recommend to use partitioned tables as it will be easier to backup them and they perform better in general.
This is not a cluster / partitioned table. This one is a sharded non-partitioned table having one common prefix. Once you start creating multiple tables with same prefix we can see them under the same prefix.
Ex:
ga_session_20190101
ga_session_20190102
both these tables will be grouped together.
To take backup of these tables you need to create a script to copy source to destination table with same name and execute that script using bq command line tool under the same project.
Is there any way in which select commands alter a sqlite database? I would assume not, but don't want to rely on that assumption. (the specific concern i had in mind was if e.g. querying the database for example creates indexes or similar for quicker retrieval in subsequent times, hence causing the sql files to change)
Asking, because i want to cache some values calculated from a sql file, and only update these values if there has been an edit to the sql file [specifically if the number of bytes file size has changed, which would indicate the sql database has changed. The specific calculations are quite computations intensive, so don't want to repeat unless neaded].
SELECT statements cannot modify the database.
SQLite sometimes needs to store temporary indexes or intermediate results, but such data goes into the temporary database, not into the actual database file.
Anyway, to find out whether a database file has changed, check the file change counter.
I am using sqlite3 (maybe sqlite4 in the future) and I need something like dynamic tables.
I have many tables with the same format: values_2012_12_27, values_2012_12_28, ... (number of tables is dynamic) and I want to select dynamically the table that receives some data.
I am using _sqlite3_prepare with INSERT INTO ? VALUES(?,?,?). Ofcourse this fails to compile (syntax error near ?). There is a nice and simple way to do this in sqlite ?
Thanks
Using SQL parameters is not possible for identifiers such as table or column names.
If you don't want to keep so many prepared statements around, just prepare them on the fly whenever you need one.
If your database were properly normalized, you would have a single big values table with an extra date column.
This organization is usually to be preferred, unless you have measured both and found that the better performance (if it actually exists) outweighs the overhead of managing multiple tables.
Afaik, SQLite stores a single database in a single file. Since this would decrease the performance when working with large databases, is it possible to explicitly tell SQLite not to store the whole DB in a single file and store different tables in different files instead?
I found out, that it is possible.
Use:
sqlite3.exe MainDB.db
ATTACH DATABASE 'SomeTableFile.db' AS stf;
Access the table from the other database file:
SELECT * FROM stf.SomeTable;
You can even join over several files:
SELECT *
FROM MainTable mt
JOIN stf.SomeTable st
ON (mt.id = st.mt_id);
https://www.sqlite.org/lang_attach.html
tameera said there is a limit of 62 attached databases but I never hit that limit so I can't confirm that.
The big advantage besides some special cases is that you limit the fragmentation in the database files and you can use the VACUUM command separately on each table!
If you don't need a join between these tables you can manually split the DB and say which tables are in which DB (=file).
I don't think that it's possible to let SQLite split your DB in multiple files, because you connect to a DB by telling the filename.
SQLite database files can grow quite large without any performance penalties.
The things that might degrade performance are:
file-locking contention
table size (if using indexes and issuing write queries)
Also, by default, SQLite limits the number of attached databases to 10.
Anyway, try partition your tables. You'll see that SQLite can grow enormously this way.
I am designing a standard ASP.Net site with a SQL database. I have a database schema and During the tests I am changing data types amongst other tasks and the data contained inside really is not that important.
I keep getting errors as the old data does not match the new rules. This is not important and I am happy to clear everything but currently, I have to export/publish the database to a .sql file then import it from scratch - which is time consuming.
Is there a quick button / feature that I have missed that allows you to reset autonumbers / IDs to 1 and delete all content, or just speed up what I currently do?
There are a few options you could take, the "fastest" really depends on your database.
To firstly answer your questions on seeding, etc - TRUNCATE TABLE will delete all information in a table (very fast, as it is not logged) and will reset your identity column.
eg:
TRUNCATE TABLE dbo.table
http://msdn.microsoft.com/en-us/library/aa260621(SQL.80).aspx
The significant restriction here is that you cannot use it on a table that is referenced by another table. In this case you can use a standard delete and then use DBCC CHECKIDENT
eg:
DELETE FROM dbo.table
GO
DBCC CHECKIDENT(dbo.table, reseed, 0)
http://msdn.microsoft.com/en-us/library/ms176057.aspx
Remember with delete to make sure you delete information in the correct order (i.e. taking into account foreign keys).
Another approach I often use is simply writing a complete tear-down / rebuild script when I want to reset the database. The basic premise is to tear down, or drop all database objects at the beginning of the script and then recreate them. This is not necessarily a solution for all scenarios, but for basic tasks works well for me. To avoid errors I would usually add my drop statements in IF statements, eg:
IF EXISTS
(
SELECT *
FROM information_schema.tables
WHERE table_name = 'table' AND table_schema = 'dbo'
)
BEGIN
DROP TABLE dbo.table
END
Why don't you write some T-SQL code to delete (or truncate, even quicker) all your tables? Be careful to take into consideration your integrity rules while clearing the tables: allways clean the tables containing the foreign key before cleaning the one containing the primary key.
If you just need to clear out data then just write a script to truncate all the data in each table. The truncate command also resets any IDENTITY fields as well.
TRUNCATE TABLE myTable
For each table you have. Then just run that script each time.
Here'a a quick way to delete all of the data in a table:
TRUNCATE TABLE YourTableName
You could write a script that would truncate all of your tables.
The alternative is to just DROP the table and re-create it.
If you really want to drop all data, then you could detach the database and create a brand new one; it's a bit extreme, but possibly faster than dropping everything first.
As others have suggested I find it preferable to maintain a script that builds the database from scratch and can tear down the database prior to rebuilding it. Develop this script just as you'd develop the rest of the application. I find it easier to understand the database through a script than by building it through a GUI, especially where there are complex relationships, triggers and so on.
It's also useful if you have other developers, and perhaps quicker and less prone to errors than copying your working database and handing it to another developer.
On release you can freeze that script and then create delta scripts for the next release which has just the changes from the initial schema to the new. This could also tear down the new objects created in the delta before recreating them so it can be easily re-run without having to wipe the entire database.
if you use Visual Studio 2010 then
open the App_Data folder of the solution and double click on the MDF File.
right click on your table , in the menu select "Show Table Data".
select all rows and delete all them.