My application logs user access to sqlite database, the my.db file grows about 5Gb every month. There're tables like "access" which logs every user access, this table should be cleared at every beginning of the month, and it holds almost all the 5gb data. Another table "user" which holds all my users, this table should always keep its data.
So every month I need to:
ctrl+c -> ctrl+v my.db to clone a file copy(for future statistics), this is very slow for 5gb file.
clear the "access" table via "delete from access" and "vacuum", it also takes some time.
It's too slow. I wonder if there is a sql command like "export/import database structure and indices" to clone only the structure to another new.db, and then "copy * from mydb.user to newdb"
Or maybe some existing tool can do this?
Here's the sample database I'll use, representing your old database. I made a table to keep, a table to lose, and an index.
$ sqlite3 old.db
sqlite> create table KeepMe (a TEXT);
sqlite> create table DeleteMe (b TEXT);
sqlite> create index DeleteMe_b on DeleteMe(b);
sqlite> insert into KeepMe values("Hello");
sqlite> insert into DeleteMe values("World");
sqlite> .quit
First copy the schema to a new database:
$ sqlite3 old.db .schema | sqlite3 new.db
Then open the new database, ATTACH to the old database, and copy over the table(s) you want:
$ sqlite3 new.db
sqlite> attach "old.db" as old;
sqlite> insert into KeepMe select * from old.KeepMe;
sqlite> .dump
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE KeepMe (a TEXT);
INSERT INTO "KeepMe" VALUES('Hello');
CREATE TABLE DeleteMe (b TEXT);
CREATE INDEX DeleteMe_b on DeleteMe(b);
COMMIT;
sqlite> .quit
Related
So I made a table using this command
.open C:/Users/Fries/Desktop/Test.db
I closed SQL lite and populated the db with the data below using notepad
CREATE TABLE Persons (
PersonID int,
LastName varchar(255),
FirstName varchar(255),
Address varchar(255),
City varchar(255)
);
When I open the db and type .tables there is an error for some reason
The commands do work when used properly.
In short what you have done, is
Create and save a database that has no tables.
Open that file with Notepad (despite warnings that you may have received)
Entered some data and saved the file.
Thus deleting eveything in the file that SQLite knows about so when you open it SQLite tells you that it's not a database e.g. the first 16 bytes must be SQLite format 3\000 (see below when the database file is opened in notepad).
Open the file from within the SQLITE3 program which has then told you it's not a database.
i.e. The file that you open is the actual database not something that you have typed into notepad.
The data, which will be in columns of a table or tables. Has to be entered via SQLITE3 commands (including SQL). The CREATE TABLE... is one such SQL command (yours is valid and has been used below as it is).
What you should be doing is starting SQLIT3 (typing sqlite3 if it's path has been added to the PATH environment variable), and then entering commands such as CREATE TABLE ......
E.G.
When you first start sqlite3 from a command prompt you will get :-
C:\Users\Mike>sqlite3
SQLite version 3.22.0 2018-01-22 18:45:57
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
A database is opened as such but it is an in memory database and would be lost if you quit.
You can create an on disk database by opening it (even if it doesn't exist) using .open and then entering commands. Alternately you can enter commands and then then use the .save command afterwards to save the in-memory database to disk.
So after the above you could do :-
sqlite> .open Test.db
sqlite> CREATE TABLE Persons ( PersonID int, LastName varchar(255), FirstName varchar(255), Address varchar(255), City varchar(255) );
sqlite> SELECT * FROM Sqlite_master;
table|Persons|Persons|3|CREATE TABLE Persons ( PersonID int, LastName varchar(255), FirstName varchar(255), Address varchar(255), City varchar(255) )
sqlite> .quit
This creates the file Test.db (which didn't exists, otherwise it would open it)
Then creates the Persons table.
And then extracts the rows in the system table named sqlite_master (lists items including tables).
Finally it quites from SQLITE3.
Perhaps you could then do
:-
C:\Users\Mike>sqlite3 SQLite version 3.22.0 2018-01-22 18:45:57 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database.
sqlite> .open Test.db
sqlite> .tables Persons
sqlite> INSERT INTO Persons VALUES (1,'Smith','Fred','Somewhere','London');
sqlite> select * FROM persons; 1|Smith|Fred|Somewhere|London sqlite> .quit
Starts SQLITE3
Opens the now existing Test.db file.
Lists the tables.
inserts a row
dispslays the rows in the Persons table (i.e. the row just added)
Finally quits SQLITE3
Opening the file Test.db in notepad :-
SQLite format 3 # .°
e e ‚tablePersonsPersonsCREATE TABLE Persons ( PersonID int, LastName varchar(255), FirstName varchar(255), Address varchar(255), City varchar(255) )
You may see some familiar data (e.g. the SQL used to create the table is stored in the sqlite_master table in the column named SQL and will be visible as is)
Other data will not be shown in a usefule/usable format.
i.e. You will find it between hard and impossible to enter the correct data to create a valid SQLite3 database file in notepad.
From everything I've read and searched on I think the answer is no because it's never mentioned. But I never saw an explicit no.
Does Sqlite allow for a schema like in Sql Server you can have dbo. Employee.FirstName? Or is it limited to Employee.FirstName?
thanks - dave
I don't think sqlite has the exact same concept as schemas have in some other databases but you can attach several databases and operate on them by name.
e.g.
$ sqlite3 mydb1
SQLite version 3.20.1 2017-08-24 16:21:36
sqlite> create table test1 (id int); insert into test1 values (1);
sqlite> .tables
test1
Above we just have 1 database, which is the default database and doesn't need to be prefixed by the name of the database. However the name of this default default database is main , so we can do:
sqlite> select * from test1;
1
sqlite> select * from main.test1;
1
We can attach another database, which will be available under the name myotherdb.
sqlite> attach database 'myotherdb' as 'myotherdb';
sqlite> create table myotherdb.test1 (id int); insert into myotherdb.test1 values (2);
Now we can use myotherdb to refer to this new database, and no prefix or the main prefix to refer to the first/default database
sqlite> select * from myotherdb.test1 where myotherdb.test1.id > 0;
2
sqlite> select * from test1 where test1.id > 0;
1
sqlite> select * from main.test1 where main.test1.id > 0;
1
Note that this will create 2 different database files
sqlite> .databases
main: /tmp/mydb1
myotherdb: /tmp/myotherdb
These 2 databases files can be opened individually later on.
For some reason the CSV file isn't imported to my database file:
sqlite> attach '/tmp/databases/d1.db' as d1;
sqlite> .mode csv
sqlite> .import '/tmp/1.csv' products
sqlite> .tables
products << not imported into my d1 database
sqlite> .databases
main:
d1: /tmp/databases/d1.db
I've tried the variation:
sqlite> .import '/tmp/1.csv' d1.products
Error: no such table: d1.products
For this .import command, the sqlite3 command-line shell will execute the following statements:
SELECT * FROM d1.products -- to check if the table exists
CREATE TABLE d1.products(...)
INSERT INTO "d1.products" VALUES(...) -- this fails
This inconsistent quoting obviously is a bug.
To work around this, open db1.db as the main database.
I have 2 SQLite databases with common data but with different purposes and I wanted to avoid reinserting data, so I was wondering if it was possible to copy a whole table from one database to another?
You'll have to attach Database X with Database Y using the ATTACH command, then run the appropriate Insert Into commands for the tables you want to transfer.
INSERT INTO X.TABLE SELECT * FROM Y.TABLE;
// "INSERT or IGNORE" if you want to ignore duplicates with same unique constraint
Or, if the columns are not matched up in order:
INSERT INTO X.TABLE(fieldname1, fieldname2) SELECT fieldname1, fieldname2 FROM Y.TABLE;
Easiest and correct way on a single line:
sqlite3 old.db ".dump mytable" | sqlite3 new.db
The primary key and the columns types will be kept.
Consider a example where I have two databases namely allmsa.db and atlanta.db. Say the database allmsa.db has tables for all msas in US and database atlanta.db is empty.
Our target is to copy the table atlanta from allmsa.db to atlanta.db.
Steps
sqlite3 atlanta.db(to go into atlanta database)
Attach allmsa.db. This can be done using the command ATTACH '/mnt/fastaccessDS/core/csv/allmsa.db' AS AM;
note that we give the entire path of the database to be attached.
check the database list using sqlite> .databases
you can see the output as
seq name file
--- --------------- ----------------------------------------------------------
0 main /mnt/fastaccessDS/core/csv/atlanta.db
2 AM /mnt/fastaccessDS/core/csv/allmsa.db
now you come to your actual target. Use the command
INSERT INTO atlanta SELECT * FROM AM.atlanta;
This should serve your purpose.
For one time action, you can use .dump and .read.
Dump the table my_table from old_db.sqlite
c:\sqlite>sqlite3.exe old_db.sqlite
sqlite> .output mytable_dump.sql
sqlite> .dump my_table
sqlite> .quit
Read the dump into the new_db.sqlite assuming the table there does not exist
c:\sqlite>sqlite3.exe new_db.sqlite
sqlite> .read mytable_dump.sql
Now you have cloned your table.
To do this for whole database, simply leave out the table name in the .dump command.
Bonus: The databases can have different encodings.
Objective-C code for copy Table from a Database to another Database
-(void) createCopyDatabase{
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory , NSUserDomainMask, YES);
NSString *documentsDir = [paths objectAtIndex:0];
NSString *maindbPath = [documentsDir stringByAppendingPathComponent:#"User.sqlite"];;
NSString *newdbPath = [documentsDir stringByAppendingPathComponent:#"User_copy.sqlite"];
NSFileManager *fileManager = [NSFileManager defaultManager];
char *error;
if ([fileManager fileExistsAtPath:newdbPath]) {
[fileManager removeItemAtPath:newdbPath error:nil];
}
sqlite3 *database;
//open database
if (sqlite3_open([newdbPath UTF8String], &database)!=SQLITE_OK) {
NSLog(#"Error to open database");
}
NSString *attachQuery = [NSString stringWithFormat:#"ATTACH DATABASE \"%#\" AS aDB",maindbPath];
sqlite3_exec(database, [attachQuery UTF8String], NULL, NULL, &error);
if (error) {
NSLog(#"Error to Attach = %s",error);
}
//Query for copy Table
NSString *sqlString = #"CREATE TABLE Info AS SELECT * FROM aDB.Info";
sqlite3_exec(database, [sqlString UTF8String], NULL, NULL, &error);
if (error) {
NSLog(#"Error to copy database = %s",error);
}
//Query for copy Table with Where Clause
sqlString = #"CREATE TABLE comments AS SELECT * FROM aDB.comments Where user_name = 'XYZ'";
sqlite3_exec(database, [sqlString UTF8String], NULL, NULL, &error);
if (error) {
NSLog(#"Error to copy database = %s",error);
}
}
The Easiest way to do is through SQLite Studio
If you don't have download from https://download.cnet.com/SQLiteStudio/3000-10254_4-75836135.html
Steps:
1.Add both the databases.
2.Click View tab and then databases as shown in the picture.
3.Right click the table you want to copy and copy it.
Paste the table after right clicking the database where you want to paste.
Now you're done
First scenario: DB1.sqlite and DB2.sqlite have the same table(t1), but DB1 is more "up to date" than DB2. If it's small, drop the table from DB2 and recreate it with the data:
> DROP TABLE IF EXISTS db2.t1; CREATE TABLE db2.t1 AS SELECT * FROM db1.t1;
Second scenario: If it's a large table, you may be better off with an INSERT if not exists type solution. If you have a Unique Key column it's more straight forward, otherwise you'd need to use a combination of fields (maybe every field) and at some point it's still faster to just drop and re-create the table; it's always more straight forward (less thinking required).
THE SETUP: open SQLite without a DB which creates a temporary in memory main database, then attach DB1.sqlite and DB2.sqlite
> sqlite3
sqlite> ATTACH "DB1.sqlite" AS db1
sqlite> ATTACH "DB2.sqlite" AS db2
and use .databases to see the attached databases and their files.
sqlite> .databases
main:
db1: /db/DB1.sqlite
db2: /db/DB2.sqlite
I needed to move data from a sql server compact database to sqlite, so using sql server 2008 you can right click on the table and select 'Script Table To' and then 'Data to Inserts'. Copy the insert statements remove the 'GO' statements and it executed successfully when applied to the sqlite database using the 'DB Browser for Sqlite' app.
If you use DB Browser for SQLite, you can copy the table from one db to another in following steps:
Open two instances of the app and load the source db and target db side by side.
If the target db does not have the table, "Copy Create Statement" from the source db and then paste the sql statement in "Execute SQL" tab and run the sql to create the table.
In the source db, export the table as a CSV file.
In the target db, import the CSV file to the table with the same table name. The app will ask you do you want to import the data to the existing table, click yes. Done.
This question already has answers here:
Is there an SQLite equivalent to MySQL's DESCRIBE [table]?
(7 answers)
Closed 7 years ago.
How can I see the structure of table in SQLite as desc was in Oracle?
PRAGMA table_info(table_name);
This will work for both: command-line and when executed against a connected database.
A link for more details and example. thanks
SQLite Pragma Command
Invoke the sqlite3 utility on the database file, and use its special dot commands:
.tables will list tables
.schema [tablename] will show the CREATE statement(s) for a table or tables
There are many other useful builtin dot commands -- see the documentation at http://www.sqlite.org/sqlite.html, section Special commands to sqlite3.
Example:
sqlite> entropy:~/Library/Mail>sqlite3 Envelope\ Index
SQLite version 3.6.12
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .tables
addresses ews_folders subjects
alarms feeds threads
associations mailboxes todo_notes
attachments messages todos
calendars properties todos_deleted_log
events recipients todos_server_snapshot
sqlite> .schema alarms
CREATE TABLE alarms (ROWID INTEGER PRIMARY KEY AUTOINCREMENT, alarm_id,
todo INTEGER, flags INTEGER, offset_days INTEGER,
reminder_date INTEGER, time INTEGER, argument,
unrecognized_data BLOB);
CREATE INDEX alarm_id_index ON alarms(alarm_id);
CREATE INDEX alarm_todo_index ON alarms(todo);
Note also that SQLite saves the schema and all information about tables in the database itself, in a magic table named sqlite_master, and it's also possible to execute normal SQL queries against that table. For example, the documentation link above shows how to derive the behavior of the .schema and .tables commands, using normal SQL commands (see section: Querying the database schema).
You can query sqlite_master
SELECT sql FROM sqlite_master WHERE name='foo';
which will return a create table SQL statement, for example:
$ sqlite3 mydb.sqlite
sqlite> create table foo (id int primary key, name varchar(10));
sqlite> select sql from sqlite_master where name='foo';
CREATE TABLE foo (id int primary key, name varchar(10))
sqlite> .schema foo
CREATE TABLE foo (id int primary key, name varchar(10));
sqlite> pragma table_info(foo)
0|id|int|0||1
1|name|varchar(10)|0||0
You should be able to see the schema by running
.schema <table>
.schema TableName
Where TableName is the name of the Table
You will get the structure by typing the command:
.schema <tableName>
If you are using PHP you can get it this way:
<?php
$dbname = 'base.db';
$db = new SQLite3($dbname);
$sturturequery = $db->query("SELECT sql FROM sqlite_master WHERE name='foo'");
$table = $sturturequery->fetchArray();
echo '<pre>' . $table['sql'] . '</pre>';
$db->close();
?>
You can use the Firefox add-on called SQLite Manager to view the database's structure clearly.