I would like to use the main database file location in function (replace or other) as input data. How can i get it?
for example: replace('c:\temp\main.db','main.db','')
I'm creating a database script for command line use under win10.
.database command result seems to be fine, but how to use in replace input?
EDIT: I have to use the location as variable, because i run the script from command line, recursively in a lot of directories with a lot of small databases, so i can't use constant value in replace.
The cmd line command:
forfiles /p c:\temp /m files.db /s /c "cmd /c c:\sqlite3.exe #path < c:\BookDbDataCopy.txt"
I want to use the path in the select as variable.
I got it :)
create table path (a text);
.mode csv
.output 'c:\data.csv'
.databases
.import 'c:\data.csv' path
The solution:
- create a table for the path
- write the command output in file
- import the file in table
- select the table in replace variable :)
select replace((select a from path where a like 'main%'),'main: c:\',''), t.* from path;
Everytime I start SQLite, I have to re-turn on headers, re-switch to column mode, re-change the separator and/or width. How can I make the settings persist??
Repeated Code
.mode column
.headers on
.separator ','
In other words, how can I save the settings so that next time I run SQLite, my preferences are automatically applied.
Create a .sqliterc file in your home directory
Ah, finally stumbled upon the answer. You can create a .sqliterc (SQLite Run Commands file) in your home directory. Every time you run sqlite, it will load the settings from the rc file.
Here's a quick command line script you can paste in the terminal for some basic settings (edit it to add additional preferences):
cat << EOF > ~/.sqliterc
.headers on
.mode column
EOF
There are alternative command line options. Something like this:
alias mysqlite='sqlite3 -column -header -separator ,'
I want the filename to be something like this top_queries_June.csv. Is it possible to do that in sqlite?
sqlite>.output (top_Queries + Select date('now')).csv
I run this batch file to create a file with the top queries.
Batch file:
sqlite3 mydb.db ".read x.sql"
x.sql:
.headers ON
.mode csv
.output Top_Queries + <current month>.csv
select * from query;
.quit
As #CL. already pointed out the sqlite3 command line utility is not flexible enough to do this. You appear to be using some kind of shell to execute sqlite3. Most shells support a feature called "here document" and the sqlite3 utility supports reading commands from a pipe. Using these you can inline the x.sql file into your shell script. In a POSIX compatible shell your code could look like this:
DATE=`date -I`
sqlite3 mydb.db <<EOF
.headers ON
.mode csv
.output Top_Queries${DATE}.csv
select * from query;
EOF
I have two databases that are created in a bash file, let's call them 1.sqlite and 2.sqlite. I'd like to combine them into a new database (3.sqlite). Is there a command in bash that'll do that?
Edit:
The answer that sputnick gave me got me on the right track. But I had to rearrange it to be:
sqlite3 1.sqlite .dump > tempdb.sqlite
Yes, try the following shell code:
sqlite3 .dump 1.sqlite > dump
sqlite3 .dump 2.sqlite >> dump
sqlite3 3.sqlite < dump
But take care of table collisions if they have the same names.
I'm trying to bulk load a lot of data ( 5.5 million rows ) into an SQLite database file.
Loading via INSERTs seems to be far too slow, so I'm trying to use the sqlite3 command line tool and the .import command.
It works perfectly if I enter the commands by hand, but I can't for the life of me work out how to automate it from a script ( .bat file or python script; I'm working on a Windows machine ).
The commands I issue at the command line are these:
> sqlite3 database.db
sqlite> CREATE TABLE log_entry ( <snip> );
sqlite> .separator "\t"
sqlite> .import logfile.log log_entry
But nothing I try will get this to work from a bat file or python script.
I've been trying things like:
sqlite3 "database.db" .separator "\t" .import logfile.log log_entry
echo '.separator "\t" .import logfile.log log_entry' | sqlite3 database.db
Surely I can do this somehow?
Create a text file with the lines you want to enter into the sqlite command line program, like this:
CREATE TABLE log_entry ( );
.separator "\t"
.import logfile.log log_entry
and then just call sqlite3 database.db < commands.txt
Alternatively you can put everything in one shell script file (thus simplifying maintenance) using heredoc import.sh :
#!/bin/bash --
sqlite3 -batch $1 <<"EOF"
CREATE TABLE log_entry ( <snip> );
.separator "\t"
.import logfile.log log_entry
EOF
...and run it:
import.sh database.db
It makes it easier to maintain just one script file.
By the way, if you need to run it under Windows, Power Shell also features heredoc
In addition this approach helps to deal with lacking script parameter support. You can use bash variables:
#!/bin/bash --
table_name=log_entry
sqlite3 -batch $1 <<EOF
CREATE TABLE ${table_name} ( <snip> );
.separator "\t"
.import logfile.log ${table_name}
EOF
Or even do a trick like this:
#!/bin/bash --
table_name=$2
sqlite3 -batch $1 <<EOF
CREATE TABLE ${table_name} ( <snip> );
.separator "\t"
.import logfile.log ${table_name}
EOF
...and run it: import.sh database.db log_entry
Create a separate text file containing all the commands you would normally type into the sqlite3 shell app:
CREATE TABLE log_entry ( <snip> );
.separator "\t"
.import /path/to/logfile.log log_entry
Save it as, say, impscript.sql.
Create a batch file which calls the sqlite3 shell with that script:
sqlite3.exe yourdatabase.db < /path/to/impscript.sql
Call the batch file.
On a side note - when importing, make sure to wrap the INSERTs in a transaction! That will give you an instant 10.000% speedup.
I just recently had a similar problem while converting Firefox' cookies.sqlite to a text file (for some downloading tool) and stumbled across this question.
I wanted to do that with a single shell line and that would be my solution applied to the above mentioned problem:
echo -e ".mode tabs\n.import logfile.log log_entry" | sqlite3 database.db
But I haven't tested that line yet. But it worked fine with the Firefox problem I mentioned above (btw via Bash on Mac OSX ):
echo -e ".mode tabs\nselect host, case when host glob '.*' then 'TRUE' else 'FALSE' end, path, case when isSecure then 'TRUE' else 'FALSE' end, expiry, name, value from moz_cookies;" | sqlite3 cookies.sqlite
sqlite3 abc.db ".read scriptname.sql"
At this point, I'm not sure what else I can add other than, I had some trouble adding a unix environment variable to the bash script suggested by nad2000.
running this:
bash dbmake.sh database.db <(sed '1d' $DATA/logfile.log | head -n 1000)
I needed to import from stdin as workaround and I found this solution:
sqlite3 $1 <<"EOF"
CREATE TABLE log_entry;
EOF
sqlite3 -separator $'\t' $1 ".import $2 log_entry"
By adding the second sqlite3 line, I was able to pass the $2 from Unix into the file parameter for .import, full path and everything.
On Windows, this should work:
(echo CREATE TABLE log_entry ( <snip> ); & echo .separator "\t" & echo .import logfile.log log_entry) | sqlite3.exe database.db
I haven't tested this particular command but from my own pursuit of solving this issue of piping multiple commands I found that the key was to enclose the echoed commands within parentheses. That being said, it is possible that you may need to tweak the above command to also escape some of those characters. For example:
(echo CREATE TABLE log_entry ^( ^<snip^> ^); & echo .separator "\t" & echo .import logfile.log log_entry) | sqlite3.exe database.db
I'm not sure if the escaping is needed in this case, but it is highly probable since the parentheses may conflict with the enclosing ones, then the "less than" and "greater than" symbols are usually interpreted as input or output which may also conflict. An extensive list of characters' escape may be found here: http://www.robvanderwoude.com/escapechars.php
here trans is table name and trans.csv is a csv file in which i have 1959 rows of data
$ sqlite3 abc.db ".separator ','"
$ sqlite3 abc.db ".import 'trans.csv' trans"
$ sqlite3 abc.db "select count(*) from trans;"
1959
but its impossible to write like as you wrote