I'm trying to export a csv file to a fixed length file.
I'm using SQLite because I have to process the file.
The problem I'm having is with the output.
Here is the code I am running from a sql file:
.output stdout --send output to screen
.echo off
.headers off
.separator ','
.mode column
.open tmp.db
DROP TABLE IF EXISTS myData;
.import 'C:\2015.csv' myData
.width 4 4 1 6 16 2 6
.output myData.txt --send output to file
select * from myData;
.output stdout --send output back to screen
.print "File was created."
DROP TABLE IF EXISTS myData;
vacuum;
.print "tmp.db vacuum complete"
.print "Type '.exit' or '.quit' to exit SQLite"
.exit
The problem is that in the first line, where I send the output to the screen, I get an error:
Error: unknown command or invalid arguments: "output". Enter ".help"
for help
The script executes, but it actually displays the results of the sql (and it also writes them to the file).
However, if I run every single command from the sqlite console, everything works fine. I only get the error when I run it from the SQLite console using .read
Any suggestions? What am I missing? I checked the docs in the SQLite website, and in the example presented they switch the output from console to file and back in the same manner.
Thank you.
I think I found the problem:
.output stdout --send output to screen
Apparently, the --send... was interpreted as an argument. Removing the -- and the comments after stdout fixed the problem.
Thanks!
The .output command expects a single parameter, not five.
This forms works fine:
.output stdout
Related
I have a Teradata bteq script exporting table definitions by running show table from a linux server. My problem is the script outputs the ddl into a file with new lines (^M) in each line and random characters at the beginning of the script. I can post process the file with linux commands to remove ^M. But since the characters at the beginning of the files are random it is difficult to code to remove them.
Script below:
#!/bin/sh
#########################################
DATADIR=[directoryname]
bteq <<EOI
.logmech LDAP
.LOGON server/user/pass;
.EXPORT DATA FILE=$DATADIR/script.sql
SHOW TABLE mydatabasename.mytablename;
.EXPORT RESET;
.IF ERRORCODE <> 0 THEN .EXIT ERRORCODE;
.LOGOFF
.EXIT
EOI
#########################################
First line of the export looks like below:
a^E_^ECREATE MULTISET TABLE mydatabasename.mytablename ,FALLBACK ,^M
Is there something I can do in the script to avoid getting the special characters?
Everytime I start SQLite, I have to re-turn on headers, re-switch to column mode, re-change the separator and/or width. How can I make the settings persist??
Repeated Code
.mode column
.headers on
.separator ','
In other words, how can I save the settings so that next time I run SQLite, my preferences are automatically applied.
Create a .sqliterc file in your home directory
Ah, finally stumbled upon the answer. You can create a .sqliterc (SQLite Run Commands file) in your home directory. Every time you run sqlite, it will load the settings from the rc file.
Here's a quick command line script you can paste in the terminal for some basic settings (edit it to add additional preferences):
cat << EOF > ~/.sqliterc
.headers on
.mode column
EOF
There are alternative command line options. Something like this:
alias mysqlite='sqlite3 -column -header -separator ,'
When I open the sqlite3 command line in Windows Command Prompt, I each time type
.mode column
.width 8
.header on
Sometimes I choose a different width, but this is easy to change as needed.
Is there some way I can have these either as defaults, or execute each time the prompt starts?
The default rc file for SQLite is ~/.sqliterc; see the man page. You just paste that lines there.
Put these commands into a file, and give the file name to sqlite3's -init option.
(The options are not in the documentation; see sqlite3 --help for a list.)
I have a text file which contains an sqlite statment where I want to dump it into .sql format but not able to excute it. Any help would be appreciated. My file is stored on desktop as verbali.txt and when i go to command prompt and to specific sqlite folder and run it just open the text file .
.mode insert
.header on
.out file.sql
select select Id ,CompanyId ,DateTime ,Serial ,DeviceId ,AgentAId ,GpsAddress ,Targa ,CommonRoadDescription ,RoadCivicNumber ,VehicleBrandDescription ,VehicleModelDescription ,VerbaliVehicleTypeDescription ,CommonColorVehicleDescription ,VerbaliRuleOneCode ,VerbaliRuleOneDescription ,VerbaliRuleOnePoints ,VerbaliClosedNoteDescription ,Points ,VerbaliMissedNotificationDescription ,MissedNotificationNote ,StatementNote from VerbaliData
Pipe the file to sqlite3, in command prompt:
sqlite3 yourdatabase.db <path\to\verbali.txt
where sqlite3 is the sqlite3.exe command line tool.
I'm making a shell script to export a sqlite query to a csv file, just like this:
#!/bin/bash
./bin/sqlite3 ./sys/xserve_sqlite.db ".headers on"
./bin/sqlite3 ./sys/xserve_sqlite.db ".mode csv"
./bin/sqlite3 ./sys/xserve_sqlite.db ".output out.csv"
./bin/sqlite3 ./sys/xserve_sqlite.db "select * from eS1100_sensor_results;"
./bin/sqlite3 ./sys/xserve_sqlite.db ".exit"
When executing the script, the output apears on the screen, instead of being saved to "out.csv". It's working doing the same method with the command line, but I don't know why the shell script fails to export data to the file.
What am I doing wrong?
Instead of the dot commands, you could use sqlite3 command options:
sqlite3 -header -csv my_db.db "select * from my_table;" > out.csv
This makes it a one-liner.
Also, you can run a sql script file:
sqlite3 -header -csv my_db.db < my_script.sql > out.csv
Use sqlite3 -help to see the list of available options.
sqlite3
You have a separate call to sqlite3 for each line; by the time your select runs, your .out out.csv has been forgotten.
Try:
#!/bin/bash
./bin/sqlite3 ./sys/xserve_sqlite.db <<!
.headers on
.mode csv
.output out.csv
select * from eS1100_sensor_results;
!
instead.
sh/bash methods
You can either call your script with a redirection:
$ your_script >out.csv
or you can insert the following as a first line in your script:
exec >out.csv
The former method allows you to specify different filenames, while the latter outputs to a specific filename. In both cases the line .output out.csv can be ignored.
I recently created a shell script that will be able to take the tables from a db file and convert them into csv files.
https://github.com/darrentu/convert-db-to-csv
Feel free to ask me any questions on my script :)
Although the question is about shell script, I think it will help few of those who are just bothered about transferring the data from the sqlite3 database to a csv file.
I found a very convinient way to do it with the firefox browser using SQLite Manager extension.
Simply connect to your sqlite database file in firefox ( SQlite manager -> connect database ) and then Table -> Export table. You will be served with some more options that you can just click and try....
In the end you get a csv file with the table u have chosen to be exported.
Using command line for Linux:
user#dell-Admin: sqlite3 #activate your sqlite database first
sqlite> .tables #search for tables if any available if already created one.
sqlite> .schema #if you want to check the schema of the table.
# once you find your table(s), then just do the following:
sqlite> .headers on #export along with headers (column names)
sqlite> .mode csv #file type is csv
sqlite> .output example.csv #you want to provide file name to export
sqlite> SELECT * from events; #If entire table is needed or select only required
sqlite> .quit #finally quit the sqlite3
Now search in your system for example.csv file and you will get it.
In one line is
sqlite3 -header -csv ./sys/xserve_sqlite.db "select * from eS1100_sensor_results;" >./out.csv
A synthesis of the answers till now:
function sqlite2csv-table() {
local db="${1}" table="${2}" output="${3}"
if test -z "$output" ; then
output="${db:r}_hi${table}.csv"
fi
[[ "$output" =~ '.csv$' ]] || output+='.csv'
echo "$0: outputting table '$table' to '$output'"
sqlite3 -header -csv "$db" "select * from ${table};" > "$output" || return $?
}
function sqlite2csv() {
local db="${1}" o="${2}"
tables=($(sqlite3 $db ".tables"))
local t
for table in $tables[#] ; do
sqlite2csv-table "$db" "$table" "${o}_${table}.csv"
done
}
Usage:
sqlite2csv some.db [/path/to/output]