I am trying to switch off spool mode in tpt export. Unfortunately I am not able to identify the option to do so. I am new to teradata. Can anyone please help.
I saw one option in the internet , Spoolmode NOSPOOL. But not sure how to use it in the tpt export script.
If by spool you mean not spooling the data and output you can try bteq:
bteq << EOF >/dev/null
.sessions 16
.logon $TDSRV/$TDUSER,$TDPASSWD
.EXPORT FILE = $OUTPUT_DIR/$OUTPUT_FILE;
.SET WIDTH 12000
.SET TITLEDASHES OFF
SELECT * FROM TABLE;
.LOGOFF
.EXIT
EOF
you do this
VARCHAR SpoolMode = 'NOSPOOL'
Related
Everytime I start SQLite, I have to re-turn on headers, re-switch to column mode, re-change the separator and/or width. How can I make the settings persist??
Repeated Code
.mode column
.headers on
.separator ','
In other words, how can I save the settings so that next time I run SQLite, my preferences are automatically applied.
Create a .sqliterc file in your home directory
Ah, finally stumbled upon the answer. You can create a .sqliterc (SQLite Run Commands file) in your home directory. Every time you run sqlite, it will load the settings from the rc file.
Here's a quick command line script you can paste in the terminal for some basic settings (edit it to add additional preferences):
cat << EOF > ~/.sqliterc
.headers on
.mode column
EOF
There are alternative command line options. Something like this:
alias mysqlite='sqlite3 -column -header -separator ,'
I'm trying to export a csv file to a fixed length file.
I'm using SQLite because I have to process the file.
The problem I'm having is with the output.
Here is the code I am running from a sql file:
.output stdout --send output to screen
.echo off
.headers off
.separator ','
.mode column
.open tmp.db
DROP TABLE IF EXISTS myData;
.import 'C:\2015.csv' myData
.width 4 4 1 6 16 2 6
.output myData.txt --send output to file
select * from myData;
.output stdout --send output back to screen
.print "File was created."
DROP TABLE IF EXISTS myData;
vacuum;
.print "tmp.db vacuum complete"
.print "Type '.exit' or '.quit' to exit SQLite"
.exit
The problem is that in the first line, where I send the output to the screen, I get an error:
Error: unknown command or invalid arguments: "output". Enter ".help"
for help
The script executes, but it actually displays the results of the sql (and it also writes them to the file).
However, if I run every single command from the sqlite console, everything works fine. I only get the error when I run it from the SQLite console using .read
Any suggestions? What am I missing? I checked the docs in the SQLite website, and in the example presented they switch the output from console to file and back in the same manner.
Thank you.
I think I found the problem:
.output stdout --send output to screen
Apparently, the --send... was interpreted as an argument. Removing the -- and the comments after stdout fixed the problem.
Thanks!
The .output command expects a single parameter, not five.
This forms works fine:
.output stdout
I have a directory with lots and lots of SQLite database files.
I need to run an sqlite3 query to establish if any database in my directory contains a specific table. Database filenames go like 001.db, 002.db, etc. If a database contains the table I require, the file is copied to a different directory.
My initial thoughts are similar to this:
while [ sqlite3 $* "SELECT * FROM table1" != "Error: no such
table: table1" ]`
do
cp $* newdir/
done
or something similar, and I need some help with correct syntaxis please...
in "$*" I meant a database file currently being processed inside the directory, not sure if it is correct...
First, you don't need to do a select in a table to see if it exists. You could run something like:
SELECT * FROM dbname.sqlite_master WHERE type='table';
or adapt it to do a count:
SELECT count(*) FROM dbname.sqlite_master WHERE type='table' and name='table_name';
Using the sqlite command line, you should get a "1" or a "0". You can use that in figuring out if the DB actually has your desired table or not and act accordingly. The script could look something like:
for i in `ls *.db`; do
HAS_TABLE=`sqlite "${i}" "select count(*) ....;"
if [[ ${HAS_TABLE} == "1" ]]; then
cp ${i} some/other/dir/${i}
fi
done
I'm writing this from memory, so the syntax of the if/condition may be off a bit (you might be ok without the quotes around the 1 value).
HTH,
I'm making a shell script to export a sqlite query to a csv file, just like this:
#!/bin/bash
./bin/sqlite3 ./sys/xserve_sqlite.db ".headers on"
./bin/sqlite3 ./sys/xserve_sqlite.db ".mode csv"
./bin/sqlite3 ./sys/xserve_sqlite.db ".output out.csv"
./bin/sqlite3 ./sys/xserve_sqlite.db "select * from eS1100_sensor_results;"
./bin/sqlite3 ./sys/xserve_sqlite.db ".exit"
When executing the script, the output apears on the screen, instead of being saved to "out.csv". It's working doing the same method with the command line, but I don't know why the shell script fails to export data to the file.
What am I doing wrong?
Instead of the dot commands, you could use sqlite3 command options:
sqlite3 -header -csv my_db.db "select * from my_table;" > out.csv
This makes it a one-liner.
Also, you can run a sql script file:
sqlite3 -header -csv my_db.db < my_script.sql > out.csv
Use sqlite3 -help to see the list of available options.
sqlite3
You have a separate call to sqlite3 for each line; by the time your select runs, your .out out.csv has been forgotten.
Try:
#!/bin/bash
./bin/sqlite3 ./sys/xserve_sqlite.db <<!
.headers on
.mode csv
.output out.csv
select * from eS1100_sensor_results;
!
instead.
sh/bash methods
You can either call your script with a redirection:
$ your_script >out.csv
or you can insert the following as a first line in your script:
exec >out.csv
The former method allows you to specify different filenames, while the latter outputs to a specific filename. In both cases the line .output out.csv can be ignored.
I recently created a shell script that will be able to take the tables from a db file and convert them into csv files.
https://github.com/darrentu/convert-db-to-csv
Feel free to ask me any questions on my script :)
Although the question is about shell script, I think it will help few of those who are just bothered about transferring the data from the sqlite3 database to a csv file.
I found a very convinient way to do it with the firefox browser using SQLite Manager extension.
Simply connect to your sqlite database file in firefox ( SQlite manager -> connect database ) and then Table -> Export table. You will be served with some more options that you can just click and try....
In the end you get a csv file with the table u have chosen to be exported.
Using command line for Linux:
user#dell-Admin: sqlite3 #activate your sqlite database first
sqlite> .tables #search for tables if any available if already created one.
sqlite> .schema #if you want to check the schema of the table.
# once you find your table(s), then just do the following:
sqlite> .headers on #export along with headers (column names)
sqlite> .mode csv #file type is csv
sqlite> .output example.csv #you want to provide file name to export
sqlite> SELECT * from events; #If entire table is needed or select only required
sqlite> .quit #finally quit the sqlite3
Now search in your system for example.csv file and you will get it.
In one line is
sqlite3 -header -csv ./sys/xserve_sqlite.db "select * from eS1100_sensor_results;" >./out.csv
A synthesis of the answers till now:
function sqlite2csv-table() {
local db="${1}" table="${2}" output="${3}"
if test -z "$output" ; then
output="${db:r}_hi${table}.csv"
fi
[[ "$output" =~ '.csv$' ]] || output+='.csv'
echo "$0: outputting table '$table' to '$output'"
sqlite3 -header -csv "$db" "select * from ${table};" > "$output" || return $?
}
function sqlite2csv() {
local db="${1}" o="${2}"
tables=($(sqlite3 $db ".tables"))
local t
for table in $tables[#] ; do
sqlite2csv-table "$db" "$table" "${o}_${table}.csv"
done
}
Usage:
sqlite2csv some.db [/path/to/output]
I'm trying to bulk load a lot of data ( 5.5 million rows ) into an SQLite database file.
Loading via INSERTs seems to be far too slow, so I'm trying to use the sqlite3 command line tool and the .import command.
It works perfectly if I enter the commands by hand, but I can't for the life of me work out how to automate it from a script ( .bat file or python script; I'm working on a Windows machine ).
The commands I issue at the command line are these:
> sqlite3 database.db
sqlite> CREATE TABLE log_entry ( <snip> );
sqlite> .separator "\t"
sqlite> .import logfile.log log_entry
But nothing I try will get this to work from a bat file or python script.
I've been trying things like:
sqlite3 "database.db" .separator "\t" .import logfile.log log_entry
echo '.separator "\t" .import logfile.log log_entry' | sqlite3 database.db
Surely I can do this somehow?
Create a text file with the lines you want to enter into the sqlite command line program, like this:
CREATE TABLE log_entry ( );
.separator "\t"
.import logfile.log log_entry
and then just call sqlite3 database.db < commands.txt
Alternatively you can put everything in one shell script file (thus simplifying maintenance) using heredoc import.sh :
#!/bin/bash --
sqlite3 -batch $1 <<"EOF"
CREATE TABLE log_entry ( <snip> );
.separator "\t"
.import logfile.log log_entry
EOF
...and run it:
import.sh database.db
It makes it easier to maintain just one script file.
By the way, if you need to run it under Windows, Power Shell also features heredoc
In addition this approach helps to deal with lacking script parameter support. You can use bash variables:
#!/bin/bash --
table_name=log_entry
sqlite3 -batch $1 <<EOF
CREATE TABLE ${table_name} ( <snip> );
.separator "\t"
.import logfile.log ${table_name}
EOF
Or even do a trick like this:
#!/bin/bash --
table_name=$2
sqlite3 -batch $1 <<EOF
CREATE TABLE ${table_name} ( <snip> );
.separator "\t"
.import logfile.log ${table_name}
EOF
...and run it: import.sh database.db log_entry
Create a separate text file containing all the commands you would normally type into the sqlite3 shell app:
CREATE TABLE log_entry ( <snip> );
.separator "\t"
.import /path/to/logfile.log log_entry
Save it as, say, impscript.sql.
Create a batch file which calls the sqlite3 shell with that script:
sqlite3.exe yourdatabase.db < /path/to/impscript.sql
Call the batch file.
On a side note - when importing, make sure to wrap the INSERTs in a transaction! That will give you an instant 10.000% speedup.
I just recently had a similar problem while converting Firefox' cookies.sqlite to a text file (for some downloading tool) and stumbled across this question.
I wanted to do that with a single shell line and that would be my solution applied to the above mentioned problem:
echo -e ".mode tabs\n.import logfile.log log_entry" | sqlite3 database.db
But I haven't tested that line yet. But it worked fine with the Firefox problem I mentioned above (btw via Bash on Mac OSX ):
echo -e ".mode tabs\nselect host, case when host glob '.*' then 'TRUE' else 'FALSE' end, path, case when isSecure then 'TRUE' else 'FALSE' end, expiry, name, value from moz_cookies;" | sqlite3 cookies.sqlite
sqlite3 abc.db ".read scriptname.sql"
At this point, I'm not sure what else I can add other than, I had some trouble adding a unix environment variable to the bash script suggested by nad2000.
running this:
bash dbmake.sh database.db <(sed '1d' $DATA/logfile.log | head -n 1000)
I needed to import from stdin as workaround and I found this solution:
sqlite3 $1 <<"EOF"
CREATE TABLE log_entry;
EOF
sqlite3 -separator $'\t' $1 ".import $2 log_entry"
By adding the second sqlite3 line, I was able to pass the $2 from Unix into the file parameter for .import, full path and everything.
On Windows, this should work:
(echo CREATE TABLE log_entry ( <snip> ); & echo .separator "\t" & echo .import logfile.log log_entry) | sqlite3.exe database.db
I haven't tested this particular command but from my own pursuit of solving this issue of piping multiple commands I found that the key was to enclose the echoed commands within parentheses. That being said, it is possible that you may need to tweak the above command to also escape some of those characters. For example:
(echo CREATE TABLE log_entry ^( ^<snip^> ^); & echo .separator "\t" & echo .import logfile.log log_entry) | sqlite3.exe database.db
I'm not sure if the escaping is needed in this case, but it is highly probable since the parentheses may conflict with the enclosing ones, then the "less than" and "greater than" symbols are usually interpreted as input or output which may also conflict. An extensive list of characters' escape may be found here: http://www.robvanderwoude.com/escapechars.php
here trans is table name and trans.csv is a csv file in which i have 1959 rows of data
$ sqlite3 abc.db ".separator ','"
$ sqlite3 abc.db ".import 'trans.csv' trans"
$ sqlite3 abc.db "select count(*) from trans;"
1959
but its impossible to write like as you wrote