I have started to use (or, try to use) sqlite for a simple catalog. What I want to do is be able to take out the information for each catalogued item from the sqlite, and export it into a text file.
e.g.
Title1, Genre1, Author1
Title2, Genre2, Author2
Title3, Genre3, Author3
I don't want these to be in columns, just a single line. Also, is there a way to use multiple different separators?
This seems like it should be relatively easy to do, but I am totally new to this and can't figure it out.
sqlite -list -separator ', ' db.db 'select * from thetable'
should do.
Like #hacker said, you can use sqlite -list when you need to specify the delimiter character. You can also use simple output such as sqlite -csv if you don't need to be terribly specific.
You probably want to refer to the SQLite command line documentation for more information about generating textual output from an interactive SQLite command, or you could man sqlite too, although it's similar information.
Related
I can do it using the GUI (not that hard) but I really would like do to it by sqlite command lines. I've googled it and have tried everything, however nothing seems to work. Please give me a hint on this! This is the last thing I've tried:
CREATE TABLE 'teste3' (
'Id' integer,
'Idade' integer,
'Sexo' text,
'Peso' integer
);
.separator ',';
.mode csv;
.import 'C:\Users\xxxx\Documents\Monografia\base_teste.csv' teste3
What I intended to do was to create a table ('teste3',done) and them "fill it" by importing a given .csv file. Instead, I keep getting this error message: "near ".": syntax error:". Then I tried to cut off the "." before separator, for example, but I got another error: "near "separator": syntax error:". I really don't know what to do. Thanks!
Dot commands like .mode and .import are not SQL statements; they are implemented by the sqlite3.exe command-line shell (which can be downloaded from the offical SQLite site).
The DB Browser for SQLite is an entirely independent tool. It does not implement these dot commands; you have to use the GUI instead.
I wrote the little bash script below and it works as intended, but I added couple comments and newlines for readability which breaks the code. Removing comments and newlines should make it a valid script.
### read all measurements from the database and list each value only once
sqlite3 -init /tmp/timeout /tmp/testje.sqlite \
'select distinct measurement from errors order by measurement;' |
### remove the first line of stdout as this is a notification rather than intended output
sed '1d' |
### loop though all found values
while read error; do
### count the number of occurences in the original table and print that
sqlite3 -init /tmp/timeout /tmp/testje.sqlite \
"select $error,count( measurement ) from errors where measurement = '$error' ;"
done
The result is like this:
134 1
136 1
139 2
159 1
Question: Is it possible with sqlite3 to translate the while-loop to SQL statements? In other words, does sqlite3 support some sort of for-loop to loop through results of a previous query?
Now I know sqlite3 is a very limited database and chances are that what I want is just too complex for it. I've been searching, for it but I'm really a database nitwit and the hits I get so far are either on a different database or solving an entirely different problem.
The easiest answer (that I do not hope for BTW) is 'sqlite3 does not support loops'.
SQLite does not support loops. Here is the entire language, you'll notice that structured programming is completely absent.
However, that's not to say that you can't get what you want without loops, using sets or some other SQL construct instead. In your case it might be as simple as:
select measurement, count( measurement ) from errors GROUP BY measurement
That will give you a list of all measurements in the errors table and a count of how often each one occurs.
In general, SQL engines are best utilized by expressing your query in a single (sometimes complex) SQL statement, which is submitted to the engine for optimization. In your example you've already codified some decisions about the strategy used to get the data from the database -- it's a tenet of SQL that the engine is better able to make those decisions than the programmer.
I have a SQLITE3 database wherein I have stored various columns. One column (cmd) in particular contains the full command line and associated parameters. Is there a way to extract just the first word in this column (just before the first space)? I am not interested in seeing the various parameters used, but do want to see the command issued.
Here's an example:
select cmd from log2 limit 3;
user-sync //depot/PATH/interface.h
user-info
user-changes -s submitted //depot/PATH/build/...#2011/12/06:18:31:10,#2012/01/18:00:05:55
From the result above, I'd like to use an inline SQL function (if available in SQLITE3) to parse on the first instance of space, and perhaps use a left function call (I know this is not available in SQLITE3) to return just the "user-sync" string. Same for "user-info" and "user-changes".
Any ideas?
Thanks.
My soluion:
sqlite> CREATE TABLE command (cmd TEXT);
sqlite> INSERT INTO command (cmd) VALUES ('ls'),('cd ~'),(' mpv movie.mkv ');
sqlite> SELECT substr(trim(cmd),1,instr(trim(cmd)||' ',' ')-1) FROM command;
ls
cd
mpv
Pros:
it's not that a dirty hack
it only uses core functions
"Finds the first occurrence" function is one of the SQLite3 Core Functions (http://www.sqlite.org/lang_corefunc.html).
Of course, it is much better to use instr(X,Y).
So you can write:
SELECT substr(cmd,1,instr(cmd,' ')-1) FROM log2
As the position of your first space character is unknown, I don't think there is a corefunction in SQLite that will help.
I think you'll have to create one http://www.sqlite.org/c3ref/create_function.html
Here's a hack
sqlite> create table test (a);
sqlite> insert into test values ("This is a test.");
sqlite> select * from test;
This is a test.
sqlite> select rtrim(substr(replace(a,' ','----------------------------------------------------------------------------------------'),1,80),'-') from test;
This
It works as long as your longest command is less than 80 characters (and you include 80 '-' characters in the substitution string -- I didn't count them!). If your commands can contain '-' just use a different character that is not allowed in the commands.
I don't believe that's something you'll be able to do within the SQL itself. SQLite's support for string handling functions is not as extensive as other RDBMSs (some of which would let you do a SUBSTR with a Reg Exp).
My suggestion is either to write your own SQL function as suggested by #Jon or just do it as a post-processing step in your app code.
New to databasing, so please let me know if I'm going about this entirely wrong.
I want to use databases to store large datasets (I use R to analyze data, which cannot load datasets larger than available RAM) and and I'm using SQLite-Manager in FireFox to import .csv files. 99% of the time I use reals, but would like to avoid all the clicking to manually cast each of 100 columns as REAL (the default in SQLite-Manager is TEXT).
Is there a way I can I can quickly/easily cast all columns as REAL? Thanks!
why don't you make a script to be interpreted by the SQLite shell?
Run sqlite my_db < script.txt with contents of scripts.txt following:
CREATE TABLE foo(
col1 REAL,
col2 REAL,
[...] generate those lines with a decent text editor
);
.separator ;
.import 'my/csv/file.csv' foo
.q
Note that dot-commands of the SQLite shell are available using “.help”. Imports are rudimentary and won't work if you have double quotes (remove them). Only the , is interpreted as a separator, you cannot escape it. If needed you can use a multicharacter separator.
Also be sure that file.csv is UTF8-encoded.
Sometimes when copying stuff into PostgreSQL I get errors that there's invalid byte sequences.
Is there an easy way using either vim or other utilities to detect byte sequences that cause errors such as: invalid invalid byte sequence for encoding "UTF8": 0xde70 and whatnot, and possibly and easy way to do a conversion?
Edit:
What my workflow is:
Dumped sqlite3 database (from trac)
Trying to replay it in postgresql
Perhaps there's an easier way?
More Edit:
Also tried these:
Running enca to detect encoding of the file
Told me it was ASCII
Tried iconv to convert from ASCII to UTF8. Got an error
What did work is deleting the couple erroneous lines that it complained about. But that didn't really solve the real problem.
Based on one short sentence, it sounds like you have text in one encoding (e.g. ANSI/ASCII) and you are telling PostgreSQL that it's actually in another encoding (Unicode UTF8). All the different tools you would be using: PostgreSQL, Bash, some programming language, another programming language, other data from somewhere else, the text editor, the IDE, etc., all have default encodings which may be different, and some step of the way, the proper conversions are not being done. I would check the flow of data where it crosses these kinds of boundaries, to ensure that either the encodings line up, or the encodings are properly detected and the text is properly converted.
If you know the encoding of the dump file, you can convert it to utf-8 by using recode. For example, if it is encoded in latin-1:
recode latin-1..utf-8 < dump_file > new_dump_file
If you are not sure about the encoding, you should see how sqlite was configured, or maybe try some trial-and-error.
I figured it out. It wasn't really an encoding issue.
SQLite's output escaped strings differently than Postgres expects. There were some cases where 'asdf\xd\foo' was outputted. I believe the '\x' was causing it to expect the following characters to be unicode encoding.
Solution to this is dumping each table individually in CSV mode in sqlite 3.
First
sqlite3 db/trac.db .schema | psql
Now, this does the trick for the most part to copy the data back in
for table in `sqlite3 db/trac.db .schema | grep TABLE | sed 's/.*TABLE \(.*\) (/\1/'`
do
echo ".mode csv\nselect * from $table;" | sqlite3 db/trac.db | psql -c "copy $table from stdin with csv"
done
Yeah, kind of a hack, but it works.