I have a table in a sqlite database with the following schema
CREATE TABLE os_logs (version STRING, user STRING, date STRING);
I set the following command to a variable called cmd.
select count(*), version
from os_logs
group by version
order by version;
After I send that command through dbGetQuery I get numeric results back for version instead of a string.
db <- dbConnect(SQLite(),"./os_backup.db")
dbGetQuery(db,cmd)
count(*) version
1421 NA
1797 0.7
6 0.71
2152 0.71
1123 0.72
3455 1
2335 1
The versions should be
0.70.1111_Product
0.71.22_Dev
0.71.33_Product
...
Any idea on why the strings in my sqlite database are being turned into numerics in R? If I do that command on the sql cmd line it works perfectly
Edit:
Here is how the tables are created. (With more info since I edited it out in the original question.
drop table vocf_logs;
CREATE TABLE vocf_logs (version STRING, driver STRING, dir STRING, uuid STRING PRIMARY KEY, t_start STRING);
CREATE TABLE log_os (uuid STRING PRIMARY KEY, os STRING);
.separator ","
.import vocf_dirs.csv vocf_logs
-- Put the OsVersion info from name_redacted into the table
UPDATE vocf_logs
SET version=(select log_os.os from log_os where uuid = vocf_logs.uuid);
What you describe should work fine. You must have done something differently or inserted it incorrectly to the db.
Here is a step by step test that does the exact same and works:
# Load package and connect
R> library(RSQLite)
R> db <- dbConnect(SQLite(),"./os_backup.db")
# Create db and insert data
R> dbSendQuery(db, "CREATE TABLE os_logs (version STRING, user STRING, date STRING);")
R> dbSendQuery(db, "INSERT INTO os_logs VALUES ('0.70.1111_Product', 'while', '2015-04-23')")
R> dbSendQuery(db, "INSERT INTO os_logs VALUES ('0.70.1111_Product', 'while', '2015-04-24')")
R> dbSendQuery(db, "INSERT INTO os_logs VALUES ('0.71.22_Dev', 'while', '2015-04-24')")
# Run query counting versions
R> dbGetQuery(db, "SELECT version, count(*) FROM os_logs GROUP BY version ORDER BY version;")
version count(*)
1 0.70.1111_Product 2
2 0.71.22_Dev 1
The creation of the original table was wrong. The method in R was correct. From the data type descriptions found here: https://www.sqlite.org/datatype3.html.
The declared type of "STRING" has an affinity of NUMERIC, not TEXT.
When table was created using type TEXT it worked as expected.
Related
Julia, SQLite3
Hi! I have an SQLite3 database and want to get information from it into a string, not DataFrame.
# Connection to database
db = SQLite.DB(raw"pictures.sqlite")
# Creation of table
SQLite.execute(db, "DROP TABLE IF EXISTS Files")
SQLite.execute(db, "CREATE TABLE IF NOT EXISTS Files
(ID INTEGER PRIMARY KEY AUTOINCREMENT,
FullName TEXT,
creation_date TEXT,
change_date TEXT,
size INTEGER,
UNIQUE (ID))")
#Another code
#This is a way to add rows to database (it works)
DBInterface.execute(db,
"INSERT INTO Files (FullName, creation_date, change_date, size)
VALUES
('$fileName', '$cdate', '$mdate', '$fileSize')
")
#Then i am trying to get one row into a string
strq = SQLite.DBInterface.execute(db, "SELECT * FROM Files WHERE ID=3")
#I can't transfornm these to string
I'm writing on Julia and need help!!
TLDR:
CSV.write(stdout, strq);
Explanation
The API of SQLite.jl assumes using DBInterface.execute for querying the database.
This method yields an object of type SQLite.Query.
Let's have a look what can we do with it:
julia> methodswith(SQLite.Query, supertypes=true)
[1] eltype(q::SQLite.Query) in SQLite at ~/.julia/packages/SQLite/wpQeE/src/tables.jl:51
[2] isempty(q::SQLite.Query) in SQLite at ~/.julia/packages/SQLite/wpQeE/src/tables.jl:15
[3] iterate(q::SQLite.Query) in SQLite at ~/.julia/packages/SQLite/wpQeE/src/tables.jl:94
[4] iterate(q::SQLite.Query, rownumber) in SQLite at ~/.julia/packages/SQLite/wpQeE/src/tables.jl:100
You can see one important thing - you can do whatever you want with SQLite.Query as long as it matches the Tables.jl interfaces. So you are not obliged to use DataFrames but there is no "pure string" implementation, such thing would not make any sense neither.
However, there are many interfaces for Tables. The one you need is CSV.jl will exactly give you something like "string representation" of your SQL table.
Try this with your code (I inserted one row to my table to see the results):
julia> CSV.write(stdout, strq);
ID,FullName,creation_date,change_date,size
1,somefilename,somecdate,somemfate,0
Today, for the first time I discovered sqldf package which I found to be very useful and convenient. Here is what the documentation says about the package:
https://www.rdocumentation.org/packages/sqldf/versions/0.4-11
sqldf is an R package for runing SQL statements on R data frames,
optimized for convenience. The user simply specifies an SQL statement
in R using data frame names in place of table names and a database
with appropriate table layouts/schema is automatically created, the
data frames are automatically loaded into the database, the specified
SQL statement is performed, the result is read back into R and the
database is deleted all automatically behind the scenes making the
database's existence transparent to the user who only specifies the
SQL statement.
So if I understand correctly, some data.frame which contains data stored in the RAM of the computer is mapped into a database on the disk temporarily as a table, then the calculation or whatever the query is supposed to do will be done and finally the result is returned back to R and all that was temporarily created in the database goes away as it never existed.
My question is, does it work other way around? Meaning, that assuming there is already a table let's say named my_table (just an example) in the database (I use PostgreSQL), is there any way to import its data from the database into a data.frame in R via sqldf? Because, currently the only way that I know is RPostgreSQL.
Thanks to G. Grothendieck for the answer. Indeed it is perfectly possible to select data from already existing tables in the database. My mistake was that I was thinking that the name of the dataframe and the corresponding table must always be the same, whereas if I understand correctly, this is only the case when a data.frame data is mapped to a temporary table in the database. As a result when I tried to select data, I had an error message saying that a table with the same name already existed in my database.
Anyway, just as a test to see whether this works, I did the following in PostgreSQL (postgres user and test database which is owned by postgres)
test=# create table person(fname text, lname text, email text);
CREATE TABLE
test=# insert into person(fname, lname, email) values ('fname-01', 'lname-01', 'fname-01.lname-01#gmail.com'), ('fname-02', 'lname-02', 'fname-02.lname-02#gmail.com'), ('fname-03', 'lname-03', 'fname-03.lname-03#gmail.com');
INSERT 0 3
test=# select * from person;
fname | lname | email
----------+----------+-----------------------------
fname-01 | lname-01 | fname-01.lname-01#gmail.com
fname-02 | lname-02 | fname-02.lname-02#gmail.com
fname-03 | lname-03 | fname-03.lname-03#gmail.com
(3 rows)
test=#
Then I wrote the following in R
options(sqldf.RPostgreSQL.user = "postgres",
sqldf.RPostgreSQL.password = "postgres",
sqldf.RPostgreSQL.dbname = "test",
sqldf.RPostgreSQL.host = "localhost",
sqldf.RPostgreSQL.port = 5432)
###
###
library(tidyverse)
library(RPostgreSQL)
library(sqldf)
###
###
result_df <- sqldf("select * from person")
And indeed we can see that result_df contains the data stored in the table person.
> result_df
fname lname email
1 fname-01 lname-01 fname-01.lname-01#gmail.com
2 fname-02 lname-02 fname-02.lname-02#gmail.com
3 fname-03 lname-03 fname-03.lname-03#gmail.com
>
>
I'm using the RODBC package inside an ORACLE DATA BASE (DB). Everything is doing very well, but I need to obtain a table from this DB and some variables as character type, not numbers.
So, I made my query like this:
e ManzResul_VIII<-sqlQuery(con,"select distinct t.fono_id_dis,
t.id_predio,
t.co_calle,
t.nu_casa,
t.x,
t.y,
t.plancheta from c_araya.proy_dist08_todo t where nvl(t.fono_tipo_dis, '-') not in ('CLIENTE', 'POTENCIAL', 'CARTERA') and nvl(t.x, 0) <> 0 ")
Is impossible to get the ID number as Character, this query change the type of my iDs variables from Character to Numeric type (The ID has a zero at the beginning, but has been changed it into a number). I have read the function description, but I can see how to manage it.
Any idea would be really appreciated, thanks in advance!
Change the query to cast the id to the data type you want:
select distinct cast(t.fono_id_dis as varchar(255)) as id
. . .
This should work with most databases.
This works for me:
library(RODBC)
con <- odbcDriverConnect('driver={SQL Server};server=[your server name];database=[your database name];trusted_connection=true')
ManzResul_VIII<-sqlQuery(con,"select distinct ('m' + id) as id from [your table]")
I'm making a connection to an oracle database using the ROracle package and DBI package. When I try to execute insert statements that have special characters, the special characters get converted to non-special characters. (I'm sure there's more correct terms for "special" and "non-special" that I'm not aware of).
First I make the following connection:
connection <- dbConnect(
dbDriver("Oracle"),
username = "xxxxx",
password = "xxxxx",
dbname = "xxxx"
)
Then I execute the following insert statement on a table I already have created. Column A has a type of nvarchar2.
dbSendQuery(connection, "insert into TEST_TABLE (A) values('£')")
This is what gets returned:
Statement: insert into TEST_TABLE (A) values('#')
Rows affected: 1
Row count: 0
Select statement: FALSE
Statement completed: TRUE
OCI prefetch: FALSE
Bulk read: 1000
Bulk write: 1000
As you can see, the "£" symbol gets replaced by a "#". I can execute the insert statement directly in PL/SQL and there's no issue, so it seems to be an issue with R. Any help is appreciated.
This was resolved by running Sys.setenv(NLS_LANG = "AMERICAN_AMERICA.AL32UTF8") before creating the connection.
Here's what I am trying to do:
import sqlite3
conn = sqlite3.connect('example.db')
c = conn.cursor()
c.execute(CREATE table Movies(index integer, mvnm text, year integer, genere text))
SyntaxError: invalid syntax
It highlights table in red color.
There are few problems:
The SQL should be provided in quotes because the execute() method accepts a
string argument.
index is a reserved keyword in SQL, so you cannot use
that as the name of your column.
If you want to run this script
repeatedly, you should add IF NOT EXISTS clause. Otherwise
consequent runs will fail due to "table already exists"
So, after these changes the statement looks like below:
c.execute("CREATE table IF NOT EXISTS Movies(myindex integer, mvnm text, year integer, genere text)")
You can then verify the table creation by logging in to sqllite:
$> ~ $ sqlite3 example.db
SQLite version 3.8.10.2 2015-05-20 18:17:19
Enter ".help" for usage hints.
sqlite> .tables
Movies