F# Query expression examples I notice look as though they grab the entire contents of a table before doing a computation. This seems less efficient than native SQL. Or should the performance be the same.
// A query expression.
let query1 =
query {
for customer in db.Customers do
select customer
}
F# query expressions are quite smart. I don't exactly understand how they work besides that they use quotations (similar to System.Linq.Expressions) to convert code to SQL. ORMs have measurable overhead, raw queries would be faster, but they're able to produce optimised queries
Given these models and ef-core context
[<CLIMutable>] type Address = { Id: int; City: string; Country: string }
[<CLIMutable>] type Person = { Id: int; Name: string; Age: int; Address: Address }
type PeopleContext() =
inherit DbContext()
[<FSharp.Core.DefaultValue>]
val mutable _people : DbSet<Person>
member db.People
with get () = db._people
and set value = db._people <- value
override _.OnConfiguring options =
options.UseSqlite("Data source=db.db3") |> ignore
I've created this query
query {
for person in ctx.People do
where (person.Age > 21)
select person.Address.City
take 1
} |> Seq.toArray |> printfn "%A"
That got translated to this sql
SELECT "a"."City"
FROM (
SELECT "p"."AddressId"
FROM "People" AS "p"
WHERE "p"."Age" > 21
LIMIT #__p_0
) AS "t"
Related
If I have a simple struct, one attribute of which contains a simple enum, how can I best store examples of this struct with their enumerations in the rusqlite database? Something like:
use rusqlite::{params, Connection, Result};
enum Sex{
Unknown,
Male,
Female,
}
struct Person{
name: String,
sex: Sex
}
fn main() -> Result<()> {
let conn = Connection::open_in_memory()?;
conn.execute(
"CREATE TABLE people(
name TEXT NOT NULL,
sex TEXT NOT NULL
)",
(), // empty list of parameters.
)?;
let person_01 = Person{
name: String::from("Adam"),
sex: Sex::Male
};
conn.execute(
"INSERT INTO people (name, sex) VALUES (?1, ?2)",
(&person_01.name, &person_01.sex),
)?;
Ok(())
}
The problem is that sqlite only allows data of restricted types (NULL, INTEGER, REAL, TEXT), trying to use TEXT here for the enum gives the following error:
error[E0277]: the trait bound `Sex: ToSql` is not satisfied
--> src/main.rs:33:9
|
31 | conn.execute(
| ------- required by a bound introduced by this call
32 | "INSERT INTO tasklist (name, sex) VALUES (?1, ?2)",
33 | (&person_01.name, &person_01.sex),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `ToSql` is not implemented for `Sex`
This error makes sense, but what is the best way to implement this? Cast the enum to int? I read here that this is "removes the guarantee that values always represent the variants of the enum", which I agree with. It would be nicer to match the string.
I have tried to do this using strum, which allows me to add to_str & from_str to the enum, allowing it to be added to the database like so:
#[derive(strum_macros::Display, strum_macros::EnumString, Debug)]
enum Sex{
Unknown,
Male,
Female,
}
...
conn.execute(
"INSERT INTO people (name, sex) VALUES (?1, ?2)",
(&person_01.name, &person_01.sex.to_string())
)?;
and retrieved like so:
let mut stmt = conn.prepare("SELECT name, sex FROM people")?;
let person_itr = stmt.query_map([], |row|{
Ok(
Person{
name: row.get(0)?,
sex: Sex::from_str(String::as_str(&row.get(1)?)).unwrap(),
}
)
});
but this feels messy. Is there a better way?
I have seen here people manually implementing FromSqlRow for the enum, but is there a better (quicker) way?
The right way to handle this is to implement ToSql and FromSql directly on your enum. This will make using it substantially more ergonomic, and possibly more efficient since you don't first have to convert to a type with an allocation, like String.
It also means the conversion to/from a string doesn't "infect" every interaction you have with the database; the conversion becomes automatic. So, while there is a bit more boilerplate, it will pay off every time you use this type in conjunction with the database.
impl ToSql for Sex {
fn to_sql(&self) -> rusqlite::Result<ToSqlOutput<'_>> {
Ok(self.to_string().into())
}
}
impl FromSql for Sex {
fn column_result(value: ValueRef<'_>) -> FromSqlResult<Self> {
value.as_str()?.parse()
.map_err(|e| FromSqlError::Other(Box::new(e)))
}
}
Now you can just do this when converting to Person:
sex: row.get(1)?,
Note that FromSqlRow is a trait specific to a postgres client; rusqlite has no such trait. If you wanted to, you could create a factory method on Person that constructs from a Row. That's up to you.
looking for help regarding queries. if i'm looking for the latest LinearState of SomeObjectState by property someProp i can do the following:
private fun lookupBySomeProp(someProp: String) : List<StateAndRef<SomeObjectState>> {
val generalCriteria = VaultQueryCriteria(Vault.StateStatus.UNCONSUMED)
val somePropIdx = builder { SomePersistentObject::someProp.equal(someProp) }
val somePropQueryCriteria = VaultCustomQueryCriteria(somePropIdx)
val queryCriteria = generalCriteria.and(somePropQueryCriteria)
val results = serviceHub.vaultService.queryBy<SomeObjectState>(queryCriteria)
// ... handle results here
}
how would i go about doing a single batch lookup for multiple SomeObjectState objects given a list of someProp identifiers?
private fun lookupBySomeProps(somePropList: List<String>) : List<StateAndRef<SomeObjectState>> {
val generalCriteria = VaultQueryCriteria(Vault.StateStatus.UNCONSUMED)
// how do i build efficiently build my bulk query?
}
i'm expecting to have to do something along the lines of the following in order return multiple states matching the list but have been have trouble properly implementing it
val somePropeIdx = builder { SomePersistentObject::someProp.in(somePropList) }
From this link: https://docs.corda.net/api-vault-query.html you can use .in() in your criteria among many other things:
Binary logical (AND, OR)
Comparison (LESS_THAN, LESS_THAN_OR_EQUAL, GREATER_THAN, GREATER_THAN_OR_EQUAL)
Equality (EQUAL, NOT_EQUAL)
Likeness (LIKE, NOT_LIKE)
Nullability (IS_NULL, NOT_NULL)
Collection based (IN, NOT_IN)
Standard SQL-92 aggregate functions (SUM, AVG, MIN, MAX, COUNT)
The reason why your query in the question didn't work is because how Corda has defined the .in() function.
You had:
val somePropeIdx = builder { SomePersistentObject::someProp.in(somePropList) }
Substituting notIn works fine:
val somePropeIdx = builder { SomePersistentObject::someProp.notIn(somePropList) }
However, in order to use .in() you have to include backticks.
So your query would be:
val somePropeIdx = builder { SomePersistentObject::someProp.`in`(somePropList) }
You can see this difference in definition in the source here.
rustc 1.38.0 (625451e37 2019-09-23)
rusqlite 0.20.0
I'm writing a program where I need to get back the id from the last insertion that sqlite just created.
db.execute("insert into short_names (short_name) values (?1)",params![short]).expect("db insert fail");
let id = db.execute("SELECT id FROM short_names WHERE short_name = '?1';",params![&short]).query(NO_PARAMS).expect("get record id fail");
let receiver = db.prepare("SELECT id FROM short_names WHERE short_name = "+short+";").expect("");
let id = receiver.query(NO_PARAMS).expect("");
println!("{:?}",id);
What I should be getting back is the id value sqlite automatically assigned with AUTOINCREMENT.
I'm getting this compiler Error:
error[E0599]: no method named `query` found for type `std::result::Result<usize, rusqlite::Error>` in the current scope
--> src/main.rs:91:100
|
91 | let id = db.execute("SELECT id FROM short_names WHERE short_name = '?1';",params![&short]).query(NO_PARAMS).expect("get record id fail");
| ^^^^^
error[E0369]: binary operation `+` cannot be applied to type `&str`
--> src/main.rs:94:83
|
94 | let receiver = db.prepare("SELECT id FROM short_names WHERE short_name = "+short+";").expect("");
| ------------------------------------------------^----- std::string::String
| | |
| | `+` cannot be used to concatenate a `&str` with a `String`
| &str
help: `to_owned()` can be used to create an owned `String` from a string reference. String concatenation appends the string on the right to the string on the left and may require reallocation. This requires ownership of the string on the left
|
94 | let receiver = db.prepare("SELECT id FROM short_names WHERE short_name = ".to_owned()+&short+";").expect("");
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^
error[E0277]: `rusqlite::Rows<'_>` doesn't implement `std::fmt::Debug`
--> src/main.rs:96:25
|
96 | println!("{:?}",id);
| ^^ `rusqlite::Rows<'_>` cannot be formatted using `{:?}` because it doesn't implement `std::fmt::Debug`
|
= help: the trait `std::fmt::Debug` is not implemented for `rusqlite::Rows<'_>`
= note: required by `std::fmt::Debug::fmt`
Line 94: I understand that rust's String is not the right type for the execute call, but I'm not sure what to do instead.
I suspect what needs to happen is the short_names table needs to be pulled form the database and then from the rust representation of the table for get the id that matches the short I'm trying to work with. I've been going off this example as a jumping off point, but It's dereferenced it's usefulness. The program I'm writing calls another program and then babysits it while this other program runs. To reduce over head I'm trying to not use OOP for this current program.
How should I structure my request to the database to get by the id I need?
Okay. First off, we are going to use a struct, because, unlike in Java, it is literally equivalent to not using one in this case, except that you gain in being able to keep things tidy.
You're trying to emulate Connection::last_insert_rowid(), which isn't a terribly smart thing to do, particularly if you are not in a transaction. We're also going to clear this up for you in a nice and neat fashion:
use rusqlite::{Connection};
pub struct ShortName {
pub id: i64,
pub name: String
}
pub fn insert_shortname(db: &Connection, name: &str) -> Result<ShortName, rusqlite::Error> {
let mut rtn = ShortName {
id: 0,
name: name.to_string()
};
db.execute("insert into short_names (short_name) values (?)",&[name])?;
rtn.id = db.last_insert_rowid();
Ok(rtn)
}
You can convince yourself that it works with this test:
#[test]
fn it_works() {
let conn = Connection::open_in_memory().expect("Could not test: DB not created");
let input:Vec<bool> = vec![];
conn.execute("CREATE TABLE short_names (id INTEGER PRIMARY KEY AUTOINCREMENT, short_name TEXT NOT NULL)", input).expect("Creation failure");
let output = insert_shortname(&conn, "Fred").expect("Insert failure");
assert_eq!(output.id, 1);
}
In rusqlite execute does not return a value. To return a value from a sqlite operation you need to use prepare and a variant of query. While much of Rust allows you to leave type up to the compiler, for rusqite you need to give the receiving variable a type.
There is not currently a way in rusqlite to take a single row out of a query. The type of rows is not a type iterator, so you need to progress over it with a while loop, that will progress based on the error type of rows. After the loop runs once it will return that there are no other row in rows and exit; if there is only one row from the query.
You can use query_named to modify the sql query your sanding. Using the named_params!{} macro will allow you to use a String to send information to the command.
use rusqlite::*;
fn main() {
let short = "lookup".to_string(); // example of a string you might use
let id:i64 = 0;
{ // open for db work
let db = Connection::open("YourDB.db").expect("db conn fail");
let mut receiver = db
.prepare("SELECT * FROM short_names WHERE short_name = :short;")
.expect("receiver failed");
let mut rows = receiver
.query_named(named_params!{ ":short": short })
.expect("rows failed");
while let Some(row) = rows.next().expect("while row failed") {
id=row.get(0).expect("get row failed");
}
} // close db work
println!("{}", id);
}
In the above example, we open a scope with {} around the database transaction, this will automatically close the db when it goes out of scope. Notice that we create our db connection and do all our work with the database solely inside the {}. This allows us to skip closing the db with the explicate command and is done by inference taken by the compiler from the scope: {}. The variables short and id, created in the scope of main(), are still available to the db scope and the rest of the scope of main(). While id is not assigned until the db scope, but it's defined outside of the scope, the scope of main, so that is where id's lifetime begins. id does not need to be mutable because it's only assigned once, if there is in fact only one row to retrieve, the while loop will only assign it once. Otherwise, if the database does not behave as expected this will result in an error.
Suppose we had a table of vendors in a SQL database that we want to load into F#:
+----+--------+--------+
| ID | Name | Parent |
+----+--------+--------+
| 1 | Nest | 2 |
| 2 | Google | NULL |
| 3 | Apple | NULL |
+----+--------+--------+
Using type providers it is easy enough to get the table into F#, but suppose we wanted to then convert the data into a sequence of Vendors, where Vendor is a type like this:
Vendor = {ID: int; Name: String; Parent: Vendor option}
How would one go about doing that? The issue is that when creating the sequence of Vendors we can't map to each row a particular Vendor, since we don't have the sequence of Vendors yet. It would be good to also assume that the application allows for cycles (A could have B as a parent and B could have A as a parent), although in the case of vendors that doesn't really make sense.
You could instead define the Vendor type as:
Vendor = {ID: int; Name: String; ParentID: int option}
But this seems much less elegant, since every time you would want to reference the parent vendor you'd have to do some sort of lookup. Is there a known solution to this? It seems like a situation that could occur often (especially when dealing with graphs or trees).
It also seems like a solution could involve some sort of lazy evaluation, but it's not clear to me how the Lazy<'T> type in F# could be applied here.
It's not a particularly elegant solution, but the one that uses lazy evaluation for the parent would look something like this: you would have two types, one that matches the schema of your table and one recursive:
type Flat = { ID: int; Name: string; ParentID : int option}
type Recursive = { ID: int; Name: string; Parent: Lazy<Recursive> option}
Then let's set up something that looks like your table:
let records =
[
{ ID = 1; Name = "Nest"; ParentID = Some 2 }
{ ID = 2; Name = "Google"; ParentID = None }
{ ID = 3; Name = "Apple"; ParentID = None }
{ ID = 4; Name = "Yin"; ParentID = Some 5 }
{ ID = 5; Name = "Yang"; ParentID = Some 4 }
]
|> List.map (fun x -> x.ID, x)
|> Map.ofList
let getRecord recID = records |> Map.find recID
And you can put it together like this:
let rec getRecordRecursive recID =
let record = getRecord recID
{
ID = record.ID
Name = record.Name
Parent =
record.ParentID
|> Option.map (fun pid ->
Lazy.Create <| fun () ->
getRecordRecursive pid)
}
So in a sense you're using the lazy type to delay the next step of recursion until you need it. Otherwise getRecordRecursive 4 would give you a stack overflow.
But there are tradeoffs - you no longer get nice behaved equality on such records, for instance. I'm not convinced you're not better off with Flat records in the long run.
I need to serialize arbitrary records into maps/dictionary.
I imagine my end type look like this:
type TabularData= array<Map<string, obj>>
But I have problems in build a generic function that accept any record and turn them into Map.
In practice, the best advice is probably to use some existing serialization library like FsPickler. However, if you really want to write your own serialization for records, then GetRecordFields (as mentioned in the comments) is the way to go.
The following takes a record and creates a map from string field names to obj values of the fields. Note that it does not handle nested records and it is not particularly fast:
open Microsoft.FSharp.Reflection
let asMap (recd:'T) =
[ for p in FSharpType.GetRecordFields(typeof<'T>) ->
p.Name, p.GetValue(recd) ]
|> Map.ofSeq
Here is a little example that calls this with a simple record:
type Person =
{ Name : string
Age : int }
asMap { Name = "Tomas"; Age = -1 }
Using the same idea mentioned by Tomas, you can create a IDictionary<K,V> from a record like this:
let asDictionary (entity: 'T) =
seq {
for prop in FSharpType.GetRecordFields(typeof<'T>) ->
prop.Name, prop.GetValue(entity)
} |> dict