How to store SQLite prepared statements for later? - sqlite

Right now I have code that uses the rusqlite sqlite bindings to open a db connection and do a bunch of db operations in my application like this:
extern crate rusqlite;
use rusqlite::SqliteConnection;
struct MyAppState {
db: SqliteConnection,
// ... pretend there's other fields here ...
}
impl MyAppState {
fn new() -> MyAppState {
let db = SqliteConnection::open(":memory:").unwrap();
MyAppState {
db: db
}
}
fn query_some_info(&mut self, arg: i64) -> i64 {
let mut stmt = self.db.prepare("SELECT ? + 1").unwrap();
let mut result_iter = stmt.query(&[&arg]).unwrap();
let result = result_iter.next().unwrap().unwrap().get(0);
result
}
}
fn main() {
let mut app = MyAppState::new();
for i in range(0, 100) {
let result = app.query_some_info(i);
println!("{}", result);
}
}
Since the prepared statement lives in a local variable, this seems to miss the point of prepared statements to some extent since I have to re-prepare it every time the function is called and the local variable comes into being. Ideally, I would prepare all my statements at most once and stash them in the MyAppState struct for the duration of the db connection.
However, since the SqliteStatement type is parameterized over the lifetime of the db connection, it borrows the connection and by extension the struct it lives in and I can't do anything with the struct anymore like return the struct by value or call &mut self methods on it (query_some_info doesn't really need to take &mut self here, but some code in my actual program does unless everything goes on to live in RefCells, which isn't the worst, I guess, but still).
Usually when the borrow checker betrays me like that, my recourse is to give up on stack discipline and put some Rc<RefCell< >> here and there until it all works out, but in this case there's some lifetimes in the types either way and I don't know how to word it in a way that appeases the borrow checker.
Ideally I'd like to write code that only prepares the statements right when the db gets opened, or maybe prepares them only once when they are first used, and then never calls prepare again during the duration of the db connection, while mostly keeping the safety of the rusqlite bindings rather than writing code against the sqlite3 C API or breaking abstraction or whatever. How do I?

You are right, indeed, that sibling references are awkward in Rust. There is a good reason though, they are not easily modeled by the ownership system.
In this particular case, I would advise you to split the structure: you can keep the prepared statements in a dedicated cache also parametrized on the lifetime of the db for example; the db instead should be instantiated at the top of your program and passed down (think dependency injection) so that the cache that depends on it can outlive the program main function.
This does mean that the db will remain borrowed, obviously.

The Statement struct has a lifetime parameter, Statement<'conn>. When you prepare the statement, you must have a reference to the Connection that outlives the statement.
extern crate rusqlite;
use rusqlite::{Connection, Statement};
struct MyAppState {
db: Connection,
}
impl MyAppState {
fn new() -> MyAppState {
let db = Connection::open(":memory:").unwrap();
MyAppState { db: db }
}
}
struct PreparedStatement<'conn> {
statement: Statement<'conn>,
}
impl<'conn> PreparedStatement<'conn> {
pub fn new<'a>(conn: &'a Connection, sql: &str) -> PreparedStatement<'a> {
PreparedStatement {
statement: conn.prepare(sql).unwrap(),
}
}
fn query_some_info(&mut self, arg: i64) -> i64 {
let mut result_iter = self.statement.query(&[&arg]).unwrap();
let result = result_iter.next().unwrap().unwrap().get(0);
result
}
}
fn main() {
let app = MyAppState::new();
let mut prepared_stmt = PreparedStatement::new(&app.db, "SELECT ? + 1");
for i in 0..100 {
let result = prepared_stmt.query_some_info(i);
println!("{}", result);
}
}
In Rust, unlike some other languages, I have found that factoring something out into a function changes its meaning. It introduces new lifetimes, which usually works against you. But in this case, that's exactly what was needed.

Related

why does Rc works in async fn

I found this example compile successfully, but isn't Rc<> thread unsafe? What if tikio start multi thread to execute test fn, won't it cause Rc to run on different threads?
use std::rc::Rc;
use tokio::join;
use std::time;
use async_std::task::{sleep};
async fn test(t:Rc<String>){
let k = t;
println!("1. test,{:?}", k);
sleep(time::Duration::from_secs(1)).await;
println!("2. test,{:?}", k);
}
#[tokio::main]
async fn main() {
let r = Rc::new("abc".to_string());
let f1 = test(r.clone());
let f2 = test(r.clone());
join!(f1,f2);
}
This is a bit nuanced, but #[tokio::main] will rewrite your async main() into something like this:
fn main() {
tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()
.unwrap()
.block_on(async {
// your code from `async main()`
})
}
The last step that actually runs everything, .block_on(), does not require the given task to be thread-safe (a.k.a. implement Send) because it will not be ran on a separate thread, even when using the multi-threaded executor:
This runs the given future on the current thread, blocking until it is complete, and yielding its resolved result.
Other tokio utilities and any spawned tasks may be ran on separate threads, and even move between them, but not the initial task.

How to use rust async_trait generic to a lifetime parameter?

I'm trying to make an async_trait where some implementations are generic to types with lifetime parameters:
use async_trait::async_trait;
struct MyLifetimeType<'a> {
s: &'a mut String,
}
#[async_trait]
trait MyTrait<T> {
async fn handle(t: T);
}
struct MyImpl;
#[async_trait]
impl<'a> MyTrait<MyLifetimeType<'a>> for MyImpl {
async fn handle(t: MyLifetimeType<'a>) {
t.s.push_str("hi");
}
}
When I try to compile this, I get
error[E0276]: impl has stricter requirements than trait
--> ...
|
18 | async fn handle(t: T);
| ---------------------- definition of `handle` from trait
...
25 | async fn handle(t: MyLifetimeType<'a>) {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ impl has extra requirement `'a: 'async_trait`
It seems that the issue is related to async_trait somehow using the lifetime parameter 'a under the hood. When I get rid of all the async and async_trait, the code compiles fine. How can I avoid this extra requirement error?
For more context, to explain why have handlers implementing MyTrait that can operate on structs containing mutable pointers: I have a function that obtains RwLockReadGuards and RwLockWriteGuards for a couple different locks, then passes the contents to a handler. For the write guards, I need some way for the handler to mutate contents, so I pass a mutable pointer.
This is a known issue. The author recommends adding an explicit lifetime bound when that error occurs:
use async_trait::async_trait;
struct MyLifetimeType<'a> {
s: &'a mut String,
}
#[async_trait]
trait MyTrait<T> {
async fn handle(&self, t: T) where T: 'async_trait;
}
struct MyImpl;
#[async_trait]
impl<'a> MyTrait<MyLifetimeType<'a>> for MyImpl {
async fn handle(&self, t: MyLifetimeType<'a>) {
t.s.push_str("hi");
}
}

Is it valid to wake a Rust future while it's being polled?

I'd like to be able to sleep my future for a single "frame" so that other work can happen. Is this a valid implementation of this idea?
use std::future::Future;
use std::task::{Context, Poll};
use std::pin::Pin;
struct Yield {
yielded: bool,
}
impl Future for Yield {
type Output = ();
fn poll(mut self: Pin<&mut Self>, ctx: &mut Context) -> Poll<()> {
if self.yielded {
Poll::Ready(())
} else {
self.yielded = true;
// This is the part I'm concerned about
ctx.waker().wake_by_ref();
Poll::Pending
}
}
}
Specifically, my concern is that the context won't "notice" the wake_by_ref call if it's made before the poll returns Pending. Does the interface contract of poll make any guarantees about this task being immediately re-polled when executed in this way?
TL;DR: Your code is valid.
Based on the contract for the waker, it has to poll your future one more time. Otherwise, it is possible to have a race condition between the Future::poll call and the counterpart of the future which actually does some work.
Let's take a look at an example:
impl Future for Foo {
type Output = ();
fn poll(self: Pin<&mut Self>, ctx: &mut Context) -> Poll<()> {
let result = communicate_with_worker(ctx); // returns false
// <-- Time point (1)
return match result {
true => Poll::Pending,
false => Poll::Ready(()),
};
}
}
At time point (1), the future has decided that it is not ready, but it's possible that the polling thread is paused here and the worker thread was scheduled and finished its work.
The worker thread will then call the waker and request the future be polled again. If the waker decided to not poll the future again since it's polling the future right now, then the waker will never receive a wake up request again.
This means that the waker may discard wake up requests which come before
poll was called, but it's not allowed to discard wake up requests which
came during the future's poll call.
The only question I have: why would you like to reschedule polling for one more frame?
Since your actual work has to be done in a separate thread (not inside fn poll) then it doesn't make any sense to reschedule polling.

Interfaces stored as value; Methods unable to update struct fields

I have a tool that I'm writing that exposes some functions that pull information out of a static database to several scripting languages that I'm embedding into the tool.
I thought; "Hey sounds like a nice use case for interfaces". So I defined an interface like so in my package scripting
type ScriptingLang interface {
RunScript(filename string) error
RunString(s string) error
Interpreter() error
Init() error
IsInit() bool
}
Then I store a map of them so I can look them up by a string defined like so in a different package.
var ScriptingLangs = make(map[string]scripting.ScriptingLang)
and a function to register them. Also some little helper functions like
func RunString(lang, s string) error {
if v, ok := ScriptingLangs[lang]; ok {
if !v.IsInit() {
v.Init()
}
return v.RunString(s)
} else {
return NoSuchLangErr
}
return nil
}
The problem that I ran into is it seams that interfaces can't have methods with pointer receivers. As a result my Lua struct that implements ScriptingLang isn't able to save it's *state because it's stored in ScriptingLangs.
I've tried updating the value stored in the map at the end of functions that save state and it didn't update the value.
To my understanding you shouldn't use pointers of interfaces so what are my options here? I would like to really keep the interfaces so I can do some neat stuff with git submodules.
A minimal example of my problem:
package main
import (
"fmt"
)
type ScriptingLang interface {
DoString(s string) error
Init() error
}
type Lua struct {
state string
}
func (l Lua) DoString(s string) error {
fmt.Printf("Doing '%v' with state '%v'\n", s, l.state)
return nil
}
func (l Lua) Init() error {
l.state = "Inited"
return nil
}
var lang ScriptingLang
func main() {
lang = Lua{}
lang.Init()
lang.DoString("Stuff")
}
If you want to mutate state, you need a pointer receiver, and your Init method doesn't have one. The fact that you're storing the value inside an interface makes no difference.
In your minimal(-ish) example, change the Init method (and any method that updates state) to have a pointer receiver, and point a pointer inside the interface and everything works:
func (l *Lua) Init() error {
l.state = "Inited"
return nil
}
...
func main() {
lang = &Lua{}
lang.Init()
lang.DoString("Stuff")
}
This article might help: http://jordanorelli.com/post/32665860244/how-to-use-interfaces-in-go

map[Task]int64 where Task is an interface

Let's say I define the following interface in a Go library:
type Task interface {
Do() error
}
func Register(task Task) { ... }
func GetId(task Task) int64 { ... }
In Register(), the library associates a unique int64 with each task instance. GetId() must return the identifier for the given task.
My initial idea was to store the association as a map[Task]int64. This seems to work fine, but I was told that it would break if an object implementing Task was not equality-comparable (for example, a struct containing a map). I still need to check if this is true.
I was going to try and use a slice of struct { task Task; id int64 } instead and just iterate over it, but that would still require equality comparable Task instances. And AFAIU there is no identity comparison in Go.
How can I have a robust mapping from Task instances to their ID?
EDIT: Both solutions proposed so far work, but they have the disadvantage that every Task implementation has to include some repetitive code to handle the IDs. I could provide that code in a TaskBase struct that could be embedded, but ideally I would prefer a solution that doesn't require implementations to even know about the IDs (they are internal to the library and have no meaning outside of it).
A more complete example: http://play.golang.org/p/1RzDiw7F9t
package main
import (
"fmt"
"math/rand"
)
type Task interface {
Do() error
ID() int64
}
type XTask struct {
id int64
// other stuff
}
func NewXTask( /*task parameters...*/) *XTask {
t := &XTask{ /*initialize members*/}
t.id = Register(t)
// possibly more initialization...
return t
}
func (t *XTask) Do() error { return nil } // stub
func (t *XTask) ID() int64 { return t.id }
var taskRegistry = map[int64]Task{}
func Register(t Task) int64 {
var id int64
for {
id = rand.Int63()
if _, exists := taskRegistry[id]; !exists {
break
}
}
taskRegistry[id] = t
return id
}
func main() {
t1 := NewXTask()
t2 := NewXTask()
fmt.Printf("%x\n", t1.ID())
fmt.Printf("%x\n", t2.ID())
}
I used an ID method as Daniel suggested, and I turned the map backwards from the way you had it. This is because the Task objects know their own ID, so a map from Task to ID is not needed. A map from ID to task however, is useful for guaranteeing uniqueness. It might come in handy some other time if you find yourself with only an ID and you need the corresponding Task object.
Also note this example is not goroutine-safe. If you need that you will have to add synchronization.

Resources