I am trying to implement part of a UI framework. In the bottom example, the impl of widgets_mut() for Vec<T> needs to return a vector of trait objects like Vec<&mut dyn AnyWidget>. Given T is constrained to impl AnyWidget I don't understand why I am getting the error:
a value of type `Vec<&mut dyn AnyWidget>` cannot be built from an iterator over elements of type `&mut T`
the trait `FromIterator<&mut T>` is not implemented for `Vec<&mut dyn AnyWidget>`
Why does the 2-tuple implementation work, but the Vec implementation which seems equivalent doesn't?
fn main() {}
pub trait AnyWidget {}
pub trait WidgetVec {
fn widgets_mut(&mut self) -> Vec<&mut dyn AnyWidget>;
}
// This implementation for a 2-tuple seems equivalent and works fine:
impl<W0: AnyWidget, W1: AnyWidget> WidgetVec for (W0, W1) {
fn widgets_mut(&mut self) -> Vec<&mut dyn AnyWidget> {
let mut v: Vec<&mut dyn AnyWidget> = Vec::with_capacity(2);
v.push(&mut self.0);
v
}
}
impl<T: AnyWidget> WidgetVec for Vec<T> {
fn widgets_mut(&mut self) -> Vec<&mut dyn AnyWidget> {
self.iter_mut()
.map(|child| child)
.collect::<Vec<&mut dyn AnyWidget>>()
}
}
All you're missing is to explicitly cast the child from &mut T to &mut dyn AnyWidget.
impl<T: AnyWidget> WidgetVec for Vec<T> {
fn widgets_mut(&mut self) -> Vec<&mut dyn AnyWidget> {
self.iter_mut()
.map(|child| child as &mut dyn AnyWidget)
.collect::<Vec<&mut dyn AnyWidget>>()
}
}
In some cases, rust will perform these kinds of conversions automatically (at "coercion sites"), but this can't be done here, because there isn't a coercion site where you're trying to cast child. See the reference on coercion sites for all the places this can be done.
Related
Part 1: What should be the signature of a function returning an async function?
pub async fn some_async_func(arg: &str) {}
// What should be sig here?
pub fn higher_order_func(action: &str) -> ???
{
some_async_func
}
Part 2: What should be the sig, if based on the action parameter, higher_order_func had to return either async_func1 or async_func2.
I am also interested in learning the performance tradeoffs if there are multiple solutions. Please note that I'd like to return the function itself as an fn pointer or an Fn* trait, and not the result of invoking it.
Returning a function
Returning the actual function pointer requires heap allocation and a wrapper:
use std::future::Future;
use std::pin::Pin;
pub async fn some_async_func(arg: &str) {}
pub fn some_async_func_wrapper<'a>(arg: &'a str)
-> Pin<Box<dyn Future<Output=()> + 'a>>
{
Box::pin(some_async_func(arg))
}
pub fn higher_order_func<'a>(action: &str)
-> fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
some_async_func_wrapper
}
Why boxing? higher_order_func needs to have a concrete return type, which is a function pointer. The pointed function needs to also have a concrete return type, which is impossible for async function since it returns opaque type. In theory, it could be possible to write return type as fn(&'a str) -> impl Future<Output=()> + 'a, but this would require much more guesswork from the compiler and currently is not supported.
If you are OK with Fn instead of fn, you can get rid of the wrapper:
pub async fn some_async_func(arg: &str) {}
pub fn higher_order_func<'a>(action: &str)
-> impl Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
|arg: &'a str| {
Box::pin(some_async_func(arg))
}
}
To return a different function based on action value, you will need to box the closure itself, which is one more heap allocation:
pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}
pub fn higher_order_func<'a>(action: &str)
-> Box<dyn Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>>
{
if action.starts_with("one") {
Box::new(|arg: &'a str| {
Box::pin(some_async_func_one(arg))
})
} else {
Box::new(|arg: &'a str| {
Box::pin(some_async_func_two(arg))
})
}
}
Alternative: returning a future
To simplify things, consider returning a future itself instead of a function pointer. This is virtually the same, but much nicer and does not require heap allocation:
pub async fn some_async_func(arg: &str) {}
pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
-> impl Future<Output=()> + 'a
{
some_async_func(arg)
}
It might look like, when higher_order_func_future is called, some_async_func is getting executed - but this is not the case. Because of the way async functions work, when you call some_async_func, no user code is getting executed. The function call returns a Future: the actual function body will be executed only when someone awaits the returned future.
You can use the new function almost the same way as the previous function:
// With higher order function returning function pointer
async fn my_function() {
let action = "one";
let arg = "hello";
higher_order_func(action)(arg).await;
}
// With higher order function returning future
async fn my_function() {
let action = "one";
let arg = "hello";
higher_order_func_future(action, arg).await;
}
Notice, once more, that in both cases the actual some_async_func body is executed only when the future is awaited.
If you wanted to be able to call different async functions based on action value, you need boxing again:
pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}
pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
-> Pin<Box<dyn Future<Output=()> + 'a>>
{
if action.starts_with("one") {
Box::pin(some_async_func_one(arg))
} else {
Box::pin(some_async_func_two(arg))
}
}
Still, this is just one heap allocation, so I strongly advise returning a future. The only scenario that I can imagine where the previous solution is better is when you want to save the boxed closure somewhere and use it many times. In this case, excessive allocation happens only once, and you spare some CPU time by dispatching the call based on action only once - when you make the closure.
Ideally, what you'd want is a nested impl trait: -> impl Fn(&str) -> impl Future<Output = ()>. But nested impl trait is not supported. However, you can emulate that using a trait.
The idea is to define a trait that will abstract over the notion of "function returning a future". If our function would take a u32, for example, it could look like:
trait AsyncFn: Fn(u32) -> Self::Future {
type Future: Future<Output = ()>;
}
impl<F, Fut> AsyncFn for F
where
F: Fn(u32) -> Fut,
Fut: Future<Output = ()>,
{
type Future = Fut;
}
And then we would take impl AsyncFn. Trying to apply that naively to &str doesn't work:
error[E0308]: mismatched types
--> src/lib.rs:16:27
|
16 | fn higher_order_func() -> impl AsyncFn {
| ^^^^^^^^^^^^ one type is more general than the other
|
= note: expected associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`
found associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`
The error may look very strange, but it arises from the fact that async fn returns a future bound by the lifetime of all of its argument, i.e. for a signature async fn foo<'a>(arg: &'a str), the future is not impl Future<Output = ()> but impl Future<Output = ()> + 'a. There is a way to capture this relationship in our trait, we just need to make it generic over the argument and use HRTB:
trait AsyncFn<Arg>: Fn(Arg) -> Self::Future {
type Future: Future<Output = ()>;
}
impl<Arg, F, Fut> AsyncFn<Arg> for F
where
F: Fn(Arg) -> Fut,
Fut: Future<Output = ()>,
{
type Future = Fut;
}
And then we specify the type as:
fn higher_order_func() -> impl for<'a> AsyncFn<&'a str> {
some_async_func
}
In addition to the great accepted answer, depending on your use case it's also possible to "fake" the higher order function and avoid any heap allocations by using a simple macro to expand the wrapper code in-place instead:
pub async fn some_async_func(arg: &str) {}
macro_rules! higher_order_func {
($action: expr) => {
some_async_func
};
}
fn main() {
let future = higher_order_func!("action")("arg");
}
When trying to write a function that receives a async closure, I found it is a little bit tricky.
I know correct version is
pub async fn f1<F, Fut>(f: F) -> u32
where
F: Fn(u32) -> Fut,
Fut: Future<Output = u32>,
{
f(9u32).await
}
However, the where clause in the Rust doc saying that it is just a convenient way to express those type, so I tried to remove the where clause like this:
pub async fn f2(f: Fn(u32) -> Future<Output = u32>) -> u32 {
f(9u32).await
}
To my surprise, Rust refused to compile it, saying that
error: trait objects must include the dyn keyword
Anyone know what is the correct way to remove the where clause?
Any magic is played here by the where clause?
In your first snippet you have to constraints:
F must implement Fn(u32) -> Fut.
Fut must implement Future<Output = u32>.
But in your second snippet there are no constraints. The syntax for an inline constraint is with the impl keyword:
pub async fn f2(f: impl Fn(u32) -> Future<Output = u32>) -> u32 {
f(9u32).await
}
This doesn't work either because the impl applies to the Fn trait only, not to the return trait, so the compiler complains that a naked trait is not allowed, that you should add dyn to it.
Bad advice, because:
pub async fn f2(f: impl Fn(u32) -> dyn Future<Output = u32>) -> u32 {
f(9u32).await
}
will not work either, because dyn Trait types are unsized and cannot be returned.
You could try another impl there, that usually works in return types but:
pub async fn f2(f: impl Fn(u32) -> impl Future<Output = u32>) -> u32 {
f(9u32).await
}
impl Trait` not allowed outside of function and method return types
The only syntax I know of to constraint the return type of an Fn trait is with the where syntax:
pub async fn f2<Fut>(f: impl Fn(u32) -> Fut) -> u32
where
Fut: Future<Output = u32>,
{
f(9u32).await
}
But now that you are in full where mode, I see little reason to keep the inline constraint. Personally I consider bad style having both syntaxes on the same function.
Assuming you're referencing Rust by Example:
The example there doesn't use trait objects (e.g. foo: Box<dyn SomeTrait>) or existential types (e.g. foo: impl SomeTrait) but adds bounds to generic types at the declaration site:
impl <A: TraitB + TraitC, D: TraitE + TraitF> MyTrait<A, D> for YourType {}
// Expressing bounds with a `where` clause
impl <A, D> MyTrait<A, D> for YourType where
A: TraitB + TraitC,
D: TraitE + TraitF {}
In your case, you'd be writing:
pub async fn f2<F: Fn(u32) -> Fut, Fut: std::future::Future<Output = u32>>(f: F) -> u32 {
f(9u32).await
}
instead of putting the Fn(u32) -> Future<Output = u32> bound as the type of f. Note that you also need to declare a generic type Fut: Future<Output = u32> for the returned future from the closure, otherwise you'd be returning a raw trait object again.
As mentioned by #rodrigo, it's also possible to use the impl SomeTrait notation to entirely remove the generic type parameter from the function.
I need to append Bar to Foo as vectors. This solution works for me:
trait AppendBar {
fn append_bar(self) -> Self;
}
impl AppendBar for Vec<String> {
fn append_bar(self) -> Vec<String> {
let mut v = self;
v.push("Bar".to_string());
v
}
}
fn main(){
let foo = vec![String::from("Foo")].append_bar();
println!("{:#?}", foo)
}
Can I do something like this:
impl AppendBar for Vec<String> {
fn append_bar(self) -> Vec<String> {
let mut v = self.push("Bar".to_string());
v
}
}
Or this:
impl AppendBar for Vec<String> {
fn append_bar(self) -> Vec<String> {
self.push("Bar".to_string())
}
}
Since I could not get it to compile with either of the last 2 attempts, I assume this is just how the language is, but I want to make sure I am not missing something to make it more simple.
There is no need to rebind self to v to achieve mutability. Just declare it as mut self:
fn append_bar(mut self) -> Vec<String> {
self.push("Bar".to_string());
self
}
Beyond that, there are no standard Vec methods for chaining modifications (consuming self and returning modified version), pretty much all methods modify the Vec in-place as &mut self.
I am trying to store async functions in a vector, but it seems like impl cannot be used in the vector type definition:
use std::future::Future;
fn main() {
let mut v: Vec<fn() -> impl Future<Output = ()>> = vec![];
v.push(haha);
}
async fn haha() {
println!("haha");
}
error[E0562]: `impl Trait` not allowed outside of function and inherent method return types
--> src/main.rs:4:28
|
4 | let mut v: Vec<fn() -> impl Future<Output = ()>> = vec![];
| ^^^^^^^^^^^^^^^^^^^^^^^^
How do I write the type inside the vector?
I found that there may be a workaround by using a type alias, so I changed the code:
use std::future::Future;
type Haha = impl Future<Output = ()>;
fn main() {
let mut v: Vec<fn() -> Haha> = vec![];
v.push(haha);
}
async fn haha() {
println!("haha");
}
This doesn't work either; this time the error occurs in the type alias:
error[E0658]: `impl Trait` in type aliases is unstable
--> src/main.rs:3:1
|
3 | type Haha = impl Future<Output = ()>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: for more information, see https://github.com/rust-lang/rust/issues/63063
error[E0308]: mismatched types
--> src/main.rs:8:12
|
8 | v.push(haha);
| ^^^^ expected opaque type, found a different opaque type
|
= note: expected type `fn() -> Haha`
found type `fn() -> impl std::future::Future {haha}`
= note: distinct uses of `impl Trait` result in different opaque types
error: could not find defining uses
--> src/main.rs:3:1
|
3 | type Haha = impl Future<Output = ()>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
How do I fix it?
You cannot use the impl Trait this way. To be able to store different types that implement a trait into the same container you have to use dynamic dispatch, by storing something like Box<dyn Trait>.
In your particular case, you do not specify if you want to store the async functions themselves or the future generated by the async functions, the solution would be somewhat different.
To store just the futures, you write a container such as:
let mut v: Vec<Box<dyn Future<Output = ()>>> = vec![];
And then just call the function, box it and store it in the container:
v.push(Box::new(haha()));
If instead you want to store the async function itself, without calling it, you need a container with a double dyn:
let mut v2: Vec<Box<dyn Fn() -> Box<dyn Future<Output = ()>>>> = vec![];
Now, since your haha function does not implement this Fn trait you need an adaptor. A lambda function will do, but don't forget the double Box:
v2.push(Box::new(|| Box::new(haha())));
Unfortunately, with these solutions you will be able to create the vector, but not to .await for your futures. For that you need the futures to implement the Unpin marker. That guarantees to the compiler that the future will not move while it is running (if it did, the implementation would be totally unsafe). You could add the + Unpin requirement to the futures, but async fn are not Unpin so you could not fill the vector. The easiest way to fix it is to use this handy function from std:
pub fn into_pin(boxed: Box<T>) -> Pin<Box<T>>
for f in v2 {
f().into_pin().await;
}
Unfortunately, it is still unstable. Fortunately, there is a From impl that does exactly the same. So you can just write:
for f in v2 {
Pin::from(f()).await;
}
In your comment below you write this code to wait for the futures:
for f in v2 {
async { f().await }
}
Note that an async block itself will evaluate to another future, so here you are just wrapping each future into another future, but nobody is waiting for that one. Actually you'll get a warning about it:
warning: unused implementer of std::future::Future that must be used.
Remember that in order to properly wait for all the futures you will need an async runtime.
rodrigo's answer is correct, but I'd prefer to use Box::pin and bake the Pin type into the API of the collection. This makes using the Future trait object (or closure trait object producing a Future trait object) easier:
use std::{future::Future, pin::Pin};
type PinFutureObj<Output> = Pin<Box<dyn Future<Output = Output>>>;
async fn collection_of_pinned_future_trait_objects() {
let v: Vec<PinFutureObj<()>> = vec![
Box::pin(haha()),
Box::pin(hehe()),
Box::pin(haha()),
Box::pin(hehe()),
];
for f in v {
f.await
}
}
async fn collection_of_closure_trait_objects() {
let v: Vec<Box<dyn Fn() -> PinFutureObj<()>>> = vec![
Box::new(|| Box::pin(haha())),
Box::new(|| Box::pin(hehe())),
Box::new(|| Box::pin(haha())),
Box::new(|| Box::pin(hehe())),
];
for f in v {
f().await
}
}
async fn haha() {
println!("haha");
}
async fn hehe() {
println!("hehe");
}
I'd also start introducing type aliases for the longer types.
In fact, this type alias already exists in the futures crate as LocalBoxFuture and can be created via FutureExt::boxed_local. There's also BoxFuture produced by FutureExt::boxed which adds common trait bounds.
use futures::future::{FutureExt, LocalBoxFuture}; // 0.3.5
async fn collection_of_pinned_future_trait_objects() {
let v: Vec<LocalBoxFuture<'static, ()>> = vec![
haha().boxed_local(),
hehe().boxed_local(),
haha().boxed_local(),
hehe().boxed_local(),
];
for f in v {
f.await
}
}
async fn collection_of_closure_trait_objects() {
let v: Vec<Box<dyn Fn() -> LocalBoxFuture<'static, ()>>> = vec![
Box::new(|| haha().boxed_local()),
Box::new(|| hehe().boxed_local()),
Box::new(|| haha().boxed_local()),
Box::new(|| hehe().boxed_local()),
];
for f in v {
f().await
}
}
async fn haha() {
println!("haha");
}
async fn hehe() {
println!("hehe");
}
See also:
How can I put an async function into a map in Rust?
Why can impl trait not be used to return multiple / conditional types?
I'm writing a Rust wrapper around a C library and while doing so I'm trying to take advantage of the "nullable pointer optimization" mentioned in The Book, but I can't find a good way to convert Option<&T> to *const T and Option<&mut T> to *mut T like what they're describing.
What I really want is to be able to call Some(&foo) as *const _. Unfortunately that doesn't work, so the next best thing I can think of is a trait on Option<T> that enables me to call Some(&foo).as_ptr(). The following code is a working definition and implementation for that trait:
use std::ptr;
trait AsPtr<T> {
fn as_ptr(&self) -> *const T;
}
impl<'a, T> AsPtr<T> for Option<&'a T> {
fn as_ptr(&self) -> *const T {
match *self {
Some(val) => val as *const _,
None => ptr::null(),
}
}
}
Now that I can call Some(&foo).as_ptr() to get a *const _, I want to be able to call Some(&mut foo).as_ptr() to get a *mut _. The following is the new trait I made to do this:
trait AsMutPtr<T> {
fn as_mut_ptr(&self) -> *mut T;
}
impl<'a, T> AsMutPtr<T> for Option<&'a mut T> {
fn as_mut_ptr(&self) -> *mut T {
match *self {
Some(val) => val as *mut _,
None => ptr::null_mut(),
}
}
}
The problem is, the AsMutPtr trait won't compile. When I try, I get the following error:
error[E0507]: cannot move out of borrowed content
--> src/lib.rs:22:15
|
22 | match *self {
| ^^^^^
| |
| cannot move out of borrowed content
| help: consider removing the `*`: `self`
23 | Some(val) => val as *mut _,
| --- data moved here
|
note: move occurs because `val` has type `&mut T`, which does not implement the `Copy` trait
--> src/lib.rs:23:18
|
23 | Some(val) => val as *mut _,
| ^^^
I don't see what changed between the two traits that causes it to fail — I didn't think adding mut would make that big a difference. I tried adding a ref, but that just causes a different error, and I wouldn't expect to need that anyway.
Why doesn't the AsMutPtr trait work?
Unfortunately, writing the trait impl for &mut T instead of &T does make a big difference. &mut T, as opposed to &T, is not Copy, therefore you cannot extract it out of a shared reference directly:
& &T ---> &T
& &mut T -/-> &mut T
This is fairly natural - otherwise aliasing of mutable references would be possible, which violates Rust borrowing rules.
You may ask where that outer & comes from. It actually comes from &self in as_mut_ptr() method. If you have an immutable reference to something, even if that something contains mutable references inside it, you won't be able to use them to mutate the data behind them. This also would be a violation of borrowing semantics.
Unfortunately, I see no way to do this without unsafe. You need to have &mut T "by value" in order to cast it to *mut T, but you can't get it "by value" through a shared reference. Therefore, I suggest you to use ptr::read():
use std::ptr;
impl<'a, T> AsMutPtr<T> for Option<&'a mut T> {
fn as_mut_ptr(&self) -> *mut T {
match *self {
Some(ref val) => unsafe { ptr::read(val) as *mut _ },
None => ptr::null_mut(),
}
}
}
val here is & &mut T because of ref qualifier in the pattern, therefore ptr::read(val) returns &mut T, aliasing the mutable reference. I think it is okay if it gets converted to a raw pointer immediately and does not leak out, but even though the result would be a raw pointer, it still means that you have two aliased mutable pointers. You should be very careful with what you do with them.
Alternatively, you may modify AsMutPtr::as_mut_ptr() to consume its target by value:
trait AsMutPtr<T> {
fn as_mut_ptr(self) -> *mut T;
}
impl<'a, T> AsMutPtr<T> for Option<&'a mut T> {
fn as_mut_ptr(self) -> *mut T {
match self {
Some(value) => value as *mut T,
None => ptr::null_mut()
}
}
}
However, in this case Option<&mut T> will be consumed by as_mut_ptr(). This may not be feasible if, for example, this Option<&mut T> is stored in a structure. I'm not really sure whether it is possible to somehow perform reborrowing manually with Option<&mut T> as opposed to just &mut T (it won't be triggered automatically); if it is possible, then by-value as_mut_ptr() is probably the best overall solution.
The problem is that you are reading an &mut out of an &, but &muts are not Copy so must be moved - and you can't move out of a const reference. This actually explains Vladimir Matveev insight about &&mut → &s in terms of more fundamental properties.
This is actually relatively simply solved. If you can read a *const _, you can read a *mut _. The two are the same type, bar a flag that says "be careful, this is being shared". Since dereferences are unsafe either way, there's actually no reason to stop you casting between the two.
So you can actually do
match *self {
Some(ref val) => val as *const _ as *mut _,
None => ptr::null_mut(),
}
Read the immutable reference, make it an immutable pointer and then make it a mutable pointer. Plus it's all done through safe Rust so we know we're not breaking any aliasing rules.
That said, it's probably a really bad idea to actually use that *mut pointer until the &mut reference is gone. I would be very hesitant with this, and try to rethink your wrapper to something safer.
Will this do what you expect?
trait AsMutPtr<T> {
fn as_mut_ptr(self) -> *mut T;
}
impl<T> AsMutPtr<T> for Option<*mut T> {
fn as_mut_ptr(self) -> *mut T {
match self {
Some(val) => val as *mut _,
None => ptr::null_mut(),
}
}
}
To avoid unsafe code, change the trait to accept &mut self instead of either self or &self:
trait AsMutPtr<T> {
fn as_mut_ptr(&mut self) -> *mut T;
}
impl<'a, T> AsMutPtr<T> for Option<&'a mut T> {
fn as_mut_ptr(&mut self) -> *mut T {
match self {
Some(v) => *v,
None => ptr::null_mut(),
}
}
}
You could also reduce the implementation to one line, if you felt like it:
fn as_mut_ptr(&mut self) -> *mut T {
self.as_mut().map_or_else(ptr::null_mut, |v| *v)
}
This can be used to give you multiple mutable raw pointers from the same source. This can easily lead you to causing mutable aliasing, so be careful:
fn example(mut v: Option<&mut u8>) {
let b = v.as_mut_ptr();
let a = v.as_mut_ptr();
}
I would recommend not converting the immutable reference to a mutable pointer as this is very likely to cause undefined behavior.