Folding over references inside a match results in a lifetime error - functional-programming

I want to build a string s by iterating over a vector of simple structs, appending different strings to acc depending on the struct.
#[derive(Clone, Debug)]
struct Point(Option<i32>, Option<i32>);
impl Point {
fn get_first(&self) -> Option<i32> {
self.0
}
}
fn main() {
let mut vec = vec![Point(None, None); 10];
vec[5] = Point(Some(1), Some(1));
let s: String = vec.iter().fold(
String::new(),
|acc, &ref e| acc + match e.get_first() {
None => "",
Some(ref content) => &content.to_string()
}
);
println!("{}", s);
}
Running this code results in the following error:
error: borrowed value does not live long enough
Some(ref content) => &content.to_string()
^~~~~~~~~~~~~~~~~~~
note: reference must be valid for the expression at 21:22...
|acc, &ref e| acc + match e.get_first() {
^
note: ...but borrowed value is only valid for the expression at 23:33
Some(ref content) => &content.to_string()
^~~~~~~~~~~~~~~~~~~~
The problem is that the lifetime of the &str I create seems to end immediately. However, if to_string() would have returned a &str in the first place, the compiler would not have complained. Then, what is the difference?
How can I make the compiler understand that I want the string references to live as long as I am constructing s?

There is a difference between the result of your branches:
"" is of type &'static str
content is of type i32, so you are converting it to a String and then from that to a &str... but this &str has the same lifetime as the String returned by to_string, which dies too early
A quick work-around, as mentioned by #Dogbert, is to move acc + inside the branches:
let s: String = vec.iter().fold(
String::new(),
|acc, &ref e| match e.get_first() {
None => acc,
Some(ref content) => acc + &content.to_string(),
}
);
However, it's a bit wasteful, because every time we have an integer, we are allocating a String (via to_string) just to immediately discard it.
A better solution is to use the write! macro instead, which just appends to the original string buffer. This means there are no wasted allocations.
use std::fmt::Write;
let s = vec.iter().fold(
String::new(),
|mut acc, &ref e| {
if let Some(ref content) = e.get_first() {
write!(&mut acc, "{}", content).expect("Should have been able to format!");
}
acc
}
);
It's maybe a bit more complicated, notably because formatting adds in error handling, but is more efficient as it only uses a single buffer.

There are multiple solutions to your problem. But first some explanations:
If to_string() would have returned a &str in the first place, the compiler would not have complained. Then, what is the difference?
Suppose there is a method to_str() that returns a &str. What would the signature look like?
fn to_str(&self) -> &str {}
To better understand the issue, lets add explicit lifetimes (that are not necessary thanks to lifetime elision):
fn to_str<'a>(&'a self) -> &'a str {}
It becomes clear that the returned &str lives as long as the receiver of the method (self). This would be OK since the receiver lives long enough for your acc + ... operation. In your case however, the .to_string() call creates a new object the only lives in the second match arm. After the arm's body is left, it will be destroyed. Therefore you can't pass a reference to it to the outer scope (in which acc + ... takes place).
So one possible solution looks like this:
let s = vec.iter().fold(
String::new(),
|acc, e| {
acc + &e.get_first()
.map(|f| f.to_string())
.unwrap_or(String::new())
}
);
It's not optimal, but luckily your default value is an empty string and the owned version of an empty string (String::new()) does not require any heap allocations, so there is no performance penalty.
However, we are still allocating once per integer. For a more efficient solution, see Matthieu M.'s answer.

Related

How to convert a vector of vectors into a vector of slices without creating a new object? [duplicate]

I have the following:
enum SomeType {
VariantA(String),
VariantB(String, i32),
}
fn transform(x: SomeType) -> SomeType {
// very complicated transformation, reusing parts of x in order to produce result:
match x {
SomeType::VariantA(s) => SomeType::VariantB(s, 0),
SomeType::VariantB(s, i) => SomeType::VariantB(s, 2 * i),
}
}
fn main() {
let mut data = vec![
SomeType::VariantA("hello".to_string()),
SomeType::VariantA("bye".to_string()),
SomeType::VariantB("asdf".to_string(), 34),
];
}
I would now like to call transform on each element of data and store the resulting value back in data. I could do something like data.into_iter().map(transform).collect(), but this will allocate a new Vec. Is there a way to do this in-place, reusing the allocated memory of data? There once was Vec::map_in_place in Rust but it has been removed some time ago.
As a work-around, I've added a Dummy variant to SomeType and then do the following:
for x in &mut data {
let original = ::std::mem::replace(x, SomeType::Dummy);
*x = transform(original);
}
This does not feel right, and I have to deal with SomeType::Dummy everywhere else in the code, although it should never be visible outside of this loop. Is there a better way of doing this?
Your first problem is not map, it's transform.
transform takes ownership of its argument, while Vec has ownership of its arguments. Either one has to give, and poking a hole in the Vec would be a bad idea: what if transform panics?
The best fix, thus, is to change the signature of transform to:
fn transform(x: &mut SomeType) { ... }
then you can just do:
for x in &mut data { transform(x) }
Other solutions will be clunky, as they will need to deal with the fact that transform might panic.
No, it is not possible in general because the size of each element might change as the mapping is performed (fn transform(u8) -> u32).
Even when the sizes are the same, it's non-trivial.
In this case, you don't need to create a Dummy variant because creating an empty String is cheap; only 3 pointer-sized values and no heap allocation:
impl SomeType {
fn transform(&mut self) {
use SomeType::*;
let old = std::mem::replace(self, VariantA(String::new()));
// Note this line for the detailed explanation
*self = match old {
VariantA(s) => VariantB(s, 0),
VariantB(s, i) => VariantB(s, 2 * i),
};
}
}
for x in &mut data {
x.transform();
}
An alternate implementation that just replaces the String:
impl SomeType {
fn transform(&mut self) {
use SomeType::*;
*self = match self {
VariantA(s) => {
let s = std::mem::replace(s, String::new());
VariantB(s, 0)
}
VariantB(s, i) => {
let s = std::mem::replace(s, String::new());
VariantB(s, 2 * *i)
}
};
}
}
In general, yes, you have to create some dummy value to do this generically and with safe code. Many times, you can wrap your whole element in Option and call Option::take to achieve the same effect .
See also:
Change enum variant while moving the field to the new variant
Why is it so complicated?
See this proposed and now-closed RFC for lots of related discussion. My understanding of that RFC (and the complexities behind it) is that there's an time period where your value would have an undefined value, which is not safe. If a panic were to happen at that exact second, then when your value is dropped, you might trigger undefined behavior, a bad thing.
If your code were to panic at the commented line, then the value of self is a concrete, known value. If it were some unknown value, dropping that string would try to drop that unknown value, and we are back in C. This is the purpose of the Dummy value - to always have a known-good value stored.
You even hinted at this (emphasis mine):
I have to deal with SomeType::Dummy everywhere else in the code, although it should never be visible outside of this loop
That "should" is the problem. During a panic, that dummy value is visible.
See also:
How can I swap in a new value for a field in a mutable reference to a structure?
Temporarily move out of borrowed content
How do I move out of a struct field that is an Option?
The now-removed implementation of Vec::map_in_place spans almost 175 lines of code, most of having to deal with unsafe code and reasoning why it is actually safe! Some crates have re-implemented this concept and attempted to make it safe; you can see an example in Sebastian Redl's answer.
You can write a map_in_place in terms of the take_mut or replace_with crates:
fn map_in_place<T, F>(v: &mut [T], f: F)
where
F: Fn(T) -> T,
{
for e in v {
take_mut::take(e, f);
}
}
However, if this panics in the supplied function, the program aborts completely; you cannot recover from the panic.
Alternatively, you could supply a placeholder element that sits in the empty spot while the inner function executes:
use std::mem;
fn map_in_place_with_placeholder<T, F>(v: &mut [T], f: F, mut placeholder: T)
where
F: Fn(T) -> T,
{
for e in v {
let mut tmp = mem::replace(e, placeholder);
tmp = f(tmp);
placeholder = mem::replace(e, tmp);
}
}
If this panics, the placeholder you supplied will sit in the panicked slot.
Finally, you could produce the placeholder on-demand; basically replace take_mut::take with take_mut::take_or_recover in the first version.

Is there a better functional way to process a vector with error checking?

I'm learning Rust and would like to know how I can improve the code below.
I have a vector of tuples of form (u32, String). The u32 values represent line numbers and the Strings are the text on the corresponding lines. As long as all the String values can be successfully parsed as integers, I want to return an Ok<Vec<i32>> containing the just parsed String values, but if not I want to return an error of some form (just an Err<String> in the example below).
I'm trying to learn to avoid mutability and use functional styles where appropriate, and the above is straightforward to do functionally if that was all that was needed. Here's what I came up with in this case:
fn data_vals(sv: &Vec<(u32, String)>) -> Result<Vec<i32>, String> {
sv.iter()
.map(|s| s.1.parse::<i32>()
.map_err(|_e| "*** Invalid data.".to_string()))
.collect()
}
However, the small catch is that I want to print an error message for every invalid value (and not just the first one), and the error messages should contain both the line number and the string values in the offending tuple.
I've managed to do it with the following code:
fn data_vals(sv: &Vec<(u32, String)>) -> Result<Vec<i32>, String> {
sv.iter()
.map(|s| (s.0, s.1.parse::<i32>()
.or_else(|e| {
eprintln!("ERROR: Invalid data value at line {}: '{}'",
s.0, s.1);
Err(e)
})))
.collect::<Vec<(u32, Result<i32, _>)>>() // Collect here to avoid short-circuit
.iter()
.map(|i| i.1
.clone()
.map_err(|_e| "*** Invalid data.".to_string()))
.collect()
}
This works, but seems rather messy and cumbersome - especially the typed collect() in the middle to avoid short-circuiting so all the errors are printed. The clone() call is also annoying, and I'm not really sure why it's needed - the compiler says I'm moving out of borrowed content otherwise, but I'm not really sure what's being moved. Is there a way it can be done more cleanly? Or should I go back to a more procedural style? When I tried, I ended up with mutable variables and a flag to indicate success and failure, which seems less elegant:
fn data_vals(sv: &Vec<(u32, String)>) -> Result<Vec<i32>, String> {
let mut datavals = Vec::new();
let mut success = true;
for s in sv {
match s.1.parse::<i32>() {
Ok(v) => datavals.push(v),
Err(_e) => {
eprintln!("ERROR: Invalid data value at line {}: '{}'",
s.0, s.1);
success = false;
},
}
}
if success {
return Ok(datavals);
} else {
return Err("*** Invalid data.".to_string());
}
}
Can someone advise me on the best way to do this? Should I stick to the procedural style here, and if so can that be improved? Or is there a cleaner functional way to do it? Or a blend of the two? Any advice appreciated.
I think that's what partition_map() from itertools is for:
use itertools::{Either, Itertools};
fn data_vals<'a>(sv: &[&'a str]) -> Result<Vec<i32>, Vec<(&'a str, std::num::ParseIntError)>> {
let (successes, failures): (Vec<_>, Vec<_>) =
sv.iter().partition_map(|s| match s.parse::<i32>() {
Ok(v) => Either::Left(v),
Err(e) => Either::Right((*s, e)),
});
if failures.len() != 0 {
Err(failures)
} else {
Ok(successes)
}
}
fn main() {
let numbers = vec!["42", "aaaezrgggtht", "..4rez41eza", "55"];
println!("{:#?}", data_vals(&numbers));
}
In a purely functional style, you have to avoid side-effects.
Printing errors is a side-effect. The preferred style would be to return an object of the style:
Result<Vec<i32>, Vec<String>>
and print the list after the data_vals function returns.
So, essentially, you want your processing to collect a list of integers, and a list of strings:
fn data_vals(sv: &Vec<(u32, String)>) -> Result<Vec<i32>, Vec<String>> {
let (ok, err): (Vec<_>, Vec<_>) = sv
.iter()
.map(|(i, s)| {
s.parse()
.map_err(|_e| format!("ERROR: Invalid data value at line {}: '{}'", i, s))
})
.partition(|e| e.is_ok());
if err.len() > 0 {
Err(err.iter().filter_map(|e| e.clone().err()).collect())
} else {
Ok(ok.iter().filter_map(|e| e.clone().ok()).collect())
}
}
fn main() {
let input = vec![(1, "0".to_string())];
let r = data_vals(&input);
assert_eq!(r, Ok(vec![0]));
let input = vec![(1, "zzz".to_string())];
let r = data_vals(&input);
assert_eq!(r, Err(vec!["ERROR: Invalid data value at line 1: 'zzz'".to_string()]));
}
Playground Link
This uses partition which does not depend on an external crate.
Side effects (eprintln!) in an iterator adapter are definitely not "functional". You should accumulate and return the errors and let the caller deal with them.
I would use fold here. The goal of fold is to reduce a list to a single value, starting from an initial value and augmenting the result with every item. This "single value" can very well be a list, though. Here, though, there are two possible lists we might want to return: a list of i32 if all values are valid, or a list of errors if there are any errors (I've chosen to return Strings for errors here, for simplicity.)
fn data_vals(sv: &[(u32, String)]) -> Result<Vec<i32>, Vec<String>> {
sv.iter().fold(
Ok(Vec::with_capacity(sv.len())),
|acc, (line_number, data)| {
let data = data
.parse::<i32>()
.map_err(|_| format!("Invalid data value at line {}: '{}'", line_number, data));
match (acc, data) {
(Ok(mut acc_data), Ok(this_data)) => {
// No errors yet; push the parsed value to the values vector.
acc_data.push(this_data);
Ok(acc_data)
}
(Ok(..), Err(this_error)) => {
// First error: replace the accumulator with an `Err` containing the first error.
Err(vec![this_error])
}
(Err(acc_errors), Ok(..)) => {
// There have been errors, but this item is valid; ignore it.
Err(acc_errors)
}
(Err(mut acc_errors), Err(this_error)) => {
// One more error: push it to the error vector.
acc_errors.push(this_error);
Err(acc_errors)
}
}
},
)
}
fn main() {
println!("{:?}", data_vals(&[]));
println!("{:?}", data_vals(&[(1, "123".into())]));
println!("{:?}", data_vals(&[(1, "123a".into())]));
println!("{:?}", data_vals(&[(1, "123".into()), (2, "123a".into())]));
println!("{:?}", data_vals(&[(1, "123a".into()), (2, "123".into())]));
println!("{:?}", data_vals(&[(1, "123a".into()), (2, "123b".into())]));
}
The initial value is Ok(Vec::with_capacity(sv.len())) (this is an optimization to avoid reallocating the vector as we push items to it; a simpler version would be Ok(vec![])). If the slice is empty, this will be fold's result; the closure will never be called.
For each item, the closure checks 1) whether there were any errors so far (indicated by the accumulator value being an Err) or not and 2) whether the current item is valid or not. I'm matching on two Result values simultaneously (by combining them in a tuple) to handle all 4 cases. The closure then returns an Ok if there are no errors so far (with all the parsed values so far) or an Err if there are any errors so far (with every invalid value found so far).
You'll notice I used the push method to add an item to a Vec. This is, strictly speaking, mutation, which is not considered "functional", but because we are moving the Vecs here, we know there are no other references to them, so we know we aren't affecting any other use of these Vecs.

Why does a &[T] argument also accept &Vec<T>?

I am working through the Rust book, namely the minigrep project. There I came across the following snippet:
fn main() {
let args: Vec<String> = env::args().collect();
let (query, filename) = parse_config(&args);
// --snip--
}
fn parse_config(args: &[String]) -> (&str, &str) {
let query = &args[1];
let filename = &args[2];
(query, filename)
}
The confusing piece for me is args: &[String]. If I replace it with args: &Vec<String>, it also works. My guess is that &[String] is a more general type annotation that matches not only &Vec<String>, but also some other types. Is that correct? If so, what other types are matched by [T]?
Generally speaking, [T] is a contiguous sequence and &[T] is a slice.
The reason why the compiler allows &[String] instead of &Vec<String> is that Vec<T> dereferences to [T]. This is called Deref coercion. It can be said that the former notation (in function parameters) is more general; it is also the preferred one. Further details about automatic dereferencing rules can be found in this question.

Print Vec using a placeholder [duplicate]

I tried the following code:
fn main() {
let v2 = vec![1; 10];
println!("{}", v2);
}
But the compiler complains:
error[E0277]: `std::vec::Vec<{integer}>` doesn't implement `std::fmt::Display`
--> src/main.rs:3:20
|
3 | println!("{}", v2);
| ^^ `std::vec::Vec<{integer}>` cannot be formatted with the default formatter
|
= help: the trait `std::fmt::Display` is not implemented for `std::vec::Vec<{integer}>`
= note: in format strings you may be able to use `{:?}` (or {:#?} for pretty-print) instead
= note: required by `std::fmt::Display::fmt`
Does anyone implement this trait for Vec<T>?
let v2 = vec![1; 10];
println!("{:?}", v2);
{} is for strings and other values which can be displayed directly to the user. There's no single way to show a vector to a user.
The {:?} formatter can be used to debug it, and it will look like:
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
Display is the trait that provides the method behind {}, and Debug is for {:?}
Does anyone implement this trait for Vec<T> ?
No.
And surprisingly, this is a demonstrably correct answer; which is rare since proving the absence of things is usually hard or impossible. So how can we be so certain?
Rust has very strict coherence rules, the impl Trait for Struct can only be done:
either in the same crate as Trait
or in the same crate as Struct
and nowhere else; let's try it:
impl<T> std::fmt::Display for Vec<T> {
fn fmt(&self, _: &mut std::fmt::Formatter) -> Result<(), std::fmt::Error> {
Ok(())
}
}
yields:
error[E0210]: type parameter `T` must be used as the type parameter for some local type (e.g., `MyStruct<T>`)
--> src/main.rs:1:1
|
1 | impl<T> std::fmt::Display for Vec<T> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ type parameter `T` must be used as the type parameter for some local type
|
= note: only traits defined in the current crate can be implemented for a type parameter
Furthermore, to use a trait, it needs to be in scope (and therefore, you need to be linked to its crate), which means that:
you are linked both with the crate of Display and the crate of Vec
neither implement Display for Vec
and therefore leads us to conclude that no one implements Display for Vec.
As a work around, as indicated by Manishearth, you can use the Debug trait, which is invokable via "{:?}" as a format specifier.
If you know the type of the elements that the vector contains, you could make a struct that takes vector as an argument and implement Display for that struct.
use std::fmt::{Display, Formatter, Error};
struct NumVec(Vec<u32>);
impl Display for NumVec {
fn fmt(&self, f: &mut Formatter) -> Result<(), Error> {
let mut comma_separated = String::new();
for num in &self.0[0..self.0.len() - 1] {
comma_separated.push_str(&num.to_string());
comma_separated.push_str(", ");
}
comma_separated.push_str(&self.0[self.0.len() - 1].to_string());
write!(f, "{}", comma_separated)
}
}
fn main() {
let numbers = NumVec(vec![1; 10]);
println!("{}", numbers);
}
Here is a one-liner which should also work for you:
println!("[{}]", v2.iter().fold(String::new(), |acc, &num| acc + &num.to_string() + ", "));
Here is
a runnable example.
In my own case, I was receiving a Vec<&str> from a function call. I did not want to change the function signature to a custom type (for which I could implement the Display trait).
For my one-of case, I was able to turn the display of my Vec into a one-liner which I used with println!() directly as follows:
println!("{}", myStrVec.iter().fold(String::new(), |acc, &arg| acc + arg));
(The lambda can be adapted for use with different data types, or for more concise Display trait implementations.)
Starting with Rust 1.58, there is a slightly more concise way to print a vector (or any other variable). This lets you put the variable you want to print inside the curly braces, instead of needing to put it at the end. For the debug formatting needed to print a vector, you add :? in the braces, like this:
fn main() {
let v2 = vec![1; 10];
println!("{v2:?}");
}
Sometimes you don't want to use something like the accepted answer
let v2 = vec![1; 10];
println!("{:?}", v2);
because you want each element to be displayed using its Display trait, not its Debug trait; however, as noted, you can't implement Display on Vec because of Rust's coherence rules. Instead of implementing a wrapper struct with the Display trait, you can implement a more general solution with a function like this:
use std::fmt;
pub fn iterable_to_str<I, D>(iterable: I) -> String
where
I: IntoIterator<Item = D>,
D: fmt::Display,
{
let mut iterator = iterable.into_iter();
let head = match iterator.next() {
None => return String::from("[]"),
Some(x) => format!("[{}", x),
};
let body = iterator.fold(head, |a, v| format!("{}, {}", a, v));
format!("{}]", body)
}
which doesn't require wrapping your vector in a struct. As long as it implements IntoIterator and the element type implements Display, you can then call:
println!("{}", iterable_to_str(it));
Is there any reason not to write the vector's content item by item w/o former collecting? *)
use std::fmt::{Display, Formatter, Error};
struct NumVec(Vec<u32>);
impl Display for NumVec {
fn fmt(&self, f: &mut Formatter) -> Result<(), Error> {
let v = &self.0;
if v.len() == 0 {
return Ok(());
}
for num in &v[0..v.len() - 1] {
if let Err(e) = write!(f, "{}, ", &num.to_string()) {
return Err(e);
}
}
write!(f, "{}", &v[v.len() - 1])
}
}
fn main() {
let numbers = NumVec(vec![1; 10]);
println!("{}", numbers);
}
*) No there isn't.
Because we want to display something, the Display trait is implemented for sure. So this is correct Rust because: the Doc says about the ToString trait:
"This trait is automatically implemented for any type which implements the Display trait. As such, ToString shouldn’t be implemented directly: Display should be implemented instead, and you get the ToString implementation for free."
In particular on microcontrollers where space is limited I definitely would go with this solution and write immediately.

How can I handle "if may be missing an else clause" when using an if inside of a map?

I'm working on a code challenge which will detect case-insensitive anagrams of a given word from a list of words.
My first cut is to use something like this:
pub fn anagrams_for(s: &'static str, v: &[&'static str]) -> Vec<&'static str> {
let mut outputs: Vec<&str> = vec![];
// Find the case-insensitive, sorted word to check
let mut s_sorted: Vec<_> = s.to_string().to_lowercase().chars().collect();
s_sorted.sort();
for word in v {
// Case-desensitize and sort each word in the slice
let mut word_sorted: Vec<_> = word.to_string().to_lowercase().chars().collect();
word_sorted.sort();
// if the case-insensitive words are the same post sort and not presort (to avoid self-anagrams), add it to the vector
if word_sorted == s_sorted && s.to_string().to_lowercase() != word.to_string().to_lowercase() {
outputs.push(word)
}
}
outputs
}
This works as expected, but is not very idiomatic. I'm now trying a second iteration which uses more functional features of Rust:
pub fn anagrams_for(s: &'static str, v: &[&'static str]) -> Vec<&'static str> {
let mut s_sorted: Vec<_> = s.to_string().to_lowercase().chars().collect();
s_sorted.sort();
v.iter().map(&|word: &str| {
let mut word_sorted: Vec<_> = word.to_string().to_lowercase().chars().collect();
word_sorted.sort();
if word_sorted == s_sorted && s.to_string().to_lowercase() != word.to_string().to_lowercase() {
word
}
}).collect()
}
I'm currently getting a few errors (most of which I could likely resolve), but the one I'm interested in solving is
if may be missing an else clause:
expected `()`,
found `&str`
(expected (),
found &-ptr) [E0308]
This is because in the case of a non-anagram, map attempts to push something into the vector (seemingly ()).
How can I handle this? It's possible that map isn't the best idiom because it requires some operation to be performed on each element in a list, not a subset (maybe filter?).
As you noticed, the problem is that in the non-anagram-case your closure (the || { ... } block) doesn't return a value.
You can solve this by using filter_map instead of map. That function takes a closure that returns Option<U> instead of U, so the last expression of your closure looks something like:
if /* ... */ {
Some(word)
} else {
None
}
Unrelated to the main question, some notes on your code:
You can remove the .to_string() calls before .to_lowercase() calls. the latter method belongs to the type str, so it works fine. Calling to_string() adds unnecessary allocations.
the & in front of the closure (&|...|) can most probably be removed...
... as can the : &str type annotation in the closures argument list

Resources