Can't use Vec two times and I can't borrow it instead - vector

I attempted to implement the Rosetta Code password generator:
extern crate rand;
use rand::prelude::*;
fn main() {
println!("Hello, world!");
let p = generate_password(12, 5);
for i in p.iter() {
println!("{:?}", i);
}
}
fn generate_password(length: i32, number: i32) -> Vec<Vec<String>> {
let lowercase = "abcdefghijklmnopqrstuvwxyz";
let uppercase = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
let listnumber = "0123456789";
let other = "!\\\"#$%&'()*+,-./:;<=>?#[]^_{|}~";
let all: Vec<char> = String::from(format!("{}{}{}{}", lowercase, uppercase, listnumber, other))
.chars()
.collect();
let mut password: Vec<String> = Vec::new();
let mut password_list: Vec<Vec<String>> = Vec::new();
for num in 1..number {
for l in 1..length {
password.push(String::from(thread_rng().choose(&all).unwrap().to_string()));
}
password_list.push(&password);
}
return password_list;
}
Rust won't allow me to use either borrowed value or direct value:
error[E0308]: mismatched types
--> src/main.rs:26:28
|
26 | password_list.push(&password);
| ^^^^^^^^^
| |
| expected struct `std::vec::Vec`, found reference
| help: consider removing the borrow: `password`
|
= note: expected type `std::vec::Vec<std::string::String>`
found type `&std::vec::Vec<std::string::String>`
The help message says I should remove the borrow because of type mismatch but it's still got an error after removing it because the value has been moved.

You've declared a type to be Vec<Vec<String>>, but you're trying to store a reference inside it.
When you remove the reference, you're getting a different error because push takes ownership of the value, so the original variable can no longer be used. But you then try to use it in the subsequent loop. The easy fix is to declare the variable inside the loop, so it is a new variable each time:
let mut password_list = Vec::new();
for num in 1..number {
let mut password = Vec::new();
for l in 1..length {
password.push(String::from(thread_rng().choose(&all).unwrap().to_string()));
}
password_list.push(password);
}
Note that you don't need a lot of the type annotations, especially on local function variables. The compiler can infer them, which makes the code a lot cleaner.

Related

Which Generic Types are allowed in the `Vec<T>` implementation of `ParallelExtend`

I am new to Rust and I am writing a recursive directory traversal program. This project is just an opportunity to learn Rust and parallelism with Rayon. This question is a continuation of this question which #KevinReid helped out a lot with.
Below is my attempt to use partition_map() followed by the errors I am receiving.
use std::fs;
use std::path::PathBuf;
use rayon::iter::{Either, IntoParallelIterator, ParallelIterator};
#[derive(Debug)]
struct Node {
path: PathBuf,
files: Vec<PathBuf>,
folders: Vec<Box<Node>>,
}
impl Node {
pub fn new(path: PathBuf) -> Self {
Node {
path: path,
files: Vec::new(),
folders: Vec::new(),
}
}
pub fn burrow(&mut self) {
let mut contents: Vec<PathBuf> = ls_dir(&self.path);
let ret: (Vec<PathBuf>, Vec<Box<Node>>) = contents.into_par_iter().partition_map(|item|
if item.is_file() {
self.files.push(item);
Either::Left(self.files)
} else {
let mut new_folder = Node::new(item);
new_folder.burrow();
self.folders.push(Box::new(new_folder));
Either::Right(self.folders)
});
println!("{:?}", ret);
}
}
fn ls_dir(path: &PathBuf) -> Vec<PathBuf> {
let mut contents: Vec<PathBuf> = Vec::new();
let current_dir: fs::ReadDir = fs::read_dir(path).unwrap();
for file in current_dir {
contents.push(file.unwrap().path());
}
contents
}
fn main() {
let root_path: &str = ".";
let root: PathBuf = PathBuf::from(root_path);
let contents: Vec<PathBuf> = ls_dir(&root);
let mut node: Node = Node::new(root);
node.burrow();
}
error[E0277]: the trait bound `Vec<PathBuf>: ParallelExtend<Vec<PathBuf>>` is not satisfied
--> src/main.rs:31:76
|
31 | let ret: (Vec<PathBuf>, Vec<Box<Node>>) = contents.into_par_iter().partition_map(|item|
| ^^^^^^^^^^^^^ the trait `ParallelExtend<Vec<PathBuf>>` is not implemented for `Vec<PathBuf>`
|
= help: the following implementations were found:
<Vec<T> as ParallelExtend<&'a T>>
<Vec<T> as ParallelExtend<T>>
note: required by a bound in `partition_map`
--> /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.5.3/src/iter/mod.rs:2167:29
|
2167 | A: Default + Send + ParallelExtend<L>,
| ^^^^^^^^^^^^^^^^^ required by this bound in `partition_map`
error[E0277]: the trait bound `Vec<Box<Node>>: ParallelExtend<Vec<Box<Node>>>` is not satisfied
--> src/main.rs:31:76
|
31 | let ret: (Vec<PathBuf>, Vec<Box<Node>>) = contents.into_par_iter().partition_map(|item|
| ^^^^^^^^^^^^^ the trait `ParallelExtend<Vec<Box<Node>>>` is not implemented for `Vec<Box<Node>>`
|
= help: the following implementations were found:
<Vec<T> as ParallelExtend<&'a T>>
<Vec<T> as ParallelExtend<T>>
note: required by a bound in `partition_map`
--> /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.5.3/src/iter/mod.rs:2168:29
|
2168 | B: Default + Send + ParallelExtend<R>,
| ^^^^^^^^^^^^^^^^^ required by this bound in `partition_map`
For more information about this error, try `rustc --explain E0277`.
error: could not compile `my-project` due to 2 previous errors
exit status 101
It appears that ParallelExtend<T> for Vec<T> is producing the error because of what T is, namely, PathBuf and Box<Node>. I have seen a smaller non-recursive example where T is &str, for example in which these errors are not present. So the question is if this is fixable and just not implemented correctly, or PathBuf and Box<T> inherently do not satisfy something that ParallelExtend requires.
To test if the type T was the issue I ran
pub fn burrow(&mut self) {
let mut contents: Vec<PathBuf> = ls_dir(&self.path);
let ret: (Vec<&str>, Vec<&str>) = contents.into_par_iter().partition_map(|item|
if item.is_file() {
self.files.push(item);
Either::Left("test")
} else {
let mut new_folder = Node::new(item);
new_folder.burrow();
self.folders.push(Box::new(new_folder));
Either::Right("test")
});
println!("{:?}", ret);
}
and received
error[E0596]: cannot borrow `*self.files` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:33:37
|
33 | ... self.files.push(item);
| ^^^^^^^^^^^^^^^^^^^^^ cannot borrow as mutable
error[E0596]: cannot borrow `*self.folders` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:38:37
|
38 | ... self.folders.push(Box::new(new_folder));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot borrow as mutable
For more information about this error, try `rustc --explain E0596`.
error: could not compile `my-project` due to 2 previous errors
exit status 101
which is a totally different issue/concept. Needless to say, with the added recursive component, I am lost in the sauce.

Parallel Recursion Fix

Quite new to Rust and trying to tackle toy problems. Trying to write a directory traversal with only Rayon.
struct Node {
path: PathBuf,
files: Vec<PathBuf>,
hashes: Vec<String>,
folders: Vec<Box<Node>>,
}
impl Node {
pub fn new(path: PathBuf) -> Self {
Node {
path: path,
files: Vec::new(),
hashes: Vec::new(),
folders: Vec::new(),
}
}
pub fn burrow(&mut self) {
let mut contents: Vec<PathBuf> = ls_dir(&self.path);
contents.par_iter().for_each(|item|
if item.is_file() {
self.files.push(*item);
} else if item.is_dir() {
let mut new_folder = Node::new(*item);
new_folder.burrow();
self.folders.push(Box::new(new_folder));
});
}
}
The errors I am receiving are
error[E0596]: cannot borrow `*self.files` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:40:37
|
40 | ... self.files.push(*item);
| ^^^^^^^^^^^^^^^^^^^^^^ cannot borrow as mutable
error[E0507]: cannot move out of `*item` which is behind a shared reference
--> src/main.rs:40:53
|
40 | ... self.files.push(*item);
| ^^^^^ move occurs because `*item` has type `PathBuf`, which does not implement the `Copy` trait
error[E0507]: cannot move out of `*item` which is behind a shared reference
--> src/main.rs:42:68
|
42 | ... let mut new_folder = Node::new(*item);
| ^^^^^ move occurs because `*item` has type `PathBuf`, which does not implement the `Copy` trait
error[E0596]: cannot borrow `*self.folders` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:44:37
|
44 | ... self.folders.push(Box::new(new_folder));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot borrow as mutable
The errors are clear in that they are preventing different threads from accessing mutable memory, but I'm just not sure how to start to address the errors.
Below is the original (non-parallel) version of burrow
pub fn burrow(&mut self) {
let mut contents: Vec<PathBuf> = ls_dir(&self.path);
for item in contents {
if item.is_file() {
self.files.push(item);
} else if item.is_dir() {
let mut new_folder = Node::new(item);
new_folder.burrow();
self.folders.push(Box::new(new_folder));
}
}
}
The best option in this case is to use ParallelIterator::partition_map() which allows you to turn a parallel iterator into two different collections according to some condition, which is exactly what you need to do.
Example program:
use rayon::iter::{Either, IntoParallelIterator, ParallelIterator};
fn main() {
let input = vec!["a", "bb", "c", "dd"];
let (chars, strings): (Vec<char>, Vec<&str>) =
input.into_par_iter().partition_map(|s| {
if s.len() == 1 {
Either::Left(s.chars().next().unwrap())
} else {
Either::Right(s)
}
});
dbg!(chars, strings);
}
If you had three different outputs, unfortunately Rayon does not support that. I haven't looked at whether it'd be possible to build using Rayon's traits, but what I would suggest as a more general (though not quite as efficient) solution is to use channels. A channel like std::sync::mpsc allows any number of threads to insert items while another thread removes them — in your case, to move them into a collection. This would not be quite as efficient as parallel collection, but in an IO-dominated problem like yours, it would not be significant.
I'm going to skip the separation of files and folders, ignore the structure, and demonstrate a simple recursive approach that gets all the files in a directory recursively:
fn burrow(dir: &Path) -> Vec<PathBuf> {
let mut contents = vec![];
for entry in std::fs::read_dir(dir).unwrap() {
let entry = entry.unwrap().path();
if entry.is_dir() {
contents.extend(burrow(&entry));
} else {
contents.push(entry);
}
}
contents
}
The first step if you want to use the parallel iterators from rayon, is to convert this loop into a non-parallel iterator chain. The best way to do that is with .flat_map() to flatten results that yield more than one element:
fn burrow(dir: &Path) -> Vec<PathBuf> {
std::fs::read_dir(dir)
.unwrap()
.flat_map(|entry| {
let entry = entry.unwrap().path();
if entry.is_dir() {
burrow(&entry)
} else {
vec![entry] // use a single-element Vec if not a directory
}
})
.collect()
}
Then to use rayon to process this iteration in parallel is to use .par_bridge() to convert an iterator into a parallel iterator. And that's it actually:
use rayon::iter::{ParallelBridge, ParallelIterator};
fn burrow(dir: &Path) -> Vec<PathBuf> {
std::fs::read_dir(dir)
.unwrap()
.par_bridge()
.flat_map(|entry| {
let entry = entry.unwrap().path();
if entry.is_dir() {
burrow(&entry)
} else {
vec![entry]
}
})
.collect()
}
See it working on the playground. You can extend on this to collect more complex results (like folders and hashes and whatever else).

Using a Vec<u8> to mock file Write

My function signature is:
fn write_header(source_pkey: PublicKey, target_pkey: PublicKey, nonce: Nonce,
mut output: &mut Box<&mut dyn Write>) -> anyhow::Result<()> {
I'm trying to test it by using a Vec<u8> to mock a file:
#[test]
fn test_write_header() {
let mut vec = Vec::<u8>::new();
let mut out = Box::new(&mut vec);
write_header(
PublicKey([1u8; 32]),
PublicKey([2u8; 32]),
Nonce([3u8; 24]),
&mut out
).unwrap();
assert_eq!(out.len(), 104);
}
but I get the error:
error[E0308]: mismatched types
--> src/encrypt.rs:45:13
|
45 | &mut out
| ^^^^^^^^ expected trait object `dyn std::io::Write`, found struct `Vec`
|
= note: expected mutable reference `&mut Box<&mut dyn std::io::Write>`
found mutable reference `&mut Box<&mut Vec<u8>>
I am using dyn std::io::Write as I need this function to accept both File and Stdout in normal operation, as well as Vec<u8> for testing.
Am I doing this all wrong, or is it a matter of convincing the compiler that Vec<u8> has the Write trait?
Update:
The Box is because this code wouldn't compile without it, due to Sized issues.
/// Open the program's output file, or stdout if there is no input file.
/// Note: stdout on Windows only accepts utf8.
pub fn open_output(output: Option<String>) -> anyhow::Result<Box<dyn Write>> {
if let Some(filename) = output {
Ok(Box::new(File::open(&filename)
.context(format!("unable to open '{filename}' for output"))?))
} else {
Ok(Box::new(stdout()))
}
}
Is there a way to remove this Box? (Apologies for a follow-on question.)
As per #FrancisGagne comments, you can use directly &mut dyn Write, and then pass a &mut vec:
use anyhow; // 1.0.52
use std::io::Write;
fn write_header(
source_pkey: (),
target_pkey: (),
nonce: (),
mut output: &mut dyn Write,
) -> anyhow::Result<()> {
output.write(&[1])?;
anyhow::Ok(())
}
#[test]
fn test_write_header() {
let mut vec = Vec::<u8>::new();
write_header(
(),
(),
(),
&mut vec
).unwrap();
assert_eq!(vec.len(), 1);
}
Playground
Also you can use impl Write as the input parameter. This is neat, since you actually can make your function work for anything that implements Write itself:
fn write_header(
source_pkey: (),
target_pkey: (),
nonce: (),
mut output: impl Write,
) -> anyhow::Result<()> {
output.write(&[1])?;
anyhow::Ok(())
}
Playground
The solution was to keep the Box at the top level, as main() does not know whether output is a Stdout, a File or something else.
I've changed write_header's signature to:
fn write_header(source_pkey: PublicKey, target_pkey: PublicKey, nonce: Nonce,
output: &mut dyn Write) -> anyhow::Result<()> {
at Fracis Gagné's suggestion, which fixes the original issue.

Unable to return a vector of string slices: borrowed value does not live long enough

I'm new to Rust and I'm having some trouble with the borrow checker. I don't understand why this code won't compile. Sorry if this is close to a previously answered question but I can't seem to find a solution in the other questions I've looked at.
I understand the similarity to Return local String as a slice (&str) but in that case it is just one string being returned and not enough for me to reason with my code in which I am trying to return a vector. From what I understand, I am trying to return references to str types that will go out of scope at the end of the function block and so should I be mapping that vector of &str into a vector of String? I am not so concerned about the performance effects of converting &str to String. First I'd just like to get it working.
This is the code, the error is in the lex function.
use std::io::prelude::*;
use std::fs::File;
use std::env;
fn open(mut s: &mut String, filename: &String) {
let mut f = match File::open(&filename) {
Err(_) => panic!("Couldn't open file"),
Ok(file) => file,
};
match f.read_to_string(&mut s) {
Err(_) => panic!("Couldn't read file"),
Ok(_) => println!("File read successfully"),
};
}
fn lex(s: &String) -> Vec<&str> {
let token_string: String = s.replace("(", " ( ")
.replace(")", " ) ");
let token_list: Vec<&str> = token_string.split_whitespace()
.collect();
token_list
}
fn main() {
let args: Vec<_> = env::args().collect();
if args.len() < 2 {
panic!("Please provide a filename");
} else {
let ref filename = args[1];
let mut s = String::new();
open(&mut s, filename);
let token_list: Vec<&str> = lex(&s);
println!("{:?}", token_list);
}
}
Here is the error message
error: borrowed value does not live long enough
self.0.borrow().values.get(idx)
^~~~~~~~~~~~~~~
reference must be valid for the anonymous lifetime #1 defined on the block at 23:54...
pub fn value(&self, idx: usize) -> Option<&Value> {
^
note: ...but borrowed value is only valid for the block at 23:54
pub fn value(&self, idx: usize) -> Option<&Value> {
^
I'm finding it hard to reason with this code because with my level of experience with Rust I can't visualise the lifetimes of these variables. Any help would be appreciated as I've spent an hour or two trying to figure this out.
The problem is that you're allocating a new String (token_string) inside the lex function and then returning an array of references to it, but token_string will get dropped (and the memory freed) as soon as it falls out of scope at the end of the function.
fn lex(s: &String) -> Vec<&str> {
let token_string: String = s.replace("(", " ( ") // <-- new String allocated
.replace(")", " ) ");
let token_list: Vec<&str> = token_string.split_whitespace()
.collect();
token_list // <-- this is just an array of wide pointers into token_string
} // <-- token_string gets freed here, so the returned pointers
// would be pointing to memory that's already been dropped!
There's a couple of ways to address this. One would be to force the caller of lex to pass in the buffer that you want to use to collect into. This would change the signature to fn lex<'a>(input: &String, buffer: &'a mut String) -> Vec<&'a str> This signature would specify that the lifetimes of the returned &strs will be at least as long as the lifetime of the buffer that's passed in.
Another way would be to just return a Vec<String> instead of Vec<&str> if you can tolerate the extra allocations.

Trait is not implemented for the type `&A` when passing an array of pairs to a function

I am trying to write the function set which calls the Rust LMDB library (docs), and an example I'm working off of.
I can't for the life of me get this to work. Here is my current attempt:
fn main() {
let env = getenv("duperdb");
let dbhandle = get_dbhandle("", &env);
let txn = new_transaction(&env);
let vec = vec![("foo", "another text"), ("bar", "and another")];
set(&dbhandle, &env, &vec);
let reader = env.get_reader().unwrap();
let db = reader.bind(&dbhandle);
let note = db.get::<&str>("foo").unwrap();
println!("NOTE: {}", note);
}
Where set is defined as:
pub fn set<A: ToMdbValue, B: ToMdbValue>(
handle: &DbHandle,
env: &Environment,
pairs: &Vec<(&A, &B)>) -> () {
let txn = new_transaction(&env);
{
let db = txn.bind(&handle);
for &(id, note) in pairs.iter() {
db.set(&id, &note).unwrap();
}
}
match txn.commit() {
Err(_) => panic!("Failed to commit!"),
Ok(_) => (),
}
}
This spits out the following error:
src/db/wrapper.rs:28:20: 28:23 error: the trait `lmdb::traits::ToMdbValue` is not implemented for the type `&A` [E0277]
src/db/wrapper.rs:28 db.set(&id, &note).unwrap();
^~~
I also tried db.set(id, note).unwrap();, but this time I get:
src/main.rs:13:5: 13:8 error: the trait `core::marker::Sized` is not implemented for the type `str` [E0277]
src/main.rs:13 set(&dbhandle, &env, &vec);
^~~
src/main.rs:13:5: 13:8 help: run `rustc --explain E0277` to see a detailed explanation
src/main.rs:13:5: 13:8 note: `str` does not have a constant size known at compile-time
src/main.rs:13:5: 13:8 note: required by `dupernote::db::wrapper::set`
src/main.rs:13:5: 13:8 error: the trait `lmdb_rs::traits::ToMdbValue` is not implemented for the type `str` [E0277]
src/main.rs:13 set(&dbhandle, &env, &vec);
^~~
I also tried stuff like:
for (id, note) in pairs.iter() {
db.set(id, note).unwrap();
}
But that doesn't work either... I don't fully understand why. Doesn't id and note have type &str, not str?
Here's an MCVE of your problem:
trait Example {}
impl Example for i32 {}
fn library_call<T>(value: T)
where T: Example,
{}
fn user_call<T>(values: &[T])
where T: Example,
{
for i in values {
library_call(i);
}
}
fn main() {
let values = vec![1, 2, 3];
user_call(&values);
}
With the error:
error: the trait `Example` is not implemented for the type `&T` [E0277]
library_call(i);
^~~~~~~~~~~~
The error message is exactly correct - Example is not implemented for &T, it's only guaranteed to be implemented for T. &T and T are different types.
Instead, you need to indicate that a reference to the generic type implements the trait you need:
fn user_call<T>(values: &[T])
where for <'a> &'a T: Example,
And then you need to make sure that a reference to the concrete type actually implements the trait:
impl<'a> Example for &'a i32 {}
Or a broader version:
impl<'a, T> Example for &'a T
where T: Example
{}
See also When should I not implement a trait for references to implementors of that trait?
The definition of the function that gives you an error (if I'm reading the docs right):
fn set(&self, key: &ToMdbValue, value: &ToMdbValue) -> MdbResult<()>
key must be a reference to a trait object. You are trying to pass a reference to a generic type implmementing ToMdbValue.
https://doc.rust-lang.org/book/trait-objects.html
I can't verify but this should work:
pub fn set(handle: &DbHandle, env: &Environment, pairs: &Vec<(&ToMdbValue, &ToMdbValue)>) -> () {
let txn = new_transaction(&env);
{
let db = txn.bind(&handle);
for &(id, note) in pairs.iter() {
db.set(id, note).unwrap();
}
}
match txn.commit() {
Err(_) => panic!("Failed to commit!"),
Ok(_) => (),
}
}
Other things: you may want to work with boxed trait objects Box<ToMdbValue>. The link above explains it. You should pass a &[YourType] rather than &Vec<[YourType]>.
I managed to get it working. I'm not sure how kosher this solution is, but I'll post it.
So now, in main(), I do the following (example with an (int, string) kv pair):
let k = 1;
let val = "hello there";
let vec = vec![(&k, &val)];
set(&dbhandle, &env, &vec);
I had to declare them separately since vec![(&1, &"hello there")] threw an error of the form borrowed value does not live long enough.
set now looks like this:
pub fn set<A, B>(handle: &DbHandle, env: &Environment, pairs: &Vec<(&A, &B)>)
-> ()
where A: ToMdbValue,
B: ToMdbValue {
let txn = new_transaction(&env);
{
let db = txn.bind(&handle);
for &(id, note) in pairs.iter() {
db.set(id, note).unwrap();
}
}
match txn.commit() {
Err(_) => panic!("Failed to commit!"),
Ok(_) => (),
}
}

Resources