How to make async system function in the bevy game engine? - asynchronous

I am currently working on a 3D voxel-based game and I want to have procedural generated chunks based on player movement.
But running the chunk generation in a simple system results in huge FPS drops.
I already tried to create a task pool that runs independent from everything else using std::sync::mpsc::channel, to generate the chunks data and mesh, which then gets requested by a normal bevy system, buffered and then spawned using commands.spawn(PbrBundle{...}).
fn chunk_loader(
pool: Res<Pool>,
mut commands: Commands,
mut meshes: ResMut<Assets<Mesh>>,
mut materials: ResMut<Assets<StandardMaterial>>,
mut chunkmap: ResMut<ChunkMap>,
mut buffer: ResMut<Buffer>, // buffer of chunk data
) {
let mut chunks = pool.request_chunks();
buffer.0.append(&mut chunks);
for _ in 0..CHUNK_UPDATES_PER_FRAME {
if let Some( (chunk, mesh) ) = buffer.0.pop() {
chunkmap.map.insert([chunk.x, chunk.y, chunk.z], chunk);
let mesh = mesh.clone();
commands.spawn_bundle(PbrBundle {
mesh: meshes.add(mesh),
transform: Transform::from_matrix(Mat4::from_scale_rotation_translation(
Vec3::splat(1.0),
Quat::from_rotation_x(0.0),
Vec3::new((chunk.x * CHUNK_SIZE as i64) as f32, (chunk.y * CHUNK_SIZE as i64) as f32, (chunk.z * CHUNK_SIZE as i64) as f32),
)),
material: materials.add(StandardMaterial {
base_color: Color::hex("993939").unwrap(),
perceptual_roughness: 0.95,
..default()
}),
..default()
}).insert(super::LoadedChunk{x: chunk.x, y: chunk.y, z: chunk.z, should_unload: false});
}
}
}
But this does not help, because it still takes up too much time.
What I need is a way to execute the chunk generation and spawning in an async fashion, which does not affect frame rate, but I do not know how I should approach this.
bevy::prelude::AsyncComputeTaskPool might do the job, but I can't find any documentation or examples, so I do not know what exactly it does, other than async an parallel iteration over queries.
I have never written any async code, can anyone help me?
EDIT
I found out that the system above is actually working quite nice.
The problem is the system I use to check, which chunks to load.
I use a HashMap to store every Chunk and everytime the player moves I test multiple chunks if they are already spawned and if not send a request to the task pool to do so.
fn chunk_generation(
mut query: Query<(&Transform, &mut Player)>,
mut chunks: ResMut<ChunkMap>,
pool: Res<Pool>,
) {
let mut player_pos = Vec3::ZERO;
let mut player_moved: bool = false;
for (transform, mut player) in query.iter_mut().next() {
player_pos = transform.translation;
if player.chunks_moved_since_update > 0 {
player_moved = true;
player.chunks_moved_since_update = 0;
}
}
let chunk_x = (player_pos.x / CHUNK_SIZE as f32).round() as i64;
let chunk_y = (player_pos.y / CHUNK_SIZE as f32).round() as i64;
let chunk_z = (player_pos.z / CHUNK_SIZE as f32).round() as i64;
// loading
if player_moved {
let mut chunks_to_load: Vec<[i64; 3]> = Vec::new();
for x in -RENDER_DISTANCE_HOR..RENDER_DISTANCE_HOR {
for y in -RENDER_DISTANCE_VER..RENDER_DISTANCE_VER {
for z in -RENDER_DISTANCE_HOR..RENDER_DISTANCE_HOR {
let chunk_x = chunk_x as i64 + x;
let chunk_y = chunk_y as i64 + y;
let chunk_z = chunk_z as i64 + z;
let chunk_coords = [chunk_x, chunk_y, chunk_z];
// check if chunk is not in world
if !chunks.map.contains_key(&chunk_coords) {
println!("generating new chunk");
let chunk = Chunk::new(chunk_x, chunk_y, chunk_z);
chunks.map.insert(chunk_coords, chunk);
chunks_to_load.push([chunk_x, chunk_y, chunk_z]);
}
}
}
}
pool.send_chunks(chunks_to_load);
}
}
Is it possible to make this async too?

bevy::tasks::AsyncComputeTaskPool wrappend in bevy::tasks::Task was the method I was searching for.
I now have a system that inserts a task into each chunk to generate itself and then poll for this task in another function like this:
#[derive(Component)]
pub struct GenTask {
pub task: Task<Vec<([i64; 3], ChunkData)>>
}
pub fn prepare_gen_tasks(
mut queue: ResMut<ChunkGenQueue>,
chunks: Res<ChunkMap>,
pool: Res<AsyncComputeTaskPool>,
mut cmds: Commands,
) {
while let Some(key) = queue.queue.pop() {
if let Some(entity) = chunks.entity(key) {
let task = pool.spawn(async move {
let chunk = Chunk::new(key);
super::generation::generate(chunk)
});
cmds.entity(entity).insert(GenTask{task});
}
}
}
pub fn apply_gen_tasks(
mut voxels: ResMut<VoxelMap>,
mut query: Query<(Entity, &mut GenTask)>,
mut mesh_queue: ResMut<ChunkMeshQueue>,
mut cmds: Commands,
) {
query.for_each_mut(|(entity, mut gen_task)| {
if let Some(chunks) = future::block_on(future::poll_once(&mut gen_task.task)) {
for (key, data) in chunks.iter() {
voxels.map.insert(*key, *data);
mesh_queue.queue.push(*key);
}
cmds.entity(entity).remove::<GenTask>();
}
return;
});
}

Related

Is there a better way to infinitely error check other than recursion?

I'm trying to make a Tic-Tac-Toe game with a custom board size. I want this to be very hard to break, so I use recursion to get the board measurements if the input is invalid or an error occurs. However, this doesn't seem very clean to me, and I was wondering if there's a better/more rusty way of achieving the same thing.
Code in main function
let board_size_str = get_board_size_string();
let (x_pos, width, height) = get_board_measurement(&board_size_str);
Functions
fn get_board_size_string() -> String {
println!("Select the size of the board in the following format: 5x5 or 7x7");
println!("The size can be from 3x3 to 30x30");
print!("Size: ");
std::io::stdout().flush().expect("Failed to flush stdout!");
let mut board_size_str = String::new();
std::io::stdin().read_line(&mut board_size_str).expect("Failed to read board size!");
println!();
board_size_str
}
fn get_board_measurement(board_size_str: &str) -> (usize, i64, i64) {
let x_pos = get_x_pos(board_size_str);
let width = get_board_width(board_size_str, x_pos);
let height = get_board_height(board_size_str, x_pos);
(x_pos, width, height)
}
fn get_x_pos(board_size_str: &str) -> usize {
let x_pos_option = board_size_str.chars().position(|c| c == 'x');
match x_pos_option {
Some(x_pos) => x_pos,
None => {
println!("Board size must contain an x!");
let board_size_str = get_board_size_string();
get_x_pos(&board_size_str)
}
}
}
fn get_board_width(board_size_str: &str, x_pos: usize) -> i64 {
let width_result = board_size_str[..x_pos].parse::<i64>();
match width_result {
Ok(width) => width,
Err(_) => {
println!("Invalid board width!");
let board_size_str = get_board_size_string();
get_board_width(&board_size_str, get_x_pos(&board_size_str))
}
}
}
fn get_board_height(board_size_str: &str, x_pos: usize) -> i64 {
let height_result = board_size_str[x_pos + 1..].trim().parse::<i64>();
match height_result {
Ok(height) => height,
Err(_) => {
println!("Invalid board height!");
let board_size_str = get_board_size_string();
get_board_height(&board_size_str, get_x_pos(&board_size_str))
}
}
}
Just use an iterative loop?
fn get_x_pos(board_size_str: &str) -> usize {
loop {
let board_size_str = get_board_size_string();
let x_pos_option = board_size_str.chars().position(|c| c == 'x');
if let Some(x_pos) = x_pos_option {
break x_pos
}
}
}
Though the structure is strange because a correct board size is a correct pattern ( 'x' ) so it's not like splitting that into three unrelated routines makes any sense, even if two of them do delegate the localisation of the x separator.
With your method you can input something like 52xkf, get an error, input 24x36, and I think you'll get a 52x36 board rather than the 24x36 you might expect, which is just odd. Would be a lot easier to just do the entire thing in a single pseudo-step:
fn parse_board_size() -> (usize, usize) {
loop {
let s = get_board_size_string();
let Some((w_s, h_s)) = s.split_once('x') else {
// complain about a missing `x` here
continue;
};
match (w_s.parse(), h_s.parse()) {
(Ok(w), Ok(s)) => {
// can add more validation here,
// or as pattern guards
return (w, s);
}
(Ok(_), Err(h_error)) => {
// h was incorrect
}
(Err(w_error), Ok(_)) => {
// w was incorrect
}
(Err(w_error), Err(h_error)) => {
// both were incorrect
}
}
}
}
Alternatively for the parsing if you don't care about custom-reporting each error case individually you can lean on Option e.g.
fn parse_board_size() -> (usize, usize) {
loop {
let s = get_board_size_string();
let Some((w_s, h_s)) = s.split_once('x') else {
// complain about a missing `x` here
continue;
};
if let Some(r) = w_s.parse().ok().zip(h_s.parse().ok()) {
break r;
}
// report generic parsing error
}
}

How do I build an iterator for walking a file tree recursively?

I want to lazily consume the nodes of a file tree one by one while sorting the siblings on each level.
In Python, I'd use a synchronous generator:
def traverse_dst(src_dir, dst_root, dst_step):
"""
Recursively traverses the source directory and yields a sequence of (src, dst) pairs;
"""
dirs, files = list_dir_groom(src_dir) # Getting immediate offspring.
for d in dirs:
step = list(dst_step)
step.append(d.name)
yield from traverse_dst(d, dst_root, step)
for f in files:
dst_path = dst_root.joinpath(step)
yield f, dst_path
In Elixir, a (lazy) stream:
def traverse_flat_dst(src_dir, dst_root, dst_step \\ []) do
{dirs, files} = list_dir_groom(src_dir) # Getting immediate offspring.
traverse = fn d ->
step = dst_step ++ [Path.basename(d)]
traverse_flat_dst(d, dst_root, step)
end
handle = fn f ->
dst_path =
Path.join(
dst_root,
dst_step
)
{f, dst_path}
end
Stream.flat_map(dirs, traverse)
|> Stream.concat(Stream.map(files, handle))
end
One can see some language features addressing recursion: yield from in Python, flat_map in Elixir; the latter looks like a classic functional approach.
It looks like whatever is lazy in Rust, it's always an iterator. How am I supposed to do more or less the same in Rust?
I'd like to preserve the structure of my recursive function with dirs and files as vectors of paths (they are optionally sorted and filtered).
Getting dirs and files is already implemented to my liking:
fn folders(dir: &Path, folder: bool) -> Result<Vec<PathBuf>, io::Error> {
Ok(fs::read_dir(dir)?
.into_iter()
.filter(|r| r.is_ok())
.map(|r| r.unwrap().path())
.filter(|r| if folder { r.is_dir() } else { !r.is_dir() })
.collect())
}
fn list_dir_groom(dir: &Path) -> (Vec<PathBuf>, Vec<PathBuf>) {
let mut dirs = folders(dir, true).unwrap();
let mut files = folders(dir, false).unwrap();
if flag("x") {
dirs.sort_unstable();
files.sort_unstable();
} else {
sort_path_slice(&mut dirs);
sort_path_slice(&mut files);
}
if flag("r") {
dirs.reverse();
files.reverse();
}
(dirs, files)
}
Vec<PathBuf can be iterated as is, and there is standard flat_map method. It should be possible to implement Elixir style, I just can't figure it out yet.
This is what I already have. Really working (traverse_flat_dst(&SRC, [].to_vec());), I mean:
fn traverse_flat_dst(src_dir: &PathBuf, dst_step: Vec<PathBuf>) {
let (dirs, files) = list_dir_groom(src_dir);
for d in dirs.iter() {
let mut step = dst_step.clone();
step.push(PathBuf::from(d.file_name().unwrap()));
println!("d: {:?}; step: {:?}", d, step);
traverse_flat_dst(d, step);
}
for f in files.iter() {
println!("f: {:?}", f);
}
}
What I want (not yet working!):
fn traverse_flat_dst_iter(src_dir: &PathBuf, dst_step: Vec<PathBuf>) {
let (dirs, files) = list_dir_groom(src_dir);
let traverse = |d| {
let mut step = dst_step.clone();
step.push(PathBuf::from(d.file_name().unwrap()));
traverse_flat_dst_iter(d, step);
};
// This is something that I just wish to be true!
flat_map(dirs, traverse) + map(files)
}
I want this function to deliver one long flat iterator of files, in the spirit of the Elixir solution. I just can't yet cope with the necessary return types and other syntax. I really hope to be clear enough this time.
What I managed to compile and run (meaningless, but the signature is what I actually want):
fn traverse_flat_dst_iter(
src_dir: &PathBuf,
dst_step: Vec<PathBuf>,
) -> impl Iterator<Item = (PathBuf, PathBuf)> {
let (dirs, files) = list_dir_groom(src_dir);
let _traverse = |d: &PathBuf| {
let mut step = dst_step.clone();
step.push(PathBuf::from(d.file_name().unwrap()));
traverse_flat_dst_iter(d, step)
};
files.into_iter().map(|f| (f, PathBuf::new()))
}
What I'm still lacking:
fn traverse_flat_dst_iter(
src_dir: &PathBuf,
dst_step: Vec<PathBuf>,
) -> impl Iterator<Item = (PathBuf, PathBuf)> {
let (dirs, files) = list_dir_groom(src_dir);
let traverse = |d: &PathBuf| {
let mut step = dst_step.clone();
step.push(PathBuf::from(d.file_name().unwrap()));
traverse_flat_dst_iter(d, step)
};
// Here is a combination amounting to an iterator,
// which delivers a (PathBuf, PathBuf) tuple on each step.
// Flat mapping with traverse, of course (see Elixir solution).
// Iterator must be as long as the number of files in the tree.
// The lines below look very close, but every possible type is mismatched :(
dirs.into_iter().flat_map(traverse)
.chain(files.into_iter().map(|f| (f, PathBuf::new())))
}
There are two approaches:
The first one is to use an existing crate, like walkdir. The benefit is it's being well tested and offers many options.
The second one is to write your own implementation of Iterator. Here's an example, and maybe the basis for your own:
struct FileIterator {
dirs: Vec<PathBuf>, // the dirs waiting to be read
files: Option<ReadDir>, // non recursive iterator over the currently read dir
}
impl From<&str> for FileIterator {
fn from(path: &str) -> Self {
FileIterator {
dirs: vec![PathBuf::from(path)],
files: None,
}
}
}
impl Iterator for FileIterator {
type Item = PathBuf;
fn next(&mut self) -> Option<PathBuf> {
loop {
while let Some(read_dir) = &mut self.files {
match read_dir.next() {
Some(Ok(entry)) => {
let path = entry.path();
if let Ok(md) = entry.metadata() {
if md.is_dir() {
self.dirs.push(path.clone());
continue;
}
}
return Some(path);
}
None => { // we consumed this directory
self.files = None;
break;
}
_ => { }
}
}
while let Some(dir) = self.dirs.pop() {
let read_dir = fs::read_dir(&dir);
if let Ok(files) = read_dir {
self.files = Some(files);
return Some(dir);
}
}
break; // no more files, no more dirs
}
return None;
}
}
playground
The advantage of writing your own iterator is that you'll tune it for your precise needs (sorting, filtering, error handling, etc.). But you'll have to deal with your own bugs.
This is the exact solution I sought. It's none of my achievement; see here. Comments are welcome.
fn traverse_flat_dst_iter(
src_dir: &PathBuf,
dst_step: Vec<PathBuf>,
) -> impl Iterator<Item = (PathBuf, PathBuf)> {
let (dirs, files) = list_dir_groom(src_dir);
let traverse = move |d: PathBuf| -> Box<dyn Iterator<Item = (PathBuf, PathBuf)>> {
let mut step = dst_step.clone();
step.push(PathBuf::from(d.file_name().unwrap()));
Box::new(traverse_flat_dst_iter(&d, step))
};
dirs.into_iter()
.flat_map(traverse)
.chain(files.into_iter().map(|f| (f, PathBuf::new())))
}
Another, more sophisticated take. One has to box things, clone parameters to be shared between lambdas, etc., to satisfy the compiler. Yet it works. Hopefully, on can get the hang of the thing.
fn traverse_dir(
src_dir: &PathBuf,
dst_step: Vec<PathBuf>,
) -> Box<dyn Iterator<Item = (PathBuf, Vec<PathBuf>)>> {
let (dirs, files) = groom(src_dir);
let destination_step = dst_step.clone(); // A clone for handle.
let traverse = move |d: PathBuf| {
let mut step = dst_step.clone();
step.push(PathBuf::from(d.file_name().unwrap()));
traverse_dir(&d, step)
};
let handle = move |f: PathBuf| (f, destination_step.clone());
if flag("r") {
// Chaining backwards.
Box::new(
files
.into_iter()
.map(handle)
.chain(dirs.into_iter().flat_map(traverse)),
)
} else {
Box::new(
dirs.into_iter()
.flat_map(traverse)
.chain(files.into_iter().map(handle)),
)
}
}

How to recursively call a closure that is stored in an Arc<Mutex<_>>?

I’m trying to transpile a dynamic language into Rust and closures are the most difficult part to implement.
I've tried using a Arc<Mutex<dyn FnMut>>, but it doesn't support recursion.
use std::sync::{Arc, Mutex};
type Data = Arc<DataUnpack>;
enum DataUnpack {
Number(f64),
Function(Box<Mutex<FnMut(Vec<Data>) -> Data>>),
}
fn call(f: Data, args: Vec<Data>) -> Data {
if let DataUnpack::Function(v) = &*f {
let f = &mut *v.lock().unwrap();
f(args)
} else {
panic!("TYPE ERR")
}
}
fn lambda(f: Box<FnMut(Vec<Data>) -> Data>) -> Data {
Arc::new(DataUnpack::Function(Box::new(Mutex::new(Box::leak(f)))))
}
fn main() {
let f: Arc<Mutex<Data>> = Arc::new(Mutex::new(Arc::new(DataUnpack::Number(0.0))));
*f.lock().unwrap() = {
let f = f.clone();
lambda(Box::new(move |xs| {
println!("Ha");
call(f.lock().unwrap().clone(), xs.clone())
}))
};
call(f.lock().unwrap().clone(), vec![]);
}
playground
It shows one Ha and then stops. Where am I wrong?

Raw Pointers not producing desired effects

#![feature(ptr_internals)]
use core::ptr::Unique;
struct PtrWrapper {
id: usize,
self_reference: Unique<Self>
}
impl PtrWrapper {
fn new() -> Self {
let dummy = unsafe {Unique::new_unchecked(std::ptr::null_mut::<PtrWrapper>())};
let mut ret = Self {id:0, self_reference: dummy };
let new_ptr = &mut ret as *mut Self;
debug_print(new_ptr);
ret.self_reference = Unique::new(new_ptr).unwrap();
debug_print(ret.self_reference.as_ptr());
ret
}
fn get_id(&self) -> usize {
self.id.clone()
}
}
fn main() {
println!("START");
let mut wrapper = PtrWrapper::new();
wrapper.id = 10;
let ptr = wrapper.self_reference.as_ptr();
unsafe {
(*ptr).id += 30;
println!("The next print isn't 40? Garbage bytes");
debug_print(ptr);
let tmp = &mut wrapper as *mut PtrWrapper;
(*tmp).id += 500;
println!("The next print isn't 540?");
debug_print(tmp);
}
println!("Below debug_print is proof of undefined behavior! Garbage bytes\n");
debug_print(wrapper.self_reference.as_ptr());
debug_print(&mut wrapper as *mut PtrWrapper);
debug_print_move(wrapper);
println!("Why is the assertion below false?");
assert_eq!(unsafe{(*ptr).id}, 540);
}
fn debug_print_move(mut wrapper: PtrWrapper) {
debug_print(&mut wrapper as *mut PtrWrapper);
}
fn debug_print(ptr: *mut PtrWrapper) {
println!("Address: {:p}", ptr);
println!("ID: {}\n", unsafe {(*ptr).get_id()});
}
The above code should compile fine in rust playground with a nightly selected version. Pay attention to the console outputs.
My question is: Why are the intermittent results not equal to the value I expect them to equal? In the case below, there is no multiple access simultaneously (single threaded), so there aren't any data races. There are, however, implicitly multiple mutable version of the object existing on the stack.
As expected, the memory location of the pointer changes with the tmp variable as well as when the entire object is moved into debug_print_move. It appears that using the tmp pointer works as expected (i.e., adds 500), however, the pointers which are obtained from the Unique<PtrWrapper> object seems to point to irrelevant locations in memory.
As Stargateur recommended, in order to solve this problem we need to Pin the object which needs to be self-referential. I ended up using:
pin-api = "0.2.1"
In cargo.toml instead of std::pin::pin. Next, I set this up the struct and its implementation:
#![feature(ptr_internals, pin_into_inner, optin_builtin_traits)]
// not available on rust-playground
extern crate pin_api;
use pin_api::{boxed::PinBox, marker::Unpin, mem::Pin};
///test
pub struct PtrWrapper<T>
where
T: std::fmt::Debug,
{
///tmp
pub obj: T,
/// pinned object
pub self_reference: *mut Self,
}
impl<T> !Unpin for PtrWrapper<T> where T: std::fmt::Debug {}
impl<T> PtrWrapper<T>
where
T: std::fmt::Debug,
{
///test
pub fn new(obj: T) -> Self {
Self {
obj,
self_reference: std::ptr::null_mut(),
}
}
///test
pub fn init(mut self: Pin<PtrWrapper<T>>) {
let mut this: &mut PtrWrapper<T> = unsafe { Pin::get_mut(&mut self) };
this.self_reference = this as *mut Self;
}
/// Debug print
pub fn print_obj(&self) {
println!("Obj value: {:#?}", self.obj);
}
}
Finally, the test function:
fn main2() {
unsafe {
println!("START");
let mut wrapper = PinBox::new(PtrWrapper::new(10));
wrapper.as_pin().init();
let m = wrapper.as_pin().self_reference;
(*m).obj += 30;
println!("The next print is 40");
debug_print(m);
let tmp = wrapper.as_pin().self_reference;
(*tmp).obj += 500;
println!("The next print is 540?");
debug_print(tmp);
debug_print(wrapper.self_reference);
let cpy = PinBox::get_mut(&mut wrapper);
debug_print_move(cpy);
std::mem::drop(wrapper);
println!("Works!");
assert_eq!(unsafe { (*m).obj }, 540);
}
}
fn debug_print_move<T>(mut wrapper: &mut PtrWrapper<T>)
where
T: std::fmt::Debug,
{
debug_print(&mut *wrapper as *mut PtrWrapper<T>);
}
fn debug_print<T>(ptr: *mut PtrWrapper<T>)
where
T: std::fmt::Debug,
{
println!("Address: {:p}", ptr);
unsafe { (*ptr).print_obj() };
}
On a side note, pin-api does not exist on rust playground. You could still use std::pin::Pin, however it would require further customization.

How do I process a range in slices in Rust?

I understand that the preferred way to iterate in Rust is through the for var in (range) syntax, but sometimes I'd like to work on more than one of the elements in that range at a time.
From a Ruby perspective, I'm trying to find a way of doing (1..100).each_slice(5) do |this_slice| in Rust.
I'm trying things like
for mut segment_start in (segment_size..max_val).step_by(segment_size) {
let this_segment = segment_start..(segment_start + segment_size).iter().take(segment_size);
}
but I keep getting errors that suggest I'm barking up the wrong type tree. The docs aren't helpful either--they just don't contain this use case.
What's the Rust way to do this?
Use chunks (or chunks_mut if you need mutability):
fn main() {
let things = [5, 4, 3, 2, 1];
for slice in things.chunks(2) {
println!("{:?}", slice);
}
}
Outputs:
[5, 4]
[3, 2]
[1]
The easiest way to combine this with a Range would be to collect the range to a Vec first (which dereferences to a slice):
fn main() {
let things: Vec<_> = (1..100).collect();
for slice in things.chunks(5) {
println!("{:?}", slice);
}
}
Another solution that is pure-iterator would be to use Itertools::chunks_lazy:
extern crate itertools;
use itertools::Itertools;
fn main() {
for chunk in &(1..100).chunks_lazy(5) {
for val in chunk {
print!("{}, ", val);
}
println!("");
}
}
Which suggests a similar solution that only requires the standard library:
fn main() {
let mut range = (1..100).peekable();
while range.peek().is_some() {
for value in range.by_ref().take(5) {
print!("{}, ", value);
}
println!("");
}
}
One trick is that Ruby and Rust have different handling here, mostly centered around efficiency.
In Ruby Enumerable can create new arrays to stuff values in without worrying about ownership and return a new array each time (check with this_slice.object_id).
In Rust, allocating a new vector each time would be pretty unusual. Additionally, you can't easily return a reference to a vector that the iterator holds due to complicated lifetime concerns.
A solution that's very similar to Ruby's is:
fn main() {
let mut range = (1..100).peekable();
while range.peek().is_some() {
let chunk: Vec<_> = range.by_ref().take(5).collect();
println!("{:?}", chunk);
}
}
Which could be wrapped up in a new iterator that hides the details:
use std::iter::Peekable;
struct InefficientChunks<I>
where I: Iterator
{
iter: Peekable<I>,
size: usize,
}
impl<I> Iterator for InefficientChunks<I>
where I: Iterator
{
type Item = Vec<I::Item>;
fn next(&mut self) -> Option<Self::Item> {
if self.iter.peek().is_some() {
Some(self.iter.by_ref().take(self.size).collect())
} else {
None
}
}
}
trait Awesome: Iterator + Sized {
fn inefficient_chunks(self, size: usize) -> InefficientChunks<Self> {
InefficientChunks {
iter: self.peekable(),
size: size,
}
}
}
impl<I> Awesome for I where I: Iterator {}
fn main() {
for chunk in (1..100).inefficient_chunks(5) {
println!("{:?}", chunk);
}
}
Collecting into a vec can easily kill your performance. An approach similar to in the question is perfectly fine.
fn chunk_range(range: Range<usize>, chunk_size: usize) -> impl Iterator<Item=Range<usize>> {
range.clone().step_by(chunk_size).map(move |block_start| {
let block_end = (block_start + chunk_size).min(range.end);
block_start..block_end
})
}

Resources