I am quite new to Rust and have some problems with async and lifetimes.
I am trying to read a USB cam in a loop using rscam crate.
The problem is that after some time the camera driver tends to freeze and returns no data.
Rscam's capture has no builtin timeout (at least the official release) - so I am trying to wrap it in tokio's timeout.
Because all runs in a loop I do not move the camera struct, but pass a reference. And here is my difficulty, as sending into future requires a static lifetime (which I cannot figure out how to do).
Or maybe there is a completely different solution to adding timeouts for blocking io?
let mut camera = Camera::new(&settings.device).unwrap();
camera.set_control(CID_JPEG_COMPRESSION_QUALITY, &95);
camera.start(&Config {
interval: (1, 30),
resolution: (settings.width, settings.height),
format: settings.format.as_bytes(),
..Default::default()
}).unwrap();
let cam_ref = &camera;
let duration = time::Duration::from_millis(settings.timeout);
loop {
let loop_start = time::Instant::now();
let frame;
let future = task::spawn_blocking(|| {
cam_ref.capture()
});
if let Ok(result) = tokio::time::timeout(duration, future).await {
frame = result.unwrap();
} else {
println!("Restarting camera");
break;
}
let buf = decode_jpeg(&frame.unwrap());
// etc
}
Related
I would like to split a stream in Rust, but I'd like to do it idiomatically without side effects in my stream functions.
This is what I have now where I pipe the stream result into a for_each
let (sender, receiver) tokio::sync::broadcast::channel(1);
let mut base_stream = inf_async_stream_gen().await
.map(function1)
.for_each(|x| {
let inner_sender = sender.clone();
async move { inner_sender.send(x); }
});
let stream_1 = BroadcastStream::new(receiver.resubscribe());
let stream_2 = BroadcastStream::new(receiver.resubscribe());
But this feels a little bit odd. Isn't receiver conceptually a Sink (despite not implementing the trait)?
Consider this pseudocode:
let k = 10;
let ptr = &k as *const k;
println!("{:p}", ptr); // prints address of pointer
let addr = format!("{:p}", ptr);
super-unsafe {
// this would obviously be super unsafe. It may even cause a STATUS_ACCESS_VIOLATION if you try getting memory from a page that the OS didn't allocate to the program!
let ptr_gen = PointerFactory::from_str(addr.as_str());
assert_eq!(k, *ptr_gen);
}
The pseudocode gets the idea across: I want to be able to get a pointer to a certain memory address by its &str representation. Is this... possible?
So essentially what you want to do is parse the string back to an integer (usize) and then interpret that value as a pointer/reference†:
fn main()
{
let i = 12i32;
let r = format!("{:p}", &i);
let x = unsafe
{
let r = r.trim_start_matches("0x");
&*(usize::from_str_radix(&r, 16).unwrap() as *const i32)
};
println!("{}", x);
}
You can try this yourself in the playground.
†As you can see, you don't even need to cast your reference into a raw pointer, the {:p} formatter takes care of representing it as a memory location (index).
Update: As E_net4 mentioned this in the comment section, it is better to use usize here, which is architecture defined unlike the machine sized one. The transmute was not necessary, so I removed it. The third point about undefined behaviour however seems obvious to whomever tries to do something like the above. This answer provides a way to achieve what the OP asked for which doesn't mean this should be used for anything else than academic/experimental purposes :)
I am trying to make a echo server that capitalize a String when it replies, to practice with tokio as an exercise. I used an array as a buffer which is annoying because what if the string overflows the buffer?
I would like to know if there is a better way to this without using an array, ideally just using a String or a vector without needing to create the buffer array.
I tried read_from_string() but is not async and ends up blocking the socket.
extern crate tokio;
use tokio::net::TcpListener;
use tokio::prelude::*;
fn main() {
let addr = "127.0.0.1:6142".parse().unwrap();
let listener = TcpListener::bind(&addr).unwrap();
let server = listener
.incoming()
.for_each(|socket| {
let (mut reader, mut writer) = socket.split();
let mut buffer = [0; 16];
reader.poll_read(&mut buffer)?;
let s = std::str::from_utf8(&buffer).unwrap();
s.to_uppercase();
writer.poll_write(&mut s.as_bytes())?;
Ok(())
})
.map_err(|e| {
eprintln!("something went wrong {}", e);
});
tokio::run(server);
}
Results:
"012345678901234567890" becomes -> "0123456789012345"
I could increase the buffer of course but it would just kick the can down the road.
I believe tokio_codec is a right tool for such tasks. Tokio documentation: https://tokio.rs/docs/going-deeper/frames/
It uses Bytes / BytesMut as its buffer - very powerful structure which will allow you to process your data however you want and avoid unnecessary copies
This question already has answers here:
How to read subprocess output asynchronously
(2 answers)
How do I read the output of a child process without blocking in Rust?
(4 answers)
Closed 3 years ago.
I have a subprocess, which may or may not write something to it's stdout in a specific amount of time, e.g. 3 seconds.
If a new line in the subprocess stdout starts with the correct thing, I want to return the line.
Optimally I would like to realize something like this:
use std::io::{BufRead, BufReader};
use std::thread;
use std::time::Duration;
pub fn wait_for_or_exit(
reader: &BufReader<&mut std::process::ChildStdout>,
wait_time: u64,
cmd: &str,
) -> Option<String> {
let signal: Arc<AtomicBool> = Arc::new(AtomicBool::new(false));
let signal_clone = signal.clone();
let child = thread::spawn(move || {
thread::sleep(Duration::from_millis(wait_time));
signal_clone.store(true, Ordering::Relaxed);
});
let mut line = String::new();
while !signal.load(Ordering::Relaxed) {
//Sleep a really small amount of time not to block cpu
thread::sleep(Duration::from_millis(10));
//This line is obviously invalid!
if reader.has_input() {
line.clear();
reader.read_line(&mut line).unwrap();
if line.starts_with(cmd) {
return Some(line);
}
}
}
None
}
The only line not working here is reader.has_input().
Obviously, if the subprocess answers much faster than the wait_time for a repeated amount of times, there will be a lot of sleeping threads, but I can take care of that with channels.
There are two approaches.
You can spin up a separate thread, and then use some mechanism (probably a channel) to signal success or failure to your waiting thread.
You can use async IO as you mentioned, such as the futures and tokio lib.
I'll demo both. I prefer the futures/Tokio approach, but if you're not familiar with the futures model, then option one might be better.
The Rust stdlib has a Channels API, and this channel actually features a recv_timeout which can help us out quite a bit.
use std::thread;
use std::time::Duration;
use std::sync::mpsc;
// this spins up a separate thread in which to wait for stuff to read
// from the BufReader<ChildStdout>
// If we successfully read, we send the string over the Channel.
// Back in the original thread, we wait for an answer over the channel
// or timeout in wait_time secs.
pub fn wait_for_or_exit(
reader: &BufReader<&mut std::process::ChildStdout>,
wait_time: u64,
cmd: &str,
) -> Option<String> {
let (sender, receiver) = mpsc::channel();
thread::spawn(move || {
let line = reader.read_line();
sender.send(line);
});
match receiver.recv_timeout(Duration::from_secs(wait_time)) {
Ok(line) => if line.starts_with(cmd)
{ Some(line) } else
{ None },
Err(mpsc::RecvTimeoutError::Timeout) => None,
Err(mpsc::RecvTimeoutError::Disconnected) => None
}
}
Option two assumes that you're building a future's based app. In order to accomplish what you want using Async IO is a file descriptor that will let us set NON_BLOCKING. Luckily we don't have to do that ourselves. The Futures and Tokio APIs handle this nicely. The trade-off, is that you have to compose your code out of non-blocking futures.
The code below was taken almost entirely from Tokio Process with a Futures timeout that comes from the Tokio API.
extern crate futures;
extern crate tokio;
extern crate tokio_process;
use std::process::Command;
use std::time::{Duration};
use futures::Future;
use tokio_process::CommandExt;
use tokio::prelude::*;
const TIMEOUT_SECS: u64 = 3;
fn main() {
// Like above, but use `output_async` which returns a future instead of
// immediately returning the `Child`.
let output = Command::new("echo").arg("hello").arg("world")
.output_async();
let future = output.map_err(|e| panic!("failed to collect output: {}", e))
.map(|output| {
assert!(output.status.success());
assert_eq!(output.stdout, b"hello world\n");
println!("received output: {}", String::from_utf8(output.stdout).unwrap());
})
.timeout(Duration::from_secs(TIMEOUT_SECS)) // here is where we say we only want to wait TIMETOUT seconds
.map_err(|_e| { println!("Timed out waiting for data"); });
tokio::run(future);
}
I am using tokio to implement a server which communicates with messages serialized with serde (bincode). Without asynchronous and futures I would do
extern crate tokio_io;
extern crate bincode;
extern crate serde;
extern crate bytes;
extern crate futures;
#[macro_use] extern crate serde_derive;
use tokio_io::{AsyncRead, AsyncWrite};
use tokio_io::io::{read_exact, write_all};
use bincode::{serialize, deserialize, deserialize_from, Infinite, serialized_size};
use std::io::Read;
use std::io::Cursor;
use futures::future::Future;
type Item = String; // Dummy, this is a complex struct with derived Serizalize
type Error = bincode::Error;
// This works
fn decode<R>(reader: &mut R) -> Result<Item, Error> where R: Read {
let message: Item = deserialize_from(reader, Infinite)?;
Ok(message)
}
fn main() {
let ser = serialize("Test", Infinite).unwrap();
let buf = Cursor::new(ser);
let mut reader = std::io::BufReader::new(buf);
println!("{:?}", decode(&mut reader))
}
But what I need is a decode function which can work with an asyncronous socket as
// I need this since I get the reader from a (tokio) socket as
// let socket = TcpListener::bind(&addr, &handle).unwrap();
// let (reader, writer) = socket.split();
fn decode_async<R>(reader: R) -> Result<Item, Error> where R: AsyncRead {
// Does not work:
let message: Item = deserialize_from(reader, Infinite)?;
Ok(message)
}
The only idea I have is to manually write the length into the buffer during encoding and then use read_exact:
// Encode with size
fn encode_async(item: &Item) -> Result<Vec<u8>, Error>{
let size = serialized_size(item);
let mut buf = serialize(&size, Infinite).unwrap();
let ser = serialize(item, Infinite).unwrap();
buf.extend(ser);
Ok(buf)
}
// Decode with size
fn decode_async<R>(reader: R) -> Box<Future<Item = Item, Error = std::io::Error>>
where R: AsyncRead + 'static {
let read = read_exact(reader, vec![0u8; 8]).and_then(|(reader, buf)| {
let size = deserialize::<u64>(&mut &buf[..]).unwrap();
Ok((reader, size as usize))
}).and_then(|(reader, size)| {
read_exact(reader, vec![0u8; size])
}).and_then(|(reader, buf)| {
let item = deserialize(&mut &buf[..]).unwrap();
Ok(item)
});
Box::new(read)
}
fn main() {
let ser = encode_async(&String::from("Test")).unwrap();
let buf = Cursor::new(ser);
let mut reader = std::io::BufReader::new(buf);
let dec = decode_async(reader).wait();
println!("{:?}", dec)
}
Is there a better way to implement the decoding?
deserialize_from can't handle IO errors, especially not of the kind WouldBlock which is returned by async (non-blocking) Readers when they are waiting for more data. That is limited by the interface: deserialize_from doesn't return a Future or a partial state, it returns the full decoded Result and wouldn't know how to combine the Reader with an event loop to handle WouldBlock without busy looping.
Theoretically, it is possible to implement an async_deserialize_from, but not by using the interfaces provided by serde unless you read the full data to decode in advance, which would defeat the purpose.
You need to read the full data using tokio_io::io::read_to_end or tokio_io::io::read_exact (what you're currently using), if you know the size of the encoded data in an "endless" stream (or in a stream followed by other data).
Stefan's answer is correct, however you might be interested in looking at the tokio-serde-* family of crates which do this for you, specifically tokio-serde-bincode. From the readme:
Utilities needed to easily implement a Tokio Bincode transport using serde for serialization and deserialization of frame values. Based on tokio-serde.
The crate has several examples of how to use it.