Related
This question already has answers here:
How to use async/await in Rust when you can't make main function async
(4 answers)
How do I synchronously return a value calculated in an asynchronous Future?
(3 answers)
Closed 3 months ago.
I have a program that does various simple things based on user selection.
fn main() {
let mut user_input = String::new(); // Initialize variable to store user input
println!("Select an option:\r\n[1] Get sysinfo\r\n[2] Read/Write File\r\n[3] Download file\r\n[4] Exit"); // Print options
io::stdin().read_line(&mut user_input).expect("You entered something weird you donkey!"); // Get user input and store in variable
let int_input = user_input.trim().parse::<i32>().unwrap(); // Convert user input to int (i32 means signed integer 32 bits)
match int_input { // If Else statement
1 => getsysinfo(), // If int_input == 1, call getsysinfo()
2 => readwritefile(),
3 => downloadfile(),
4 => process::exit(1), // end program
_ => println!("You didn't choose one of the given options, you donkey!") // input validation
}
}
My function downloadfile() looks like this, referenced from the Rust cookbook on downloading files.
error_chain! {
foreign_links {
Io(std::io::Error);
HttpRequest(reqwest::Error);
}
}
async fn downloadfile() -> Result<()> {
let tmp_dir = Builder::new().prefix("example").tempdir()?;
let target = "localhost:8000/downloaded.txt";
let response = reqwest::get(target).await?;
let mut dest = {
let fname = response
.url()
.path_segments()
.and_then(|segments| segments.last())
.and_then(|name| if name.is_empty() { None } else { Some(name) })
.unwrap_or("tmp.bin");
println!("File to download: {}", fname);
let fname = tmp_dir.path().join(fname);
println!("Will be located under: {:?}", fname);
File::create(fname)?
};
let content = response.text().await?;
copy(&mut content.as_bytes(), &mut dest)?;
Ok(())
}
I get the following error:
`match` arms have incompatible types
expected unit type `()`
found opaque type `impl Future<Output = std::result::Result<(), Error>>`
I presume its because the async function returns a Future type, so how can I make this code work?
You need to use the block_on function.
Add futures as a dependency in your cargo.toml for the following example to work.
use futures::executor::block_on;
async fn hello() -> String {
return String::from("Hello world!");
}
fn main() {
let output = block_on(hello());
println!("{output}");
}
I want to write a server using the current master branch of Hyper that saves a message that is delivered by a POST request and sends this message to every incoming GET request.
I have this, mostly copied from the Hyper examples directory:
extern crate futures;
extern crate hyper;
extern crate pretty_env_logger;
use futures::future::FutureResult;
use hyper::{Get, Post, StatusCode};
use hyper::header::{ContentLength};
use hyper::server::{Http, Service, Request, Response};
use futures::Stream;
struct Echo {
data: Vec<u8>,
}
impl Echo {
fn new() -> Self {
Echo {
data: "text".into(),
}
}
}
impl Service for Echo {
type Request = Request;
type Response = Response;
type Error = hyper::Error;
type Future = FutureResult<Response, hyper::Error>;
fn call(&self, req: Self::Request) -> Self::Future {
let resp = match (req.method(), req.path()) {
(&Get, "/") | (&Get, "/echo") => {
Response::new()
.with_header(ContentLength(self.data.len() as u64))
.with_body(self.data.clone())
},
(&Post, "/") => {
//self.data.clear(); // argh. &self is not mutable :(
// even if it was mutable... how to put the entire body into it?
//req.body().fold(...) ?
let mut res = Response::new();
if let Some(len) = req.headers().get::<ContentLength>() {
res.headers_mut().set(ContentLength(0));
}
res.with_body(req.body())
},
_ => {
Response::new()
.with_status(StatusCode::NotFound)
}
};
futures::future::ok(resp)
}
}
fn main() {
pretty_env_logger::init().unwrap();
let addr = "127.0.0.1:12346".parse().unwrap();
let server = Http::new().bind(&addr, || Ok(Echo::new())).unwrap();
println!("Listening on http://{} with 1 thread.", server.local_addr().unwrap());
server.run().unwrap();
}
How do I turn the req.body() (which seems to be a Stream of Chunks) into a Vec<u8>? I assume I must somehow return a Future that consumes the Stream and turns it into a single Vec<u8>, maybe with fold(). But I have no clue how to do that.
Hyper 0.13 provides a body::to_bytes function for this purpose.
use hyper::body;
use hyper::{Body, Response};
pub async fn read_response_body(res: Response<Body>) -> Result<String, hyper::Error> {
let bytes = body::to_bytes(res.into_body()).await?;
Ok(String::from_utf8(bytes.to_vec()).expect("response was not valid utf-8"))
}
I'm going to simplify the problem to just return the total number of bytes, instead of echoing the entire stream.
Futures 0.3
Hyper 0.13 + TryStreamExt::try_fold
See euclio's answer about hyper::body::to_bytes if you just want all the data as one giant blob.
Accessing the stream allows for more fine-grained control:
use futures::TryStreamExt; // 0.3.7
use hyper::{server::Server, service, Body, Method, Request, Response}; // 0.13.9
use std::convert::Infallible;
use tokio; // 0.2.22
#[tokio::main]
async fn main() {
let addr = "127.0.0.1:12346".parse().expect("Unable to parse address");
let server = Server::bind(&addr).serve(service::make_service_fn(|_conn| async {
Ok::<_, Infallible>(service::service_fn(echo))
}));
println!("Listening on http://{}.", server.local_addr());
if let Err(e) = server.await {
eprintln!("Error: {}", e);
}
}
async fn echo(req: Request<Body>) -> Result<Response<Body>, hyper::Error> {
let (parts, body) = req.into_parts();
match (parts.method, parts.uri.path()) {
(Method::POST, "/") => {
let entire_body = body
.try_fold(Vec::new(), |mut data, chunk| async move {
data.extend_from_slice(&chunk);
Ok(data)
})
.await;
entire_body.map(|body| {
let body = Body::from(format!("Read {} bytes", body.len()));
Response::new(body)
})
}
_ => {
let body = Body::from("Can only POST to /");
Ok(Response::new(body))
}
}
}
Unfortunately, the current implementation of Bytes is no longer compatible with TryStreamExt::try_concat, so we have to switch back to a fold.
Futures 0.1
hyper 0.12 + Stream::concat2
Since futures 0.1.14, you can use Stream::concat2 to stick together all the data into one:
fn concat2(self) -> Concat2<Self>
where
Self: Sized,
Self::Item: Extend<<Self::Item as IntoIterator>::Item> + IntoIterator + Default,
use futures::{
future::{self, Either},
Future, Stream,
}; // 0.1.25
use hyper::{server::Server, service, Body, Method, Request, Response}; // 0.12.20
use tokio; // 0.1.14
fn main() {
let addr = "127.0.0.1:12346".parse().expect("Unable to parse address");
let server = Server::bind(&addr).serve(|| service::service_fn(echo));
println!("Listening on http://{}.", server.local_addr());
let server = server.map_err(|e| eprintln!("Error: {}", e));
tokio::run(server);
}
fn echo(req: Request<Body>) -> impl Future<Item = Response<Body>, Error = hyper::Error> {
let (parts, body) = req.into_parts();
match (parts.method, parts.uri.path()) {
(Method::POST, "/") => {
let entire_body = body.concat2();
let resp = entire_body.map(|body| {
let body = Body::from(format!("Read {} bytes", body.len()));
Response::new(body)
});
Either::A(resp)
}
_ => {
let body = Body::from("Can only POST to /");
let resp = future::ok(Response::new(body));
Either::B(resp)
}
}
}
You could also convert the Bytes into a Vec<u8> via entire_body.to_vec() and then convert that to a String.
See also:
How do I convert a Vector of bytes (u8) to a string
hyper 0.11 + Stream::fold
Similar to Iterator::fold, Stream::fold takes an accumulator (called init) and a function that operates on the accumulator and an item from the stream. The result of the function must be another future with the same error type as the original. The total result is itself a future.
fn fold<F, T, Fut>(self, init: T, f: F) -> Fold<Self, F, Fut, T>
where
F: FnMut(T, Self::Item) -> Fut,
Fut: IntoFuture<Item = T>,
Self::Error: From<Fut::Error>,
Self: Sized,
We can use a Vec as the accumulator. Body's Stream implementation returns a Chunk. This implements Deref<[u8]>, so we can use that to append each chunk's data to the Vec.
extern crate futures; // 0.1.23
extern crate hyper; // 0.11.27
use futures::{Future, Stream};
use hyper::{
server::{Http, Request, Response, Service}, Post,
};
fn main() {
let addr = "127.0.0.1:12346".parse().unwrap();
let server = Http::new().bind(&addr, || Ok(Echo)).unwrap();
println!(
"Listening on http://{} with 1 thread.",
server.local_addr().unwrap()
);
server.run().unwrap();
}
struct Echo;
impl Service for Echo {
type Request = Request;
type Response = Response;
type Error = hyper::Error;
type Future = Box<futures::Future<Item = Response, Error = Self::Error>>;
fn call(&self, req: Self::Request) -> Self::Future {
match (req.method(), req.path()) {
(&Post, "/") => {
let f = req.body()
.fold(Vec::new(), |mut acc, chunk| {
acc.extend_from_slice(&*chunk);
futures::future::ok::<_, Self::Error>(acc)
})
.map(|body| Response::new().with_body(format!("Read {} bytes", body.len())));
Box::new(f)
}
_ => panic!("Nope"),
}
}
}
You could also convert the Vec<u8> body to a String.
See also:
How do I convert a Vector of bytes (u8) to a string
Output
When called from the command line, we can see the result:
$ curl -X POST --data hello http://127.0.0.1:12346/
Read 5 bytes
Warning
All of these solutions allow a malicious end user to POST an infinitely sized file, which would cause the machine to run out of memory. Depending on the intended use, you may wish to establish some kind of cap on the number of bytes read, potentially writing to the filesystem at some breakpoint.
See also:
How do I apply a limit to the number of bytes read by futures::Stream::concat2?
Most of the answers on this topic are outdated or overly complicated. The solution is pretty simple:
/*
WARNING for beginners!!! This use statement
is important so we can later use .data() method!!!
*/
use hyper::body::HttpBody;
let my_vector: Vec<u8> = request.into_body().data().await.unwrap().unwrap().to_vec();
let my_string = String::from_utf8(my_vector).unwrap();
You can also use body::to_bytes as #euclio answered. Both approaches are straight-forward! Don't forget to handle unwrap properly.
Using warp.rs 0.2.2, let's consider a basic web service with one route for GET /:
#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
let getRoot = warp::get().and(warp::path::end()).and_then(routes::getRoot);
warp::serve(getRoot).run(([0, 0, 0, 0], 3030)).await;
Ok(())
}
My goal is to use ? for error handling in the route handlers, so let's write one that can error and return early in crate::routes:
use crate::errors::ServiceError;
use url::Url;
pub async fn getRoot() -> Result<impl warp::Reply, warp::Rejection> {
let _parsed_url = Url::parse(&"https://whydoesn.it/work?").map_err(ServiceError::from)?;
Ok("Hello world !")
}
This version works.
Here the error that's returned by Url::parse() is a url::ParseError
To convert between error types, from url::ParseError to ServiceError, then from ServiceError to warp::Rejection, I've written some error helpers in crate::errors:
#[derive(thiserror::Error, Debug)]
pub enum ServiceError {
#[error(transparent)]
Other(#[from] anyhow::Error), // source and Display delegate to anyhow::Error
}
impl warp::reject::Reject for ServiceError {}
impl From<ServiceError> for warp::reject::Rejection {
fn from(e: ServiceError) -> Self {
warp::reject::custom(e)
}
}
impl From<url::ParseError> for ServiceError {
fn from(e: url::ParseError) -> Self {
ServiceError::Other(e.into())
}
}
Now, the above works, and I'm trying to shorten the second code block to use ? for error handling directly, and convert automatically from the underlying error (here url::ParseError) to a warp::Rejection.
Here's what I've tried:
use crate::errors::ServiceError;
use url::Url;
pub async fn getRoot() -> Result<impl warp::Reply, ServiceError> {
let _parsed_url = Url::parse(&"https://whydoesn.it/work?")?;
Ok("Hello world !")
}
The url::ParseError returned by Url::Parse will convert fine into a ServiceError to return, but returning a ServiceError from my handler doesn't work.
The first compilation error I get is:
error[E0277]: the trait bound `errors::ServiceError: warp::reject::sealed::CombineRejection<warp::reject::Rejection>` is not satisfied
--> src/main.rs:102:54
|
102 | let getRoot = warp::get().and(warp::path::end()).and_then(routes::getRoot);
| ^^^^^^^^ the trait `warp::reject::sealed::CombineRejection<warp::reject::Rejection>` is not implemented for `errors::ServiceError`
Is there a way I can keep the short error handling using ? only and either:
make ServiceError implement warp::reject::sealed::CombineRejection<warp::reject::Rejection> ?
work around that ?
You can implement From to convert your error type into warp::Rejection using reject::custom. Rejection encapsulates custom types which you can later choose to inspect inside of a recover handler.
This example uses a plain error struct, but if you have an error enum you can match on the variants inside the recovery handler and perform different logic as needed.
use serde::Deserialize;
use snafu::{ensure, Snafu};
use std::convert::Infallible;
use warp::{
filters::{any, query, BoxedFilter},
http::StatusCode,
reject::Reject,
Filter, Rejection, Reply,
};
// A normal error type, created by SNAFU
#[derive(Debug, Snafu)]
#[snafu(display("Expected a value less than 10, but it was {}", value))]
struct LessThanTenError {
value: i32,
}
// A function that might fail
fn validate(value: i32) -> Result<i32, LessThanTenError> {
ensure!(value < 10, LessThanTenContext { value });
Ok(value)
}
// We need a custom type to later extract from the `Rejection`. In
// this case, we can reuse the error type itself.
impl Reject for LessThanTenError {}
// To allow using `?`, we implement a conversion from our error to
// `Rejection`
impl From<LessThanTenError> for Rejection {
fn from(other: LessThanTenError) -> Self {
warp::reject::custom(other)
}
}
#[tokio::main]
async fn main() {
let api = simple_math().recover(report_invalid);
let p: std::net::SocketAddr = "0.0.0.0:8888".parse().unwrap();
warp::serve(api).run(p).await;
}
#[derive(Debug, Deserialize)]
struct QueryParams {
a: i32,
b: i32,
}
fn simple_math() -> BoxedFilter<(impl Reply,)> {
any::any()
.and(query::query())
.and_then(|args: QueryParams| async move {
// Look at us using those question marks!
let a = validate(args.a)?;
let b = validate(args.b)?;
let sum = validate(a + b)?;
// We specify that we are returning an error type of
// `Rejection`, which allows the compiler to know what
// type to convert to when using `?` here.
Ok::<_, Rejection>(format!("The sum is {}", sum))
})
.boxed()
}
async fn report_invalid(r: Rejection) -> Result<impl Reply, Infallible> {
if let Some(e) = r.find::<LessThanTenError>() {
// It was our specific error type, do whatever we want. We
// will just print out the error text.
Ok(warp::reply::with_status(
e.to_string(),
StatusCode::BAD_REQUEST,
))
} else {
// Do prettier error reporting for the default error here.
Ok(warp::reply::with_status(
String::from("Something bad happened"),
StatusCode::INTERNAL_SERVER_ERROR,
))
}
}
[dependencies]
serde = { version = "1.0.118", features = ["derive"] }
snafu = "0.6.10"
tokio = { version = "0.2.23", features = ["full"] }
warp = "0.2.5"
% curl 'http://127.0.0.1:8888'
< HTTP/1.1 500 Internal Server Error
Something bad happened
% curl -v 'http://127.0.0.1:8888?a=1&b=2'
< HTTP/1.1 200 OK
The sum is 3
% curl -v 'http://127.0.0.1:8888?a=6&b=5'
< HTTP/1.1 400 Bad Request
Expected a value less than 10, but it was 11
See also:
Is there a way to do validation as part of a filter in Warp?
When should I implement std::convert::From vs std::convert::Into?
How do you define custom `Error` types in Rust?
From my findings, there are two solutions.
Abandon ? in favor of your own macro that constructs and returns a response if there is an error.
Use PR #458 by cjbassi instead of the mainline release by:
Implementing warp::reply::Reply on your error type so that it converts into the correct user facing error message.
Replace warp = "0.2" with warp = { git = "https://github.com/cjbassi/warp.git", branch = "error"} in your Cargo.toml file
use .map_async instead of .and_then for handlers
I have a function in Rust using try! that attempts to collect all files in a directory recursively and insert them into a vector. Because the function uses try! to check errors, the compiler seems to expect an io::Result return from the function, and doesn't let me include the vector because the try! macro only returns a result. I need the vector to be returned.
Code is as follows:
mod os{
use std::io;
use std::fs::{self, DirEntry};
//use std::fs;
use std::path::Path;
// one possible implementation of walking a directory only visiting files
pub fn visit_dirs(dir: &Path, cb: &Fn(&DirEntry)) -> (io::Result<()>,Vec<String>) {
let mut filevec: Vec<String> = Vec::new();
if try!(fs::metadata(dir)).is_dir() {
for entry in try!(fs::read_dir(dir)) {
let entry = try!(entry);
if try!(fs::metadata(entry.path())).is_dir() {
try!(visit_dirs(&entry.path(), cb));
} else {
cb(&entry);
}
}
}
(Ok(()),filevec)
}
fn push_path_to_vec(p:&DirEntry,v:Vec<String>){
v.push(p.path().to_str().unwrap().to_string());
}}
Here is the error:
<std macros>:5:8: 6:42 error: mismatched types:
expected `(core::result::Result<(), std::io::error::Error>, collections::vec::Vec<collections::string::String>)`
found `core::result::Result<_, _>`
(expected tuple,
found enum `core::result::Result`) [E0308]
I wonder if there's any idiomatic way to do this that I've missed.
The return type of visit_dirs is wrong. The function should return a Result, but right now it returns a tuple. Since try! only works for functions returning a Result, your code doesn't compile. You can change the return value of visit_dirs in order to fix it:
pub fn visit_dirs(dir: &Path, cb: &Fn(&DirEntry)) -> io::Result<Vec<String>>
The new definition means that a Vec<String> will be stored in the Result upon success. With some minor tweaks, the code is accepted by the compiler (see below)
mod os{
use std::io;
use std::fs::{self, DirEntry};
//use std::fs;
use std::path::Path;
// one possible implementation of walking a directory only visiting files
pub fn visit_dirs(dir: &Path, cb: &Fn(&DirEntry)) -> io::Result<Vec<String>> {
let mut filevec: Vec<String> = Vec::new();
if try!(fs::metadata(dir)).is_dir() {
for entry in try!(fs::read_dir(dir)) {
let entry = try!(entry);
if try!(fs::metadata(entry.path())).is_dir() {
try!(visit_dirs(&entry.path(), cb));
} else {
cb(&entry);
}
}
}
Ok(filevec)
}
fn push_path_to_vec(p:&DirEntry,mut v:Vec<String>){
v.push(p.path().to_str().unwrap().to_string());
}}
I am trying to write the function set which calls the Rust LMDB library (docs), and an example I'm working off of.
I can't for the life of me get this to work. Here is my current attempt:
fn main() {
let env = getenv("duperdb");
let dbhandle = get_dbhandle("", &env);
let txn = new_transaction(&env);
let vec = vec![("foo", "another text"), ("bar", "and another")];
set(&dbhandle, &env, &vec);
let reader = env.get_reader().unwrap();
let db = reader.bind(&dbhandle);
let note = db.get::<&str>("foo").unwrap();
println!("NOTE: {}", note);
}
Where set is defined as:
pub fn set<A: ToMdbValue, B: ToMdbValue>(
handle: &DbHandle,
env: &Environment,
pairs: &Vec<(&A, &B)>) -> () {
let txn = new_transaction(&env);
{
let db = txn.bind(&handle);
for &(id, note) in pairs.iter() {
db.set(&id, ¬e).unwrap();
}
}
match txn.commit() {
Err(_) => panic!("Failed to commit!"),
Ok(_) => (),
}
}
This spits out the following error:
src/db/wrapper.rs:28:20: 28:23 error: the trait `lmdb::traits::ToMdbValue` is not implemented for the type `&A` [E0277]
src/db/wrapper.rs:28 db.set(&id, ¬e).unwrap();
^~~
I also tried db.set(id, note).unwrap();, but this time I get:
src/main.rs:13:5: 13:8 error: the trait `core::marker::Sized` is not implemented for the type `str` [E0277]
src/main.rs:13 set(&dbhandle, &env, &vec);
^~~
src/main.rs:13:5: 13:8 help: run `rustc --explain E0277` to see a detailed explanation
src/main.rs:13:5: 13:8 note: `str` does not have a constant size known at compile-time
src/main.rs:13:5: 13:8 note: required by `dupernote::db::wrapper::set`
src/main.rs:13:5: 13:8 error: the trait `lmdb_rs::traits::ToMdbValue` is not implemented for the type `str` [E0277]
src/main.rs:13 set(&dbhandle, &env, &vec);
^~~
I also tried stuff like:
for (id, note) in pairs.iter() {
db.set(id, note).unwrap();
}
But that doesn't work either... I don't fully understand why. Doesn't id and note have type &str, not str?
Here's an MCVE of your problem:
trait Example {}
impl Example for i32 {}
fn library_call<T>(value: T)
where T: Example,
{}
fn user_call<T>(values: &[T])
where T: Example,
{
for i in values {
library_call(i);
}
}
fn main() {
let values = vec![1, 2, 3];
user_call(&values);
}
With the error:
error: the trait `Example` is not implemented for the type `&T` [E0277]
library_call(i);
^~~~~~~~~~~~
The error message is exactly correct - Example is not implemented for &T, it's only guaranteed to be implemented for T. &T and T are different types.
Instead, you need to indicate that a reference to the generic type implements the trait you need:
fn user_call<T>(values: &[T])
where for <'a> &'a T: Example,
And then you need to make sure that a reference to the concrete type actually implements the trait:
impl<'a> Example for &'a i32 {}
Or a broader version:
impl<'a, T> Example for &'a T
where T: Example
{}
See also When should I not implement a trait for references to implementors of that trait?
The definition of the function that gives you an error (if I'm reading the docs right):
fn set(&self, key: &ToMdbValue, value: &ToMdbValue) -> MdbResult<()>
key must be a reference to a trait object. You are trying to pass a reference to a generic type implmementing ToMdbValue.
https://doc.rust-lang.org/book/trait-objects.html
I can't verify but this should work:
pub fn set(handle: &DbHandle, env: &Environment, pairs: &Vec<(&ToMdbValue, &ToMdbValue)>) -> () {
let txn = new_transaction(&env);
{
let db = txn.bind(&handle);
for &(id, note) in pairs.iter() {
db.set(id, note).unwrap();
}
}
match txn.commit() {
Err(_) => panic!("Failed to commit!"),
Ok(_) => (),
}
}
Other things: you may want to work with boxed trait objects Box<ToMdbValue>. The link above explains it. You should pass a &[YourType] rather than &Vec<[YourType]>.
I managed to get it working. I'm not sure how kosher this solution is, but I'll post it.
So now, in main(), I do the following (example with an (int, string) kv pair):
let k = 1;
let val = "hello there";
let vec = vec![(&k, &val)];
set(&dbhandle, &env, &vec);
I had to declare them separately since vec![(&1, &"hello there")] threw an error of the form borrowed value does not live long enough.
set now looks like this:
pub fn set<A, B>(handle: &DbHandle, env: &Environment, pairs: &Vec<(&A, &B)>)
-> ()
where A: ToMdbValue,
B: ToMdbValue {
let txn = new_transaction(&env);
{
let db = txn.bind(&handle);
for &(id, note) in pairs.iter() {
db.set(id, note).unwrap();
}
}
match txn.commit() {
Err(_) => panic!("Failed to commit!"),
Ok(_) => (),
}
}