How do I call an async function in a match statement under a non-async main function in Rust? [duplicate] - asynchronous

This question already has answers here:
How to use async/await in Rust when you can't make main function async
(4 answers)
How do I synchronously return a value calculated in an asynchronous Future?
(3 answers)
Closed 3 months ago.
I have a program that does various simple things based on user selection.
fn main() {
let mut user_input = String::new(); // Initialize variable to store user input
println!("Select an option:\r\n[1] Get sysinfo\r\n[2] Read/Write File\r\n[3] Download file\r\n[4] Exit"); // Print options
io::stdin().read_line(&mut user_input).expect("You entered something weird you donkey!"); // Get user input and store in variable
let int_input = user_input.trim().parse::<i32>().unwrap(); // Convert user input to int (i32 means signed integer 32 bits)
match int_input { // If Else statement
1 => getsysinfo(), // If int_input == 1, call getsysinfo()
2 => readwritefile(),
3 => downloadfile(),
4 => process::exit(1), // end program
_ => println!("You didn't choose one of the given options, you donkey!") // input validation
}
}
My function downloadfile() looks like this, referenced from the Rust cookbook on downloading files.
error_chain! {
foreign_links {
Io(std::io::Error);
HttpRequest(reqwest::Error);
}
}
async fn downloadfile() -> Result<()> {
let tmp_dir = Builder::new().prefix("example").tempdir()?;
let target = "localhost:8000/downloaded.txt";
let response = reqwest::get(target).await?;
let mut dest = {
let fname = response
.url()
.path_segments()
.and_then(|segments| segments.last())
.and_then(|name| if name.is_empty() { None } else { Some(name) })
.unwrap_or("tmp.bin");
println!("File to download: {}", fname);
let fname = tmp_dir.path().join(fname);
println!("Will be located under: {:?}", fname);
File::create(fname)?
};
let content = response.text().await?;
copy(&mut content.as_bytes(), &mut dest)?;
Ok(())
}
I get the following error:
`match` arms have incompatible types
expected unit type `()`
found opaque type `impl Future<Output = std::result::Result<(), Error>>`
I presume its because the async function returns a Future type, so how can I make this code work?

You need to use the block_on function.
Add futures as a dependency in your cargo.toml for the following example to work.
use futures::executor::block_on;
async fn hello() -> String {
return String::from("Hello world!");
}
fn main() {
let output = block_on(hello());
println!("{output}");
}

Related

How to use predicates in an async function?

I use thirtyfour in my Rust script, and it uses tokio as the async runtime.
When I use find in a Vec::iter, it doesn't work as I expect:
#[tokio::main]
async fn main() -> WebDriverResult<()> {
let dropdown = driver.find_element(By::Tag("select")).await?;
dropdown
.find_elements(By::Tag("option"))
.await?
.iter()
.find(|&&x| x.text() == book_time.date) // Error, x.text() return a futures::Future type while book_time.date return a &str.
.click()
.await?;
}
After I tried Ibraheem Ahmed's solution, I met more errors:
let dropdown = driver.find_element(By::Tag("select")).await?;
let elements = dropdown.find_elements(By::Tag("option")).await?;
let stream = stream::iter(elements);
let elements = stream.filter(|x| async move { x.text().await.unwrap() == target_date });
error: lifetime may not live long enough
--> src\main.rs:125:38
|
125 | let elements = stream.filter(|x| async move { x.text().await.unwrap() == target_date });
| -- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
| ||
| |return type of closure is impl futures::Future
| has type `&'1 thirtyfour::WebElement<'_>`
There's a good thread on the Rust User's Forum that covers a similar question.
I tried the code snippets below by modifying the thirtyfour tokio_async example. I didn't have your full context, so I created an example that found a link based on its text on the wikipedia.org home page.
filter_map.
let target_value = "Terms of Use";
let found_elements: Vec<_> = stream
.filter_map(|x| async move {
if let Ok(text) = x.text().await {
if text == target_value {
println!("found");
return Some(x);
}
}
None
})
.collect()
.await;
while loop (which is probably not what you are after, but could be a simple solution if your logic fits easily inside)...
while let Some(element) = stream.next().await {
let text = element.text().await?;
println!("link text result: {}", text);
}
You can use a Stream, which is the asynchronous version of Iterator:
use futures::stream::{self, StreamExt};
fn main() {
// ...
let stream = stream::iter(elements).await?;
let elements = stream
.filter(|x| async move { x.text().await.as_deref() == Ok(book_time.date) })
.next()
.click()
.await?;
}

How can I mutate the HTML inside a hyper::Response? [duplicate]

I want to write a server using the current master branch of Hyper that saves a message that is delivered by a POST request and sends this message to every incoming GET request.
I have this, mostly copied from the Hyper examples directory:
extern crate futures;
extern crate hyper;
extern crate pretty_env_logger;
use futures::future::FutureResult;
use hyper::{Get, Post, StatusCode};
use hyper::header::{ContentLength};
use hyper::server::{Http, Service, Request, Response};
use futures::Stream;
struct Echo {
data: Vec<u8>,
}
impl Echo {
fn new() -> Self {
Echo {
data: "text".into(),
}
}
}
impl Service for Echo {
type Request = Request;
type Response = Response;
type Error = hyper::Error;
type Future = FutureResult<Response, hyper::Error>;
fn call(&self, req: Self::Request) -> Self::Future {
let resp = match (req.method(), req.path()) {
(&Get, "/") | (&Get, "/echo") => {
Response::new()
.with_header(ContentLength(self.data.len() as u64))
.with_body(self.data.clone())
},
(&Post, "/") => {
//self.data.clear(); // argh. &self is not mutable :(
// even if it was mutable... how to put the entire body into it?
//req.body().fold(...) ?
let mut res = Response::new();
if let Some(len) = req.headers().get::<ContentLength>() {
res.headers_mut().set(ContentLength(0));
}
res.with_body(req.body())
},
_ => {
Response::new()
.with_status(StatusCode::NotFound)
}
};
futures::future::ok(resp)
}
}
fn main() {
pretty_env_logger::init().unwrap();
let addr = "127.0.0.1:12346".parse().unwrap();
let server = Http::new().bind(&addr, || Ok(Echo::new())).unwrap();
println!("Listening on http://{} with 1 thread.", server.local_addr().unwrap());
server.run().unwrap();
}
How do I turn the req.body() (which seems to be a Stream of Chunks) into a Vec<u8>? I assume I must somehow return a Future that consumes the Stream and turns it into a single Vec<u8>, maybe with fold(). But I have no clue how to do that.
Hyper 0.13 provides a body::to_bytes function for this purpose.
use hyper::body;
use hyper::{Body, Response};
pub async fn read_response_body(res: Response<Body>) -> Result<String, hyper::Error> {
let bytes = body::to_bytes(res.into_body()).await?;
Ok(String::from_utf8(bytes.to_vec()).expect("response was not valid utf-8"))
}
I'm going to simplify the problem to just return the total number of bytes, instead of echoing the entire stream.
Futures 0.3
Hyper 0.13 + TryStreamExt::try_fold
See euclio's answer about hyper::body::to_bytes if you just want all the data as one giant blob.
Accessing the stream allows for more fine-grained control:
use futures::TryStreamExt; // 0.3.7
use hyper::{server::Server, service, Body, Method, Request, Response}; // 0.13.9
use std::convert::Infallible;
use tokio; // 0.2.22
#[tokio::main]
async fn main() {
let addr = "127.0.0.1:12346".parse().expect("Unable to parse address");
let server = Server::bind(&addr).serve(service::make_service_fn(|_conn| async {
Ok::<_, Infallible>(service::service_fn(echo))
}));
println!("Listening on http://{}.", server.local_addr());
if let Err(e) = server.await {
eprintln!("Error: {}", e);
}
}
async fn echo(req: Request<Body>) -> Result<Response<Body>, hyper::Error> {
let (parts, body) = req.into_parts();
match (parts.method, parts.uri.path()) {
(Method::POST, "/") => {
let entire_body = body
.try_fold(Vec::new(), |mut data, chunk| async move {
data.extend_from_slice(&chunk);
Ok(data)
})
.await;
entire_body.map(|body| {
let body = Body::from(format!("Read {} bytes", body.len()));
Response::new(body)
})
}
_ => {
let body = Body::from("Can only POST to /");
Ok(Response::new(body))
}
}
}
Unfortunately, the current implementation of Bytes is no longer compatible with TryStreamExt::try_concat, so we have to switch back to a fold.
Futures 0.1
hyper 0.12 + Stream::concat2
Since futures 0.1.14, you can use Stream::concat2 to stick together all the data into one:
fn concat2(self) -> Concat2<Self>
where
Self: Sized,
Self::Item: Extend<<Self::Item as IntoIterator>::Item> + IntoIterator + Default,
use futures::{
future::{self, Either},
Future, Stream,
}; // 0.1.25
use hyper::{server::Server, service, Body, Method, Request, Response}; // 0.12.20
use tokio; // 0.1.14
fn main() {
let addr = "127.0.0.1:12346".parse().expect("Unable to parse address");
let server = Server::bind(&addr).serve(|| service::service_fn(echo));
println!("Listening on http://{}.", server.local_addr());
let server = server.map_err(|e| eprintln!("Error: {}", e));
tokio::run(server);
}
fn echo(req: Request<Body>) -> impl Future<Item = Response<Body>, Error = hyper::Error> {
let (parts, body) = req.into_parts();
match (parts.method, parts.uri.path()) {
(Method::POST, "/") => {
let entire_body = body.concat2();
let resp = entire_body.map(|body| {
let body = Body::from(format!("Read {} bytes", body.len()));
Response::new(body)
});
Either::A(resp)
}
_ => {
let body = Body::from("Can only POST to /");
let resp = future::ok(Response::new(body));
Either::B(resp)
}
}
}
You could also convert the Bytes into a Vec<u8> via entire_body.to_vec() and then convert that to a String.
See also:
How do I convert a Vector of bytes (u8) to a string
hyper 0.11 + Stream::fold
Similar to Iterator::fold, Stream::fold takes an accumulator (called init) and a function that operates on the accumulator and an item from the stream. The result of the function must be another future with the same error type as the original. The total result is itself a future.
fn fold<F, T, Fut>(self, init: T, f: F) -> Fold<Self, F, Fut, T>
where
F: FnMut(T, Self::Item) -> Fut,
Fut: IntoFuture<Item = T>,
Self::Error: From<Fut::Error>,
Self: Sized,
We can use a Vec as the accumulator. Body's Stream implementation returns a Chunk. This implements Deref<[u8]>, so we can use that to append each chunk's data to the Vec.
extern crate futures; // 0.1.23
extern crate hyper; // 0.11.27
use futures::{Future, Stream};
use hyper::{
server::{Http, Request, Response, Service}, Post,
};
fn main() {
let addr = "127.0.0.1:12346".parse().unwrap();
let server = Http::new().bind(&addr, || Ok(Echo)).unwrap();
println!(
"Listening on http://{} with 1 thread.",
server.local_addr().unwrap()
);
server.run().unwrap();
}
struct Echo;
impl Service for Echo {
type Request = Request;
type Response = Response;
type Error = hyper::Error;
type Future = Box<futures::Future<Item = Response, Error = Self::Error>>;
fn call(&self, req: Self::Request) -> Self::Future {
match (req.method(), req.path()) {
(&Post, "/") => {
let f = req.body()
.fold(Vec::new(), |mut acc, chunk| {
acc.extend_from_slice(&*chunk);
futures::future::ok::<_, Self::Error>(acc)
})
.map(|body| Response::new().with_body(format!("Read {} bytes", body.len())));
Box::new(f)
}
_ => panic!("Nope"),
}
}
}
You could also convert the Vec<u8> body to a String.
See also:
How do I convert a Vector of bytes (u8) to a string
Output
When called from the command line, we can see the result:
$ curl -X POST --data hello http://127.0.0.1:12346/
Read 5 bytes
Warning
All of these solutions allow a malicious end user to POST an infinitely sized file, which would cause the machine to run out of memory. Depending on the intended use, you may wish to establish some kind of cap on the number of bytes read, potentially writing to the filesystem at some breakpoint.
See also:
How do I apply a limit to the number of bytes read by futures::Stream::concat2?
Most of the answers on this topic are outdated or overly complicated. The solution is pretty simple:
/*
WARNING for beginners!!! This use statement
is important so we can later use .data() method!!!
*/
use hyper::body::HttpBody;
let my_vector: Vec<u8> = request.into_body().data().await.unwrap().unwrap().to_vec();
let my_string = String::from_utf8(my_vector).unwrap();
You can also use body::to_bytes as #euclio answered. Both approaches are straight-forward! Don't forget to handle unwrap properly.

How do I handle errors in Warp using both Rejection and the question-mark operator?

Using warp.rs 0.2.2, let's consider a basic web service with one route for GET /:
#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
let getRoot = warp::get().and(warp::path::end()).and_then(routes::getRoot);
warp::serve(getRoot).run(([0, 0, 0, 0], 3030)).await;
Ok(())
}
My goal is to use ? for error handling in the route handlers, so let's write one that can error and return early in crate::routes:
use crate::errors::ServiceError;
use url::Url;
pub async fn getRoot() -> Result<impl warp::Reply, warp::Rejection> {
let _parsed_url = Url::parse(&"https://whydoesn.it/work?").map_err(ServiceError::from)?;
Ok("Hello world !")
}
This version works.
Here the error that's returned by Url::parse() is a url::ParseError
To convert between error types, from url::ParseError to ServiceError, then from ServiceError to warp::Rejection, I've written some error helpers in crate::errors:
#[derive(thiserror::Error, Debug)]
pub enum ServiceError {
#[error(transparent)]
Other(#[from] anyhow::Error), // source and Display delegate to anyhow::Error
}
impl warp::reject::Reject for ServiceError {}
impl From<ServiceError> for warp::reject::Rejection {
fn from(e: ServiceError) -> Self {
warp::reject::custom(e)
}
}
impl From<url::ParseError> for ServiceError {
fn from(e: url::ParseError) -> Self {
ServiceError::Other(e.into())
}
}
Now, the above works, and I'm trying to shorten the second code block to use ? for error handling directly, and convert automatically from the underlying error (here url::ParseError) to a warp::Rejection.
Here's what I've tried:
use crate::errors::ServiceError;
use url::Url;
pub async fn getRoot() -> Result<impl warp::Reply, ServiceError> {
let _parsed_url = Url::parse(&"https://whydoesn.it/work?")?;
Ok("Hello world !")
}
The url::ParseError returned by Url::Parse will convert fine into a ServiceError to return, but returning a ServiceError from my handler doesn't work.
The first compilation error I get is:
error[E0277]: the trait bound `errors::ServiceError: warp::reject::sealed::CombineRejection<warp::reject::Rejection>` is not satisfied
--> src/main.rs:102:54
|
102 | let getRoot = warp::get().and(warp::path::end()).and_then(routes::getRoot);
| ^^^^^^^^ the trait `warp::reject::sealed::CombineRejection<warp::reject::Rejection>` is not implemented for `errors::ServiceError`
Is there a way I can keep the short error handling using ? only and either:
make ServiceError implement warp::reject::sealed::CombineRejection<warp::reject::Rejection> ?
work around that ?
You can implement From to convert your error type into warp::Rejection using reject::custom. Rejection encapsulates custom types which you can later choose to inspect inside of a recover handler.
This example uses a plain error struct, but if you have an error enum you can match on the variants inside the recovery handler and perform different logic as needed.
use serde::Deserialize;
use snafu::{ensure, Snafu};
use std::convert::Infallible;
use warp::{
filters::{any, query, BoxedFilter},
http::StatusCode,
reject::Reject,
Filter, Rejection, Reply,
};
// A normal error type, created by SNAFU
#[derive(Debug, Snafu)]
#[snafu(display("Expected a value less than 10, but it was {}", value))]
struct LessThanTenError {
value: i32,
}
// A function that might fail
fn validate(value: i32) -> Result<i32, LessThanTenError> {
ensure!(value < 10, LessThanTenContext { value });
Ok(value)
}
// We need a custom type to later extract from the `Rejection`. In
// this case, we can reuse the error type itself.
impl Reject for LessThanTenError {}
// To allow using `?`, we implement a conversion from our error to
// `Rejection`
impl From<LessThanTenError> for Rejection {
fn from(other: LessThanTenError) -> Self {
warp::reject::custom(other)
}
}
#[tokio::main]
async fn main() {
let api = simple_math().recover(report_invalid);
let p: std::net::SocketAddr = "0.0.0.0:8888".parse().unwrap();
warp::serve(api).run(p).await;
}
#[derive(Debug, Deserialize)]
struct QueryParams {
a: i32,
b: i32,
}
fn simple_math() -> BoxedFilter<(impl Reply,)> {
any::any()
.and(query::query())
.and_then(|args: QueryParams| async move {
// Look at us using those question marks!
let a = validate(args.a)?;
let b = validate(args.b)?;
let sum = validate(a + b)?;
// We specify that we are returning an error type of
// `Rejection`, which allows the compiler to know what
// type to convert to when using `?` here.
Ok::<_, Rejection>(format!("The sum is {}", sum))
})
.boxed()
}
async fn report_invalid(r: Rejection) -> Result<impl Reply, Infallible> {
if let Some(e) = r.find::<LessThanTenError>() {
// It was our specific error type, do whatever we want. We
// will just print out the error text.
Ok(warp::reply::with_status(
e.to_string(),
StatusCode::BAD_REQUEST,
))
} else {
// Do prettier error reporting for the default error here.
Ok(warp::reply::with_status(
String::from("Something bad happened"),
StatusCode::INTERNAL_SERVER_ERROR,
))
}
}
[dependencies]
serde = { version = "1.0.118", features = ["derive"] }
snafu = "0.6.10"
tokio = { version = "0.2.23", features = ["full"] }
warp = "0.2.5"
% curl 'http://127.0.0.1:8888'
< HTTP/1.1 500 Internal Server Error
Something bad happened
% curl -v 'http://127.0.0.1:8888?a=1&b=2'
< HTTP/1.1 200 OK
The sum is 3
% curl -v 'http://127.0.0.1:8888?a=6&b=5'
< HTTP/1.1 400 Bad Request
Expected a value less than 10, but it was 11
See also:
Is there a way to do validation as part of a filter in Warp?
When should I implement std::convert::From vs std::convert::Into?
How do you define custom `Error` types in Rust?
From my findings, there are two solutions.
Abandon ? in favor of your own macro that constructs and returns a response if there is an error.
Use PR #458 by cjbassi instead of the mainline release by:
Implementing warp::reply::Reply on your error type so that it converts into the correct user facing error message.
Replace warp = "0.2" with warp = { git = "https://github.com/cjbassi/warp.git", branch = "error"} in your Cargo.toml file
use .map_async instead of .and_then for handlers

Unable to use SQLite3 in Rust via FFI due to a misused prepared statement [duplicate]

This question already has answers here:
Raw pointer turns null passing from Rust to C
(1 answer)
How to stop memory leaks when using `as_ptr()`?
(1 answer)
Closed 5 years ago.
I am porting some code to Rust that reads a database path from stdin, opens the database and loops for queries. I have done something similar in C so I am pretty sure that the problem is my non-understanding of Rust FFI.
I am using the sqlite3 binding provided by libsqlite3-sys, the one from rusqlite. The whole code is here.
open_connection initializes a pointer and passes it to sqlite3_open_v2 and checks if everything went well.
// Line 54 of complete code
fn open_connection(s: String) -> Result<RawConnection, SQLite3Error> {
unsafe {
let mut db: *mut sqlite3::sqlite3 = mem::uninitialized();
let r = sqlite3::sqlite3_open_v2(CString::new(s).unwrap().as_ptr(),
&mut db,
sqlite3::SQLITE_OPEN_CREATE |
sqlite3::SQLITE_OPEN_READWRITE,
ptr::null());
match r {
sqlite3::SQLITE_OK => Ok(RawConnection { db: db }),
_ => return Err(SQLite3Error::OpenError),
}
}
}
I create an SQL statement by converting the query from a Rust String to a C String, creating another pointer for the location of the statement itself and I go on creating a statement and checking the output:
// Line 35 of complete code
fn create_statement(conn: &RawConnection, query: String) -> Result<Statement, SQLite3Error> {
let len = query.len();
let raw_query = CString::new(query).unwrap().as_ptr();
unsafe {
let mut stmt: *mut sqlite3::sqlite3_stmt = mem::uninitialized();
if stmt.is_null() {
println!("Now it is null!");
}
match sqlite3::sqlite3_prepare_v2(conn.db,
raw_query,
len as i32,
&mut stmt,
ptr::null_mut()) {
sqlite3::SQLITE_OK => Ok(Statement { stmt: stmt }),
_ => Err(SQLite3Error::StatementError),
}
}
}
I try to execute a statement
// Line 81 of complete code
fn execute_statement(conn: &RawConnection, stmt: Statement) -> Result<Cursor, SQLite3Error> {
match unsafe { sqlite3::sqlite3_step(stmt.stmt) } {
sqlite3::SQLITE_OK => Ok(Cursor::OKCursor),
sqlite3::SQLITE_DONE => Ok(Cursor::DONECursor),
sqlite3::SQLITE_ROW => {
let n_columns = unsafe { sqlite3::sqlite3_column_count(stmt.stmt) } as i32;
let mut types: Vec<EntityType> = Vec::new();
for i in 0..n_columns {
types.push(match unsafe { sqlite3::sqlite3_column_type(stmt.stmt, i) } {
sqlite3::SQLITE_INTEGER => EntityType::Integer,
sqlite3::SQLITE_FLOAT => EntityType::Float,
sqlite3::SQLITE_TEXT => EntityType::Text,
sqlite3::SQLITE_BLOB => EntityType::Blob,
sqlite3::SQLITE_NULL => EntityType::Null,
_ => EntityType::Null,
})
}
Ok(Cursor::RowsCursor {
stmt: stmt,
num_columns: n_columns,
types: types,
previous_status: sqlite3::SQLITE_ROW,
})
}
x => {
println!("{}", x);
return Err(SQLite3Error::ExecuteError);
}
}
}
This always fails with the error code 21 MISUSE. Usually, this error happens when you try to execute a NULL statement but I have no idea how to figure it out.
Do you see any other problem that may cause a code 21 MISUSE?

Unable to return a vector of string slices: borrowed value does not live long enough

I'm new to Rust and I'm having some trouble with the borrow checker. I don't understand why this code won't compile. Sorry if this is close to a previously answered question but I can't seem to find a solution in the other questions I've looked at.
I understand the similarity to Return local String as a slice (&str) but in that case it is just one string being returned and not enough for me to reason with my code in which I am trying to return a vector. From what I understand, I am trying to return references to str types that will go out of scope at the end of the function block and so should I be mapping that vector of &str into a vector of String? I am not so concerned about the performance effects of converting &str to String. First I'd just like to get it working.
This is the code, the error is in the lex function.
use std::io::prelude::*;
use std::fs::File;
use std::env;
fn open(mut s: &mut String, filename: &String) {
let mut f = match File::open(&filename) {
Err(_) => panic!("Couldn't open file"),
Ok(file) => file,
};
match f.read_to_string(&mut s) {
Err(_) => panic!("Couldn't read file"),
Ok(_) => println!("File read successfully"),
};
}
fn lex(s: &String) -> Vec<&str> {
let token_string: String = s.replace("(", " ( ")
.replace(")", " ) ");
let token_list: Vec<&str> = token_string.split_whitespace()
.collect();
token_list
}
fn main() {
let args: Vec<_> = env::args().collect();
if args.len() < 2 {
panic!("Please provide a filename");
} else {
let ref filename = args[1];
let mut s = String::new();
open(&mut s, filename);
let token_list: Vec<&str> = lex(&s);
println!("{:?}", token_list);
}
}
Here is the error message
error: borrowed value does not live long enough
self.0.borrow().values.get(idx)
^~~~~~~~~~~~~~~
reference must be valid for the anonymous lifetime #1 defined on the block at 23:54...
pub fn value(&self, idx: usize) -> Option<&Value> {
^
note: ...but borrowed value is only valid for the block at 23:54
pub fn value(&self, idx: usize) -> Option<&Value> {
^
I'm finding it hard to reason with this code because with my level of experience with Rust I can't visualise the lifetimes of these variables. Any help would be appreciated as I've spent an hour or two trying to figure this out.
The problem is that you're allocating a new String (token_string) inside the lex function and then returning an array of references to it, but token_string will get dropped (and the memory freed) as soon as it falls out of scope at the end of the function.
fn lex(s: &String) -> Vec<&str> {
let token_string: String = s.replace("(", " ( ") // <-- new String allocated
.replace(")", " ) ");
let token_list: Vec<&str> = token_string.split_whitespace()
.collect();
token_list // <-- this is just an array of wide pointers into token_string
} // <-- token_string gets freed here, so the returned pointers
// would be pointing to memory that's already been dropped!
There's a couple of ways to address this. One would be to force the caller of lex to pass in the buffer that you want to use to collect into. This would change the signature to fn lex<'a>(input: &String, buffer: &'a mut String) -> Vec<&'a str> This signature would specify that the lifetimes of the returned &strs will be at least as long as the lifetime of the buffer that's passed in.
Another way would be to just return a Vec<String> instead of Vec<&str> if you can tolerate the extra allocations.

Resources