I am trying to rewrite the proxy example of Asynchronous Programming in Rust book by migrating to :
futures-preview = { version = "0.3.0-alpha.19", features = ["async-await"]}`
hyper = "0.13.0-alpha.4"`
from:
futures-preview = { version = "=0.3.0-alpha.17", features = ["compat"] }`
hyper = "0.12.9"
The current example converts the returned Future from a futures 0.3 into a futures 0.1, because hyper = "0.12.9" is not compatible with futures 0.3's async/await.
My code:
use {
futures::future::{FutureExt, TryFutureExt},
hyper::{
rt::run,
service::{make_service_fn, service_fn},
Body, Client, Error, Request, Response, Server, Uri,
},
std::net::SocketAddr,
std::str::FromStr,
};
fn forward_uri<B>(forward_url: &'static str, req: &Request<B>) -> Uri {
let forward_uri = match req.uri().query() {
Some(query) => format!("{}{}?{}", forward_url, req.uri().path(), query),
None => format!("{}{}", forward_url, req.uri().path()),
};
Uri::from_str(forward_uri.as_str()).unwrap()
}
async fn call(
forward_url: &'static str,
mut _req: Request<Body>,
) -> Result<Response<Body>, hyper::Error> {
*_req.uri_mut() = forward_uri(forward_url, &_req);
let url_str = forward_uri(forward_url, &_req);
let res = Client::new().get(url_str).await;
res
}
async fn run_server(forward_url: &'static str, addr: SocketAddr) {
let forwarded_url = forward_url;
let serve_future = service_fn(move |req| call(forwarded_url, req).boxed());
let server = Server::bind(&addr).serve(serve_future);
if let Err(err) = server.await {
eprintln!("server error: {}", err);
}
}
fn main() {
// Set the address to run our socket on.
let addr = SocketAddr::from(([127, 0, 0, 1], 3000));
let url = "http://127.0.0.1:9061";
let futures_03_future = run_server(url, addr);
run(futures_03_future);
}
First, I receive this error for server in run_server function:
the trait tower_service::Service<&'a
hyper::server::tcp::addr_stream::AddrStream> is not implemented for
hyper::service::service::ServiceFn<[closure#src/main.rs:35:35: 35:78
forwarded_url:_], hyper::body::body::Body>
Also, I cannot use hyper::rt::run because it might have been implemented differently in hyper = 0.13.0-alpha.4.
I will be grateful if you tell me your ideas on how to fix it.
By this issue, to create a new service for each connection you need to create MakeService in hyper = "0.13.0-alpha.4". You can create MakeService with a closure by using make_service_fn.
Also, I cannot use hyper::rt::run because it might have been implemented differently in hyper = 0.13.0-alpha.4.
Correct, under the hood hyper::rt::run was calling tokio::run, it has been removed from the api but currently i don't know the reason. You can run your future with calling tokio::run by yourself or use #[tokio::main] annotation. To do this you need to add tokio to your cargo:
#this is the version of tokio inside hyper "0.13.0-alpha.4"
tokio = "=0.2.0-alpha.6"
then change your run_server like this:
async fn run_server(forward_url: &'static str, addr: SocketAddr) {
let server = Server::bind(&addr).serve(make_service_fn(move |_| {
async move { Ok::<_, Error>(service_fn(move |req| call(forward_url, req))) }
}));
if let Err(err) = server.await {
eprintln!("server error: {}", err);
}
}
and main :
#[tokio::main]
async fn main() -> () {
// Set the address to run our socket on.
let addr = SocketAddr::from(([127, 0, 0, 1], 3000));
let url = "http://www.google.com:80"; // i have tested with google
run_server(url, addr).await
}
Related
I have a small program that executes the aws s3 cli commands but with different arguments. I'm using the Command crate and the the command makes a network call and returns some response. At first I have this synchronous & single-threaded implementation:
fn make_call<'a>(_name: &'a str, _bucket_poll: &mut BucketPoll<'a>) -> Option<BucketDetails<'a>> {
let invoke_result = invoke_network_call(_name);
let mut bucket = BucketDetails::new(_name);
match invoke_result {
Ok(invoke_str) => {
bucket.output = invoke_str;
_bucket_poll.insert_bucket(bucket.clone());
_bucket_poll.successful_count += 1;
Some(bucket)
}
Err(_) => {
_bucket_poll.insert_bucket(bucket);
None
}
}
}
// I invoke this function in sequential order, something like
make_call('name_1');
make_call('name_2');
make_call('name_3');
Because I don't really care at which order this function is executed, I decided to learn Tokio to help with performance. I changed the make_call function to be async:
async fn make_call_race() -> ExecutionResult {
let bucket_poll = BucketPoll::new();
let bucket_poll_guard = Arc::new(Mutex::new(bucket_poll));
loop {
let bucket_details = tokio::select! {
Some(bucket_details) = make_call_async("name_1", &bucket_poll_guard) => bucket_details,
Some(bucket_details) = make_call_async("name_2", &bucket_poll_guard) => bucket_details,
Some(bucket_details) = make_call_async("name_3", &bucket_poll_guard) => bucket_details,
Some(bucket_details) = make_call_async("name_4", &bucket_poll_guard) => bucket_details,
else => { break }
};
success_printer(bucket_details);
}
// more printing, no more network calls
ExecutionResult::Success
}
make_call_async is essentially the same as make_call:
async fn make_call_async<'a>(
_name: &'a str,
_bucket_poll_guard: &'a Arc<Mutex<BucketPoll<'a>>>,
) -> Option<BucketDetails<'a>> {
{
if let Ok(bucket_poll_guard) = _bucket_poll_guard.lock() {
if bucket_poll_guard.has_polled(_name) {
return None;
}
}
}
let invoke_result = invoke_network_call(_name);
let mut bucket = BucketDetails::new(_name);
match invoke_result {
Ok(invoke_str) => {
bucket.output = invoke_str;
{
if let Ok(mut bucket_poll_guard) = _bucket_poll_guard.lock() {
bucket_poll_guard.insert_bucket(bucket.clone());
bucket_poll_guard.successful_count += 1;
}
}
Some(bucket)
}
Err(_) => {
{
if let Ok(mut bucket_poll_guard) = _bucket_poll_guard.lock() {
bucket_poll_guard.insert_bucket(bucket);
}
}
None
}
}
}
When I run the async version, I do see that my network calls are made a random order but I do not notice any speedups. I increased the number of network calls to ~50ish invocations but the runtime is nearly the same if not slightly worse. As I am new to async programming and Rust in general, I would like to understand why my async implementation does not seem to offer any improvement.
Extra:
Here is the invoke_network_call method:
fn invoke_network_call(_name: &str) -> core::result::Result<String, AwsCliError> {
let output = Command::new("aws")
.arg("s3")
.arg("ls")
.arg(_name)
.output()
.expect("Could not list s3 objects");
if !output.status.success() {
err_printer(format!("Failed to list s3 objects for bucket {}.", _name));
return Err(AwsCliError);
}
let output_str = get_stdout_string_from_output(&output);
Ok(output_str)
}
EDIT: yorodm's comment makes sense. What I did was use Tokio's Command instead of std::process's Command and made the invoke_network_call async. This reduced my runtime by half. Thank you!
You could rewrite invoke_network_call using an async version of Command.
async fn invoke_network_call(_name: &str) -> core::result::Result<String, AwsCliError> {
let output = tokio::process::Command::new("aws")
.arg("s3")
.arg("ls")
.arg(_name)
.output()
.await
.expect("Could not list s3 objects");
if !output.status.success() {
err_printer(format!("Failed to list s3 objects for bucket {}.", _name));
return Err(AwsCliError);
}
let output_str = get_stdout_string_from_output(&output);
Ok(output_str)
}
Thus removing the blocking std::process::Command call. However I would say that if you're going to access AWS services you should go with rusoto
I want to write a server using the current master branch of Hyper that saves a message that is delivered by a POST request and sends this message to every incoming GET request.
I have this, mostly copied from the Hyper examples directory:
extern crate futures;
extern crate hyper;
extern crate pretty_env_logger;
use futures::future::FutureResult;
use hyper::{Get, Post, StatusCode};
use hyper::header::{ContentLength};
use hyper::server::{Http, Service, Request, Response};
use futures::Stream;
struct Echo {
data: Vec<u8>,
}
impl Echo {
fn new() -> Self {
Echo {
data: "text".into(),
}
}
}
impl Service for Echo {
type Request = Request;
type Response = Response;
type Error = hyper::Error;
type Future = FutureResult<Response, hyper::Error>;
fn call(&self, req: Self::Request) -> Self::Future {
let resp = match (req.method(), req.path()) {
(&Get, "/") | (&Get, "/echo") => {
Response::new()
.with_header(ContentLength(self.data.len() as u64))
.with_body(self.data.clone())
},
(&Post, "/") => {
//self.data.clear(); // argh. &self is not mutable :(
// even if it was mutable... how to put the entire body into it?
//req.body().fold(...) ?
let mut res = Response::new();
if let Some(len) = req.headers().get::<ContentLength>() {
res.headers_mut().set(ContentLength(0));
}
res.with_body(req.body())
},
_ => {
Response::new()
.with_status(StatusCode::NotFound)
}
};
futures::future::ok(resp)
}
}
fn main() {
pretty_env_logger::init().unwrap();
let addr = "127.0.0.1:12346".parse().unwrap();
let server = Http::new().bind(&addr, || Ok(Echo::new())).unwrap();
println!("Listening on http://{} with 1 thread.", server.local_addr().unwrap());
server.run().unwrap();
}
How do I turn the req.body() (which seems to be a Stream of Chunks) into a Vec<u8>? I assume I must somehow return a Future that consumes the Stream and turns it into a single Vec<u8>, maybe with fold(). But I have no clue how to do that.
Hyper 0.13 provides a body::to_bytes function for this purpose.
use hyper::body;
use hyper::{Body, Response};
pub async fn read_response_body(res: Response<Body>) -> Result<String, hyper::Error> {
let bytes = body::to_bytes(res.into_body()).await?;
Ok(String::from_utf8(bytes.to_vec()).expect("response was not valid utf-8"))
}
I'm going to simplify the problem to just return the total number of bytes, instead of echoing the entire stream.
Futures 0.3
Hyper 0.13 + TryStreamExt::try_fold
See euclio's answer about hyper::body::to_bytes if you just want all the data as one giant blob.
Accessing the stream allows for more fine-grained control:
use futures::TryStreamExt; // 0.3.7
use hyper::{server::Server, service, Body, Method, Request, Response}; // 0.13.9
use std::convert::Infallible;
use tokio; // 0.2.22
#[tokio::main]
async fn main() {
let addr = "127.0.0.1:12346".parse().expect("Unable to parse address");
let server = Server::bind(&addr).serve(service::make_service_fn(|_conn| async {
Ok::<_, Infallible>(service::service_fn(echo))
}));
println!("Listening on http://{}.", server.local_addr());
if let Err(e) = server.await {
eprintln!("Error: {}", e);
}
}
async fn echo(req: Request<Body>) -> Result<Response<Body>, hyper::Error> {
let (parts, body) = req.into_parts();
match (parts.method, parts.uri.path()) {
(Method::POST, "/") => {
let entire_body = body
.try_fold(Vec::new(), |mut data, chunk| async move {
data.extend_from_slice(&chunk);
Ok(data)
})
.await;
entire_body.map(|body| {
let body = Body::from(format!("Read {} bytes", body.len()));
Response::new(body)
})
}
_ => {
let body = Body::from("Can only POST to /");
Ok(Response::new(body))
}
}
}
Unfortunately, the current implementation of Bytes is no longer compatible with TryStreamExt::try_concat, so we have to switch back to a fold.
Futures 0.1
hyper 0.12 + Stream::concat2
Since futures 0.1.14, you can use Stream::concat2 to stick together all the data into one:
fn concat2(self) -> Concat2<Self>
where
Self: Sized,
Self::Item: Extend<<Self::Item as IntoIterator>::Item> + IntoIterator + Default,
use futures::{
future::{self, Either},
Future, Stream,
}; // 0.1.25
use hyper::{server::Server, service, Body, Method, Request, Response}; // 0.12.20
use tokio; // 0.1.14
fn main() {
let addr = "127.0.0.1:12346".parse().expect("Unable to parse address");
let server = Server::bind(&addr).serve(|| service::service_fn(echo));
println!("Listening on http://{}.", server.local_addr());
let server = server.map_err(|e| eprintln!("Error: {}", e));
tokio::run(server);
}
fn echo(req: Request<Body>) -> impl Future<Item = Response<Body>, Error = hyper::Error> {
let (parts, body) = req.into_parts();
match (parts.method, parts.uri.path()) {
(Method::POST, "/") => {
let entire_body = body.concat2();
let resp = entire_body.map(|body| {
let body = Body::from(format!("Read {} bytes", body.len()));
Response::new(body)
});
Either::A(resp)
}
_ => {
let body = Body::from("Can only POST to /");
let resp = future::ok(Response::new(body));
Either::B(resp)
}
}
}
You could also convert the Bytes into a Vec<u8> via entire_body.to_vec() and then convert that to a String.
See also:
How do I convert a Vector of bytes (u8) to a string
hyper 0.11 + Stream::fold
Similar to Iterator::fold, Stream::fold takes an accumulator (called init) and a function that operates on the accumulator and an item from the stream. The result of the function must be another future with the same error type as the original. The total result is itself a future.
fn fold<F, T, Fut>(self, init: T, f: F) -> Fold<Self, F, Fut, T>
where
F: FnMut(T, Self::Item) -> Fut,
Fut: IntoFuture<Item = T>,
Self::Error: From<Fut::Error>,
Self: Sized,
We can use a Vec as the accumulator. Body's Stream implementation returns a Chunk. This implements Deref<[u8]>, so we can use that to append each chunk's data to the Vec.
extern crate futures; // 0.1.23
extern crate hyper; // 0.11.27
use futures::{Future, Stream};
use hyper::{
server::{Http, Request, Response, Service}, Post,
};
fn main() {
let addr = "127.0.0.1:12346".parse().unwrap();
let server = Http::new().bind(&addr, || Ok(Echo)).unwrap();
println!(
"Listening on http://{} with 1 thread.",
server.local_addr().unwrap()
);
server.run().unwrap();
}
struct Echo;
impl Service for Echo {
type Request = Request;
type Response = Response;
type Error = hyper::Error;
type Future = Box<futures::Future<Item = Response, Error = Self::Error>>;
fn call(&self, req: Self::Request) -> Self::Future {
match (req.method(), req.path()) {
(&Post, "/") => {
let f = req.body()
.fold(Vec::new(), |mut acc, chunk| {
acc.extend_from_slice(&*chunk);
futures::future::ok::<_, Self::Error>(acc)
})
.map(|body| Response::new().with_body(format!("Read {} bytes", body.len())));
Box::new(f)
}
_ => panic!("Nope"),
}
}
}
You could also convert the Vec<u8> body to a String.
See also:
How do I convert a Vector of bytes (u8) to a string
Output
When called from the command line, we can see the result:
$ curl -X POST --data hello http://127.0.0.1:12346/
Read 5 bytes
Warning
All of these solutions allow a malicious end user to POST an infinitely sized file, which would cause the machine to run out of memory. Depending on the intended use, you may wish to establish some kind of cap on the number of bytes read, potentially writing to the filesystem at some breakpoint.
See also:
How do I apply a limit to the number of bytes read by futures::Stream::concat2?
Most of the answers on this topic are outdated or overly complicated. The solution is pretty simple:
/*
WARNING for beginners!!! This use statement
is important so we can later use .data() method!!!
*/
use hyper::body::HttpBody;
let my_vector: Vec<u8> = request.into_body().data().await.unwrap().unwrap().to_vec();
let my_string = String::from_utf8(my_vector).unwrap();
You can also use body::to_bytes as #euclio answered. Both approaches are straight-forward! Don't forget to handle unwrap properly.
How can I make an HTTP request from Rust? I can't seem to find anything in the core library.
I don't need to parse the output, just make a request and check the HTTP response code.
Bonus marks if someone can show me how to URL encode the query parameters on my URL!
The easiest way to make HTTP requests in Rust is with the reqwest crate:
use std::error::Error;
fn main() -> Result<(), Box<dyn Error>> {
let resp = reqwest::blocking::get("https://httpbin.org/ip")?.text()?;
println!("{:#?}", resp);
Ok(())
}
In Cargo.toml:
[dependencies]
reqwest = { version = "0.11", features = ["blocking"] }
Async
Reqwest also supports making asynchronous HTTP requests using Tokio:
use std::error::Error;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let resp = reqwest::get("https://httpbin.org/ip")
.await?
.text()
.await?;
println!("{:#?}", resp);
Ok(())
}
In Cargo.toml:
[dependencies]
reqwest = "0.11"
tokio = { version = "1", features = ["full"] }
Hyper
Reqwest is an easy to use wrapper around Hyper, which is a popular HTTP library for Rust. You can use it directly if you need more control over managing connections. A Hyper-based example is below and is largely inspired by an example in its documentation:
use hyper::{body::HttpBody as _, Client, Uri};
use std::error::Error;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let client = Client::new();
let res = client
.get(Uri::from_static("http://httpbin.org/ip"))
.await?;
println!("status: {}", res.status());
let buf = hyper::body::to_bytes(res).await?;
println!("body: {:?}", buf);
}
In Cargo.toml:
[dependencies]
hyper = { version = "0.14", features = ["full"] }
tokio = { version = "1", features = ["full"] }
Original answer (Rust 0.6)
I believe what you're looking for is in the standard library. now in rust-http and Chris Morgan's answer is the standard way in current Rust for the foreseeable future. I'm not sure how far I can take you (and hope I'm not taking you the wrong direction!), but you'll want something like:
// Rust 0.6 -- old code
extern mod std;
use std::net_ip;
use std::uv;
fn main() {
let iotask = uv::global_loop::get();
let result = net_ip::get_addr("www.duckduckgo.com", &iotask);
io::println(fmt!("%?", result));
}
As for encoding, there are some examples in the unit tests in src/libstd/net_url.rs.
Update: This answer refers to fairly ancient history. For the current best practices, please look at Isaac Aggrey's answer instead.
I've been working on rust-http, which has become the de facto HTTP library for Rust (Servo uses it); it's far from complete and very poorly documented at present. Here's an example of making a request and doing something with the status code:
extern mod http;
use http::client::RequestWriter;
use http::method::Get;
use http::status;
use std::os;
fn main() {
let request = RequestWriter::new(Get, FromStr::from_str(os::args()[1]).unwrap());
let response = match request.read_response() {
Ok(response) => response,
Err(_request) => unreachable!(), // Uncaught condition will have failed first
};
if response.status == status::Ok {
println!("Oh goodie, I got me a 200 OK response!");
} else {
println!("That URL ain't returning 200 OK, it returned {} instead", response.status);
}
}
Run this code with a URL as the sole command-line argument and it'll check the status code! (HTTP only; no HTTPS.)
Compare with src/examples/client/client.rs for an example that does a little more.
rust-http is tracking the master branch of rust. At present it'll work in the just-released Rust 0.8, but there are likely to be breaking changes soon. Actually, no version of rust-http works on Rust 0.8—there was a breaking change which can't be worked around in privacy rules just before the release, leaving something that rust-http depends on in extra::url inaccessible. This has since been fixed, but it leaves rust-http incompatible with Rust 0.8.
As for the query string encoding matter, at present that should be done with extra::url::Query (a typedef for ~[(~str, ~str)]). Appropriate functions for conversions:
extra::url::query_to_str
extra::url::query_from_str (sorry, can't use this just at present as it's private. PR to make it public about to come. In the mean time, this link actually shouldn't work, it's only available because of https://github.com/mozilla/rust/issues/7476.)
Using curl bindings. Stick this in your Cargo.toml:
[dependencies.curl]
git = "https://github.com/carllerche/curl-rust"
...and this in the src/main.rs:
extern crate curl;
use curl::http;
fn main(){
let resp = http::handle()
.post("http://localhost:3000/login", "username=dude&password=sikrit")
.exec().unwrap();
println!("code={}; headers={}; body={}",
resp.get_code(), resp.get_headers(), resp.get_body());
}
I prefer Crates with low dependency count, so I would recommend these:
MinReq (0 deps)
use minreq;
fn main() -> Result<(), minreq::Error> {
let o = minreq::get("https://speedtest.lax.hivelocity.net").send()?;
let s = o.as_str()?;
print!("{}", s);
Ok(())
}
HTTP_Req (35 deps)
use {http_req::error, http_req::request, std::io, std::io::Write};
fn main() -> Result<(), error::Error> {
let mut a = Vec::new();
request::get("https://speedtest.lax.hivelocity.net", &mut a)?;
io::stdout().write(&a)?;
Ok(())
}
To elaborate on Isaac Aggrey's answer, here's an example of making a POST request with query parameters using the reqwest library.
Cargo.toml
[package]
name = "play_async"
version = "0.1.0"
edition = "2018"
[dependencies]
reqwest = "0.10.4"
tokio = { version = "0.2.21", features = ["macros"] }
Code
use reqwest::Client;
type Error = Box<dyn std::error::Error>;
type Result<T, E = Error> = std::result::Result<T, E>;
async fn post_greeting() -> Result<()> {
let client = Client::new();
let req = client
// or use .post, etc.
.get("https://webhook.site/1dff66fd-07ff-4cb5-9a77-681efe863747")
.header("Accepts", "application/json")
.query(&[("hello", "1"), ("world", "ABCD")]);
let res = req.send().await?;
println!("{}", res.status());
let body = res.bytes().await?;
let v = body.to_vec();
let s = String::from_utf8_lossy(&v);
println!("response: {} ", s);
Ok(())
}
#[tokio::main]
async fn main() -> Result<()> {
post_greeting().await?;
Ok(())
}
Go to https://webhook.site and create your webhook link and change the code to match. You'll see the request was received on server in realtime.
This example was originally based on Bastian Gruber's example and has been updated for modern Rust syntax and newer crate versions.
Building upon Patrik Stas' answer, if you want to do an HTTP form URL-encoded POST, here is what you have to do. In this case, it's to get an OAuth client_credentials token.
Cargo.toml
[dependencies]
reqwest = "0.10.4"
tokio = { version = "0.2.21", features = ["macros"] }
Code
use reqwest::{Client, Method};
type Error = Box<dyn std::error::Error>;
type Result<T, E = Error> = std::result::Result<T, E>;
async fn print_access_token() -> Result<()> {
let client = Client::new();
let host = "login.microsoftonline.com";
let tenant = "TENANT";
let client_id = "CLIENT_ID";
let client_secret = "CLIENT_SECRET";
let scope = "https://graph.microsoft.com/.default";
let grant_type = "client_credentials";
let url_string = format!("https://{}/{}/oauth2/v2.0/token", host, tenant);
let body = format!(
"client_id={}&client_secret={}&scope={}&grant_type={}",
client_id, client_secret, scope, grant_type,
);
let req = client.request(Method::POST, &url_string).body(body);
let res = req.send().await?;
println!("{}", res.status());
let body = res.bytes().await?;
let v = body.to_vec();
let s = String::from_utf8_lossy(&v);
println!("response: {} ", s);
Ok(())
}
#[tokio::main]
async fn main() -> Result<()> {
print_access_token().await?;
Ok(())
}
This will print something like the following.
200 OK
response: {"token_type":"Bearer","expires_in":3599,"ext_expires_in":3599,"access_token":"ACCESS_TOKEN"}
Dropping a version here that uses the surf crate (dual to the tide crate):
let res = surf::get("https://httpbin.org/get").await?;
assert_eq!(res.status(), 200);
Using hyper "0.13"
Also using hyper-tls for HTTPS support.
File Cargo.toml
hyper = "0.13"
hyper-tls = "0.4.1"
tokio = { version = "0.2", features = ["full"] }
Code
extern crate hyper;
use hyper::Client;
use hyper::body::HttpBody as _;
use tokio::io::{stdout, AsyncWriteExt as _};
use hyper_tls::HttpsConnector;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
// HTTP only
// let client = Client::new();
// http or https connections
let client = Client::builder().build::<_, hyper::Body>(HttpsConnector::new());
let mut resp = client.get("https://catfact.ninja/fact".parse()?).await?;
println!("Response: {}", resp.status());
while let Some(chunk) = resp.body_mut().data().await {
stdout().write_all(&chunk?).await?;
}
Ok(())
}
Adapted from https://hyper.rs/guides/client/basic/
Simple http request with this crate: wsd
fn test() {
wsd::http::get("https://docs.rs/", |data| {
println!("status = {}, data = {}", data.status(), data.text());
});
}
I have a concept project where the client sends a server a number (PrimeClientRequest), the server computes if the value is prime or not, and returns a response (PrimeClientResponse). I want the client to be a simple CLI which prompts the user for a number, sends the request to the server, and displays the response. Ideally I want to do this using TcpClient from Tokio and Streams from Futures-Rs.
I've written a Tokio server using services and I want to reuse the same codec and proto for the client.
Part of the client is a function called read_prompt which returns a Stream. Essentially it is an infinite loop at which each iteration reads in some input from stdin.
Here's the relevant code:
main.rs
use futures::{Future, Stream};
use std::env;
use std::net::SocketAddr;
use tokio_core::reactor::Core;
use tokio_prime::protocol::PrimeClientProto;
use tokio_prime::request::PrimeRequest;
use tokio_proto::TcpClient;
use tokio_service::Service;
mod cli;
fn main() {
let mut core = Core::new().unwrap();
let handle = core.handle();
let addr_string = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let remote_addr = addr_string.parse::<SocketAddr>().unwrap();
println!("Connecting on {}", remote_addr);
let tcp_client = TcpClient::new(PrimeClientProto).connect(&remote_addr, &handle);
core.run(tcp_client.and_then(|client| {
client
.call(PrimeRequest { number: Ok(0) })
.and_then(|response| {
println!("RESP = {:?}", response);
Ok(())
})
})).unwrap();
}
cli.rs
use futures::{Future, Sink, Stream};
use futures::sync::mpsc;
use std::{io, thread};
use std::io::{Stdin, Stdout};
use std::io::prelude::*;
pub fn read_prompt() -> impl Stream<Item = u64, Error = ()> {
let (tx, rx) = mpsc::channel(1);
thread::spawn(move || loop {
let thread_tx = tx.clone();
let input = prompt(io::stdout(), io::stdin()).unwrap();
let parsed_input = input
.parse::<u64>()
.map_err(|_| io::Error::new(io::ErrorKind::Other, "invalid u64"));
thread_tx.send(parsed_input.unwrap()).wait().unwrap();
});
rx
}
fn prompt(stdout: Stdout, stdin: Stdin) -> io::Result<String> {
let mut stdout_handle = stdout.lock();
stdout_handle.write(b"> ")?;
stdout_handle.flush()?;
let mut buf = String::new();
let mut stdin_handle = stdin.lock();
stdin_handle.read_line(&mut buf)?;
Ok(buf.trim().to_string())
}
With the code above, the client sends a single request to the server before the client terminates. I want to be able to use the stream generated from read_prompt to provide input to the TcpClient and make a request per item in the stream. How would I go about doing this?
The full code can be found at joshleeb/tokio-prime.
The solution I have come up with (so far) has been to use the LoopFn in the Future-Rs crate. It's not ideal as a new connection still has to be made but it is at least a step in the right direction.
main.rs
use futures::{future, Future};
use std::{env, io};
use std::net::SocketAddr;
use tokio_core::reactor::{Core, Handle};
use tokio_prime::protocol::PrimeClientProto;
use tokio_prime::request::PrimeRequest;
use tokio_proto::TcpClient;
use tokio_service::Service;
mod cli;
fn handler<'a>(
handle: &'a Handle, addr: &'a SocketAddr
) -> impl Future<Item = (), Error = ()> + 'a {
cli::prompt(io::stdin(), io::stdout())
.and_then(move |number| {
TcpClient::new(PrimeClientProto)
.connect(addr, handle)
.and_then(move |client| Ok((client, number)))
})
.and_then(|(client, number)| {
client
.call(PrimeRequest { number: Ok(number) })
.and_then(|response| {
println!("{:?}", response);
Ok(())
})
})
.or_else(|err| {
println!("! {}", err);
Ok(())
})
}
fn main() {
let mut core = Core::new().unwrap();
let handle = core.handle();
let addr_string = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let remote_addr = addr_string.parse::<SocketAddr>().unwrap();
println!("Connecting on {}", remote_addr);
let client = future::loop_fn((), |_| {
handler(&handle, &remote_addr)
.map(|_| -> future::Loop<(), ()> { future::Loop::Continue(()) })
});
core.run(client).ok();
}
cli.rs
use futures::prelude::*;
use std::io;
use std::io::{Stdin, Stdout};
use std::io::prelude::*;
#[async]
pub fn prompt(stdin: Stdin, stdout: Stdout) -> io::Result<u64> {
let mut stdout_handle = stdout.lock();
stdout_handle.write(b"> ")?;
stdout_handle.flush()?;
let mut buf = String::new();
let mut stdin_handle = stdin.lock();
stdin_handle.read_line(&mut buf)?;
parse_input(buf.trim().to_string())
}
fn parse_input(s: String) -> io::Result<u64> {
s.parse::<u64>()
.map_err(|_| io::Error::new(io::ErrorKind::Other, "invalid u64"))
}
How can I make an HTTP request from Rust? I can't seem to find anything in the core library.
I don't need to parse the output, just make a request and check the HTTP response code.
Bonus marks if someone can show me how to URL encode the query parameters on my URL!
The easiest way to make HTTP requests in Rust is with the reqwest crate:
use std::error::Error;
fn main() -> Result<(), Box<dyn Error>> {
let resp = reqwest::blocking::get("https://httpbin.org/ip")?.text()?;
println!("{:#?}", resp);
Ok(())
}
In Cargo.toml:
[dependencies]
reqwest = { version = "0.11", features = ["blocking"] }
Async
Reqwest also supports making asynchronous HTTP requests using Tokio:
use std::error::Error;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let resp = reqwest::get("https://httpbin.org/ip")
.await?
.text()
.await?;
println!("{:#?}", resp);
Ok(())
}
In Cargo.toml:
[dependencies]
reqwest = "0.11"
tokio = { version = "1", features = ["full"] }
Hyper
Reqwest is an easy to use wrapper around Hyper, which is a popular HTTP library for Rust. You can use it directly if you need more control over managing connections. A Hyper-based example is below and is largely inspired by an example in its documentation:
use hyper::{body::HttpBody as _, Client, Uri};
use std::error::Error;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let client = Client::new();
let res = client
.get(Uri::from_static("http://httpbin.org/ip"))
.await?;
println!("status: {}", res.status());
let buf = hyper::body::to_bytes(res).await?;
println!("body: {:?}", buf);
}
In Cargo.toml:
[dependencies]
hyper = { version = "0.14", features = ["full"] }
tokio = { version = "1", features = ["full"] }
Original answer (Rust 0.6)
I believe what you're looking for is in the standard library. now in rust-http and Chris Morgan's answer is the standard way in current Rust for the foreseeable future. I'm not sure how far I can take you (and hope I'm not taking you the wrong direction!), but you'll want something like:
// Rust 0.6 -- old code
extern mod std;
use std::net_ip;
use std::uv;
fn main() {
let iotask = uv::global_loop::get();
let result = net_ip::get_addr("www.duckduckgo.com", &iotask);
io::println(fmt!("%?", result));
}
As for encoding, there are some examples in the unit tests in src/libstd/net_url.rs.
Update: This answer refers to fairly ancient history. For the current best practices, please look at Isaac Aggrey's answer instead.
I've been working on rust-http, which has become the de facto HTTP library for Rust (Servo uses it); it's far from complete and very poorly documented at present. Here's an example of making a request and doing something with the status code:
extern mod http;
use http::client::RequestWriter;
use http::method::Get;
use http::status;
use std::os;
fn main() {
let request = RequestWriter::new(Get, FromStr::from_str(os::args()[1]).unwrap());
let response = match request.read_response() {
Ok(response) => response,
Err(_request) => unreachable!(), // Uncaught condition will have failed first
};
if response.status == status::Ok {
println!("Oh goodie, I got me a 200 OK response!");
} else {
println!("That URL ain't returning 200 OK, it returned {} instead", response.status);
}
}
Run this code with a URL as the sole command-line argument and it'll check the status code! (HTTP only; no HTTPS.)
Compare with src/examples/client/client.rs for an example that does a little more.
rust-http is tracking the master branch of rust. At present it'll work in the just-released Rust 0.8, but there are likely to be breaking changes soon. Actually, no version of rust-http works on Rust 0.8—there was a breaking change which can't be worked around in privacy rules just before the release, leaving something that rust-http depends on in extra::url inaccessible. This has since been fixed, but it leaves rust-http incompatible with Rust 0.8.
As for the query string encoding matter, at present that should be done with extra::url::Query (a typedef for ~[(~str, ~str)]). Appropriate functions for conversions:
extra::url::query_to_str
extra::url::query_from_str (sorry, can't use this just at present as it's private. PR to make it public about to come. In the mean time, this link actually shouldn't work, it's only available because of https://github.com/mozilla/rust/issues/7476.)
Using curl bindings. Stick this in your Cargo.toml:
[dependencies.curl]
git = "https://github.com/carllerche/curl-rust"
...and this in the src/main.rs:
extern crate curl;
use curl::http;
fn main(){
let resp = http::handle()
.post("http://localhost:3000/login", "username=dude&password=sikrit")
.exec().unwrap();
println!("code={}; headers={}; body={}",
resp.get_code(), resp.get_headers(), resp.get_body());
}
I prefer Crates with low dependency count, so I would recommend these:
MinReq (0 deps)
use minreq;
fn main() -> Result<(), minreq::Error> {
let o = minreq::get("https://speedtest.lax.hivelocity.net").send()?;
let s = o.as_str()?;
print!("{}", s);
Ok(())
}
HTTP_Req (35 deps)
use {http_req::error, http_req::request, std::io, std::io::Write};
fn main() -> Result<(), error::Error> {
let mut a = Vec::new();
request::get("https://speedtest.lax.hivelocity.net", &mut a)?;
io::stdout().write(&a)?;
Ok(())
}
To elaborate on Isaac Aggrey's answer, here's an example of making a POST request with query parameters using the reqwest library.
Cargo.toml
[package]
name = "play_async"
version = "0.1.0"
edition = "2018"
[dependencies]
reqwest = "0.10.4"
tokio = { version = "0.2.21", features = ["macros"] }
Code
use reqwest::Client;
type Error = Box<dyn std::error::Error>;
type Result<T, E = Error> = std::result::Result<T, E>;
async fn post_greeting() -> Result<()> {
let client = Client::new();
let req = client
// or use .post, etc.
.get("https://webhook.site/1dff66fd-07ff-4cb5-9a77-681efe863747")
.header("Accepts", "application/json")
.query(&[("hello", "1"), ("world", "ABCD")]);
let res = req.send().await?;
println!("{}", res.status());
let body = res.bytes().await?;
let v = body.to_vec();
let s = String::from_utf8_lossy(&v);
println!("response: {} ", s);
Ok(())
}
#[tokio::main]
async fn main() -> Result<()> {
post_greeting().await?;
Ok(())
}
Go to https://webhook.site and create your webhook link and change the code to match. You'll see the request was received on server in realtime.
This example was originally based on Bastian Gruber's example and has been updated for modern Rust syntax and newer crate versions.
Building upon Patrik Stas' answer, if you want to do an HTTP form URL-encoded POST, here is what you have to do. In this case, it's to get an OAuth client_credentials token.
Cargo.toml
[dependencies]
reqwest = "0.10.4"
tokio = { version = "0.2.21", features = ["macros"] }
Code
use reqwest::{Client, Method};
type Error = Box<dyn std::error::Error>;
type Result<T, E = Error> = std::result::Result<T, E>;
async fn print_access_token() -> Result<()> {
let client = Client::new();
let host = "login.microsoftonline.com";
let tenant = "TENANT";
let client_id = "CLIENT_ID";
let client_secret = "CLIENT_SECRET";
let scope = "https://graph.microsoft.com/.default";
let grant_type = "client_credentials";
let url_string = format!("https://{}/{}/oauth2/v2.0/token", host, tenant);
let body = format!(
"client_id={}&client_secret={}&scope={}&grant_type={}",
client_id, client_secret, scope, grant_type,
);
let req = client.request(Method::POST, &url_string).body(body);
let res = req.send().await?;
println!("{}", res.status());
let body = res.bytes().await?;
let v = body.to_vec();
let s = String::from_utf8_lossy(&v);
println!("response: {} ", s);
Ok(())
}
#[tokio::main]
async fn main() -> Result<()> {
print_access_token().await?;
Ok(())
}
This will print something like the following.
200 OK
response: {"token_type":"Bearer","expires_in":3599,"ext_expires_in":3599,"access_token":"ACCESS_TOKEN"}
Dropping a version here that uses the surf crate (dual to the tide crate):
let res = surf::get("https://httpbin.org/get").await?;
assert_eq!(res.status(), 200);
Using hyper "0.13"
Also using hyper-tls for HTTPS support.
File Cargo.toml
hyper = "0.13"
hyper-tls = "0.4.1"
tokio = { version = "0.2", features = ["full"] }
Code
extern crate hyper;
use hyper::Client;
use hyper::body::HttpBody as _;
use tokio::io::{stdout, AsyncWriteExt as _};
use hyper_tls::HttpsConnector;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
// HTTP only
// let client = Client::new();
// http or https connections
let client = Client::builder().build::<_, hyper::Body>(HttpsConnector::new());
let mut resp = client.get("https://catfact.ninja/fact".parse()?).await?;
println!("Response: {}", resp.status());
while let Some(chunk) = resp.body_mut().data().await {
stdout().write_all(&chunk?).await?;
}
Ok(())
}
Adapted from https://hyper.rs/guides/client/basic/
Simple http request with this crate: wsd
fn test() {
wsd::http::get("https://docs.rs/", |data| {
println!("status = {}, data = {}", data.status(), data.text());
});
}