How do you make a GET request in Rust? - http

I noticed that Rust doesn't have a builtin library to deal with HTTP, it only has a net module that deals with raw IP and TCP protocols.
I need to take a &str of the URL, make a HTTP GET request, and if successful return either a String or &str that corresponds to the HTML or JSON or other response in string form.
It would look something like:
use somelib::http;
let response = http::get(&"http://stackoverflow.com");
match response {
Some(suc) => suc,
None => panic!
}

The current best practice for this particular problem is to use the reqwest crate, as specified in the Rust Cookbook. This code is slightly adapted from the cookbook to run standalone:
extern crate reqwest; // 0.9.18
use std::io::Read;
fn run() -> Result<(), Box<dyn std::error::Error>> {
let mut res = reqwest::get("http://httpbin.org/get")?;
let mut body = String::new();
res.read_to_string(&mut body)?;
println!("Status: {}", res.status());
println!("Headers:\n{:#?}", res.headers());
println!("Body:\n{}", body);
Ok(())
}
As the cookbook mentions, this code will be executed synchronously.
See also:
How can I perform parallel asynchronous HTTP GET requests with reqwest?

Take a look at Hyper.
Sending a GET request is as simple as this.
let client = Client::new();
let res = client.get("http://example.domain").send().unwrap();
assert_eq!(res.status, hyper::Ok);
You can find more examples in the documentation.
Edit:
It seems that Hyper got a bit more complicated since they started to use Tokio. Here is updated version.
extern crate futures;
extern crate hyper;
extern crate tokio_core;
use std::io::{self, Write};
use futures::{Future, Stream};
use hyper::Client;
use tokio_core::reactor::Core;
fn main() {
let mut core = Core::new().unwrap();
let client = Client::new(&core.handle());
let uri = "http://httpbin.org/ip".parse().unwrap();
let work =
client.get(uri).and_then(|res| {
println!("Response: {}", res.status());
res.body().for_each(|chunk| {
io::stdout()
.write_all(&chunk)
.map_err(From::from)
})
});
core.run(work).unwrap();
}
And here are the required dependencies.
[dependencies]
futures = "0.1"
hyper = "0.11"
tokio-core = "0.1"

Try to go for reqwest:
extern crate reqwest;
fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut res = reqwest::get("https://httpbin.org/headers")?;
// copy the response body directly to stdout
std::io::copy(&mut res, &mut std::io::stdout())?;
Ok(())
}

Be aware that hyper has lots of dependencies and reqwest not only depends on hyper but also brings quite a few additional dependencies. This may or may not be an issue for you, but it's something you should be aware of.
An alternative you might want to consider is minreq. That crate has only 1 dependency by default (the log crate) and a few more that depend on optional features. For example, if you activate the https-native feature, it will also add native-tls as a dependency.

Related

How to send large custom struct over HTTP in Rust lang using reqwest, tokio and actix_web

Issue
I have a client that needs to send the following custom data structure to an API:
#[derive(Serialize, Deserialize)]
pub struct FheSum {
pub server_keys: ServerKey,
pub value1: FheUint8,
pub value2: FheUint8,
}
The code for the client is the following:
let fhe_post: FheSum = FheSum {
server_keys: server_keys.to_owned(),
value1: value_api.to_owned(),
value2: value_api_2.to_owned(),
};
let client = reqwest::blocking::Client::builder()
.timeout(None)
.build().unwrap();
let response = client
.post("http://127.0.0.1:8000/computesum")
.json(&fhe_post)
.send().unwrap();
let response_json: Result<FheSumResult, reqwest::Error> = response.json();
match response_json {
Ok(j) => {
let result_api: u8 = FheUint8::decrypt(&j.result, &client_keys);
println!("Final Result: {}", result_api)
},
Err(e) => println!("{:}", e),
};
In the API, I have the following definition of an HttpServer:
HttpServer::new(|| {
let json_cfg = actix_web::web::JsonConfig::default()
.limit(std::usize::MAX);
App::new()
.app_data(json_cfg)
.service(integers::computesum)
})
.client_disconnect_timeout(std::time::Duration::from_secs(3000))
.client_request_timeout(std::time::Duration::from_secs(3000))
.max_connection_rate(std::usize::MAX)
.bind(("127.0.0.1", 8000))?
.run()
.await
And the associated endpoint the client is trying to access:
#[post("/computesum")]
pub async fn computesum(req: Json<FheSum>) -> HttpResponse {
let req: FheSum = req.into_inner();
let recovered: FheSum = FheSum::new(
req.server_keys,
req.value1,
req.value2,
);
set_server_key(recovered.server_keys);
let result_api_enc: FheSumResult = FheSumResult::new(recovered.value1 + recovered.value2);
HttpResponse::Ok()
.content_type(ContentType::json())
.json(&result_api_enc)
}
Problem
The structs are the same in both the client and the server. This code works when using common data types such as Strings. The issue is when using this data structures. The memory occupied, obtained with mem::size_of_val which returns the size in bytes, is the following:
Size 1: 2488
Size 2: 32
Size 3: 32
The result has been obtained in bytes, so, given the limit established in the HttpServer, this shouldn't be an issue. Timeouts have also been set at much higher values than commonly needed.
Even with this changes, the client always shows Killed, and doesn't display the answer from the server, not giving any clues on what the problem might be.
The client is killing the process before being able to process the server's response. I want to find a way to send these custom data types to the HTTP server without the connection closing before the operation has finished.
I have already tried different libraries for the client such as the acw crate, apart from reqwest and the result is the same. I have also tried not using reqwest in blocking mode, and the error persists.

How to POST a multipart form using async version of reqwest crate?

The documentation for https://docs.rs/reqwest/latest/reqwest/blocking/multipart/struct.Form.html shows this example to create a multipart::Form from a file.
let file = reqwest::blocking::multipart::Form::new().file("key", "/path/to/file")?;
let response = reqwest::blocking::Client::new()
.post("https:://test.com.br/send")
.multipart(file)
.send()
.unwrap();
But the function ".file" is only available on the blocking version of reqwest (reqwest::blocking::multipart::Form).
I've tested the blocking version, and i was able to send a form. But i can't find a way to do this using the async version (reqwest::multipart::Form).
Is there a alternative way to make this call using the async version?
If you want to post just a file using async, please check this answer. If you want to construct a multipart form, try to use reqwest::multipart::Part to construct a body, while wrap your file into a stream.
i m successfully update file with multipart with this code(you could change patch to post)
pub async fn update_form(&self, con: &Collection, id: &str, path: &str) -> String {
let url = [&self.url_struct(con), "/", id].concat();
let file = fs::read(path).unwrap();
let file_part = reqwest::multipart::Part::bytes(file)
.file_name("bg.jpg")
.mime_str("image/jpg")
.unwrap();
let form = reqwest::multipart::Form::new().part("img", file_part);
let client = reqwest::Client::new();
match client
.patch(url)
.headers(construct_headers_form())
.multipart(form)
.send()
.await
{
Ok(res) => res.text().await.unwrap_or("no message".to_string()),
Err(_) => "{\"error\":400}".to_string(),
}
}

Hyper cannot find Server module

I'm writing a "hello world" HTTP server with Hyper, however I am not able to find the Server and rt modules when trying to import them.
When invoking cargo run, then I see this error message:
26 | let server = hyper::Server::bind(&addr).serve(router);
| ^^^^^^ could not find `Server` in `hyper`
I must be missing something obvious about Rust and Hyper. What I am trying to do is writing something as dry/simple as possible with just the HTTP layer and some basic routes. I would like to include as little as possible 3rd party dependencies e.g avoiding Tokio which I think involves async behavior, but I am not sure about the context as I am new to Rust.
Looks like I must use futures, so I included this dependency and perhaps futures only work with the async reserved word (which I am not sure if it comes from Tokio or Rust itself).
What confuses me is that in the Hyper examples I do see imports like use hyper::{Body, Request, Response, Server};, so that Server thing must be there, somewhere.
These are the dependencies in Cargo.toml:
hyper = "0.14.12"
serde_json = "1.0.67"
futures = "0.3.17"
This is the code in main.rs:
use futures::future;
use hyper::service::service_fn;
use hyper::{Body, Method, Response, StatusCode};
use serde_json::json;
fn main() {
let router = || {
service_fn(|req| match (req.method(), req.uri().path()) {
(&Method::GET, "/foo") => {
let mut res = Response::new(
Body::from(json!({"message": "bar"}).to_string())
);
future::ok(res)
},
(_, _) => {
let mut res = Response::new(
Body::from(json!({"content": "route not found"}).to_string())
);
*res.status_mut() = StatusCode::NOT_FOUND;
future::ok(res)
}
})
};
let addr = "127.0.0.1:8080".parse::<std::net::SocketAddr>().unwrap();
let server = hyper::Server::bind(&addr).serve(router); // <== this line fails to compile
// hyper::rt::run(server.map_err(|e| {
// eprintln!("server error: {}", e);
// }));
}
How do I make the code above compile and run?
According to documentation, you are missing one module namespace in your call hyper::server::Server:
let server = hyper::server::Server::bind(&addr).serve(router)
In order to use server you need to activate the feature flag in cargo:
hyper = { version = "0.14.12", features = ["server"] }

How do I gracefully shutdown the Tokio runtime in response to a SIGTERM?

I have a main function, where I create a Tokio runtime and run two futures on it.
use tokio;
fn main() {
let mut runtime = tokio::runtime::Runtime::new().unwrap();
runtime.spawn(MyMegaFutureNumberOne {});
runtime.spawn(MyMegaFutureNumberTwo {});
// Some code to 'join' them after receiving an OS signal
}
How do I receive a SIGTERM, wait for all unfinished tasks (NotReadys) and exit the application?
Dealing with signals is tricky and it would be too broad to explain how to handle all possible cases. The implementation of signals is not standard across platforms, so my answer is specific to Linux. If you want to be more cross-platform, use the POSIX function sigaction combined with pause; this will offer you more control.
One way to achieve what you want is to use the tokio_signal crate to catch signals, like this: (doc example)
extern crate futures;
extern crate tokio;
extern crate tokio_signal;
use futures::prelude::*;
use futures::Stream;
use std::time::{Duration, Instant};
use tokio_signal::unix::{Signal, SIGINT, SIGTERM};
fn main() -> Result<(), Box<::std::error::Error>> {
let mut runtime = tokio::runtime::Runtime::new()?;
let sigint = Signal::new(SIGINT).flatten_stream();
let sigterm = Signal::new(SIGTERM).flatten_stream();
let stream = sigint.select(sigterm);
let deadline = tokio::timer::Delay::new(Instant::now() + Duration::from_secs(5))
.map(|()| println!("5 seconds are over"))
.map_err(|e| eprintln!("Failed to wait: {}", e));
runtime.spawn(deadline);
let (item, _rest) = runtime
.block_on_all(stream.into_future())
.map_err(|_| "failed to wait for signals")?;
let item = item.ok_or("received no signal")?;
if item == SIGINT {
println!("received SIGINT");
} else {
assert_eq!(item, SIGTERM);
println!("received SIGTERM");
}
Ok(())
}
This program will wait for all current tasks to complete and will catch the selected signals. This doesn't seem to work on Windows as it instantly shuts down the program.
The other answer is for Tokio version 0.1.x, which is very old. For Tokio version 1.x.y, the official Tokio tutorial has a page on this topic: Graceful shutdown

How do I set the request headers using Reqwest?

I need to make a GET request to a website with a cookie using the Reqwest library. I figured out how to send a GET request:
let response = reqwest::get("http://example.com")?;
How do I send the same request but adding some custom headers?
Reqwest 0.10
Starting at the crate's documentation, we see:
For a single request, you can use the get shortcut method.
get's documentation states:
This function creates a new internal Client on each call, and so should not be used if making many requests. Create a Client instead.
Client has a request method which states:
Returns a RequestBuilder, which will allow setting headers and request body before sending.
RequestBuilder has a header method. This can be used as:
use reqwest::header::USER_AGENT;
let client = reqwest::Client::new();
let res = client
.get("https://www.rust-lang.org")
.header(USER_AGENT, "My Rust Program 1.0")
.send()
.await?;
How do I add custom headers?
If you look at the signature for header, you will see that it accepts generic types:
pub fn header<K, V>(self, key: K, value: V) -> RequestBuilder where
HeaderName: TryFrom<K>,
<HeaderName as TryFrom<K>>::Error: Into<Error>,
HeaderValue: TryFrom<V>,
<HeaderValue as TryFrom<V>>::Error: Into<Error>,
There is an implementation of TryFrom<&'a str> for HeaderName, so you can use a string literal:
use reqwest; // 0.10.0
use tokio; // 0.2.6
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = reqwest::Client::new();
let res = client
.get("https://www.rust-lang.org")
.header("X-My-Custom-Header", "foo")
.send()
.await?;
Ok(())
}
Send Cookie in reqwest client with version ~0.9.19
use reqwest; // 0.9.19
use http::{HeaderMap, HeaderValue, header::{COOKIE}};
// create client first
let init_client = reqwest::Client::builder()
.cookie_store(true).build().unwrap();
// create Header Map
// Here cookie store is optional based on if making more than one request with the // same client
let mut headers = HeaderMap::new();
headers.insert(COOKIE, HeaderValue::from_str("key=value").unwrap());
let resp = init_client.get("api")
.headers(headers)
.query(&[("name", "foo")])
.send()
.map(|resp|{
println!("{:?}", resp.status());
resp
})
.map_err(|err|{
println!("{:?}", err);
err
});
Hope This may help.

Resources