How do I set the request headers using Reqwest? - http

I need to make a GET request to a website with a cookie using the Reqwest library. I figured out how to send a GET request:
let response = reqwest::get("http://example.com")?;
How do I send the same request but adding some custom headers?

Reqwest 0.10
Starting at the crate's documentation, we see:
For a single request, you can use the get shortcut method.
get's documentation states:
This function creates a new internal Client on each call, and so should not be used if making many requests. Create a Client instead.
Client has a request method which states:
Returns a RequestBuilder, which will allow setting headers and request body before sending.
RequestBuilder has a header method. This can be used as:
use reqwest::header::USER_AGENT;
let client = reqwest::Client::new();
let res = client
.get("https://www.rust-lang.org")
.header(USER_AGENT, "My Rust Program 1.0")
.send()
.await?;
How do I add custom headers?
If you look at the signature for header, you will see that it accepts generic types:
pub fn header<K, V>(self, key: K, value: V) -> RequestBuilder where
HeaderName: TryFrom<K>,
<HeaderName as TryFrom<K>>::Error: Into<Error>,
HeaderValue: TryFrom<V>,
<HeaderValue as TryFrom<V>>::Error: Into<Error>,
There is an implementation of TryFrom<&'a str> for HeaderName, so you can use a string literal:
use reqwest; // 0.10.0
use tokio; // 0.2.6
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let client = reqwest::Client::new();
let res = client
.get("https://www.rust-lang.org")
.header("X-My-Custom-Header", "foo")
.send()
.await?;
Ok(())
}

Send Cookie in reqwest client with version ~0.9.19
use reqwest; // 0.9.19
use http::{HeaderMap, HeaderValue, header::{COOKIE}};
// create client first
let init_client = reqwest::Client::builder()
.cookie_store(true).build().unwrap();
// create Header Map
// Here cookie store is optional based on if making more than one request with the // same client
let mut headers = HeaderMap::new();
headers.insert(COOKIE, HeaderValue::from_str("key=value").unwrap());
let resp = init_client.get("api")
.headers(headers)
.query(&[("name", "foo")])
.send()
.map(|resp|{
println!("{:?}", resp.status());
resp
})
.map_err(|err|{
println!("{:?}", err);
err
});
Hope This may help.

Related

How to send large custom struct over HTTP in Rust lang using reqwest, tokio and actix_web

Issue
I have a client that needs to send the following custom data structure to an API:
#[derive(Serialize, Deserialize)]
pub struct FheSum {
pub server_keys: ServerKey,
pub value1: FheUint8,
pub value2: FheUint8,
}
The code for the client is the following:
let fhe_post: FheSum = FheSum {
server_keys: server_keys.to_owned(),
value1: value_api.to_owned(),
value2: value_api_2.to_owned(),
};
let client = reqwest::blocking::Client::builder()
.timeout(None)
.build().unwrap();
let response = client
.post("http://127.0.0.1:8000/computesum")
.json(&fhe_post)
.send().unwrap();
let response_json: Result<FheSumResult, reqwest::Error> = response.json();
match response_json {
Ok(j) => {
let result_api: u8 = FheUint8::decrypt(&j.result, &client_keys);
println!("Final Result: {}", result_api)
},
Err(e) => println!("{:}", e),
};
In the API, I have the following definition of an HttpServer:
HttpServer::new(|| {
let json_cfg = actix_web::web::JsonConfig::default()
.limit(std::usize::MAX);
App::new()
.app_data(json_cfg)
.service(integers::computesum)
})
.client_disconnect_timeout(std::time::Duration::from_secs(3000))
.client_request_timeout(std::time::Duration::from_secs(3000))
.max_connection_rate(std::usize::MAX)
.bind(("127.0.0.1", 8000))?
.run()
.await
And the associated endpoint the client is trying to access:
#[post("/computesum")]
pub async fn computesum(req: Json<FheSum>) -> HttpResponse {
let req: FheSum = req.into_inner();
let recovered: FheSum = FheSum::new(
req.server_keys,
req.value1,
req.value2,
);
set_server_key(recovered.server_keys);
let result_api_enc: FheSumResult = FheSumResult::new(recovered.value1 + recovered.value2);
HttpResponse::Ok()
.content_type(ContentType::json())
.json(&result_api_enc)
}
Problem
The structs are the same in both the client and the server. This code works when using common data types such as Strings. The issue is when using this data structures. The memory occupied, obtained with mem::size_of_val which returns the size in bytes, is the following:
Size 1: 2488
Size 2: 32
Size 3: 32
The result has been obtained in bytes, so, given the limit established in the HttpServer, this shouldn't be an issue. Timeouts have also been set at much higher values than commonly needed.
Even with this changes, the client always shows Killed, and doesn't display the answer from the server, not giving any clues on what the problem might be.
The client is killing the process before being able to process the server's response. I want to find a way to send these custom data types to the HTTP server without the connection closing before the operation has finished.
I have already tried different libraries for the client such as the acw crate, apart from reqwest and the result is the same. I have also tried not using reqwest in blocking mode, and the error persists.

How to POST a multipart form using async version of reqwest crate?

The documentation for https://docs.rs/reqwest/latest/reqwest/blocking/multipart/struct.Form.html shows this example to create a multipart::Form from a file.
let file = reqwest::blocking::multipart::Form::new().file("key", "/path/to/file")?;
let response = reqwest::blocking::Client::new()
.post("https:://test.com.br/send")
.multipart(file)
.send()
.unwrap();
But the function ".file" is only available on the blocking version of reqwest (reqwest::blocking::multipart::Form).
I've tested the blocking version, and i was able to send a form. But i can't find a way to do this using the async version (reqwest::multipart::Form).
Is there a alternative way to make this call using the async version?
If you want to post just a file using async, please check this answer. If you want to construct a multipart form, try to use reqwest::multipart::Part to construct a body, while wrap your file into a stream.
i m successfully update file with multipart with this code(you could change patch to post)
pub async fn update_form(&self, con: &Collection, id: &str, path: &str) -> String {
let url = [&self.url_struct(con), "/", id].concat();
let file = fs::read(path).unwrap();
let file_part = reqwest::multipart::Part::bytes(file)
.file_name("bg.jpg")
.mime_str("image/jpg")
.unwrap();
let form = reqwest::multipart::Form::new().part("img", file_part);
let client = reqwest::Client::new();
match client
.patch(url)
.headers(construct_headers_form())
.multipart(form)
.send()
.await
{
Ok(res) => res.text().await.unwrap_or("no message".to_string()),
Err(_) => "{\"error\":400}".to_string(),
}
}

How to make a gRPC firestore listen request in Rust?

Using gRPC bindings from https://github.com/gkkachi/firestore-grpc I was able to puzzle together something that is seemingly working but does not receive any content:
Creating the request:
let req = ListenRequest {
database: format!("projects/{}/databases/(default)", project_id),
labels: HashMap::new(),
target_change: Some(TargetChange::AddTarget(Target {
// "Rust" in hex: https://github.com/googleapis/python-firestore/issues/51
target_id: 0x52757374,
once: false,
target_type: Some(TargetType::Documents(DocumentsTarget {
documents: vec![users_collection],
})),
resume_type: None,
})),
};
Sending it:
let mut req = Request::new(stream::iter(vec![req]));
let metadata = req.metadata_mut();
metadata.insert(
"google-cloud-resource-prefix",
MetadataValue::from_str(&db).unwrap(),
);
println!("sending request");
let res = get_client(&token).await?.listen(req).await?;
let mut res = res.into_inner();
while let Some(msg) = res.next().await {
println!("getting response");
dbg!(msg);
}
(full code in this repo).
The request can be made but the stream does not contain any actual content. The only hint I get from the debug logs is
[2021-10-27T14:54:39Z DEBUG h2::codec::framed_write] send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) }
[2021-10-27T14:54:39Z DEBUG h2::proto::connection] Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library)
Any idea what is missing?
The crucial thing I was missing as pointed out in the rust users forum was that the request stream was immediately ending which caused the connection to close. The send frame=GoAway was actually send by the client (facepalm).
To keep the connection open and receive responses we can keep the input stream pending: Request::new(stream::iter(vec![req]).chain(stream::pending())). There will be a better way to set things up and keep control over subsequent input requests but this is enough to fix the example.

Are there any options for logging request times in grpc-node?

Are there any options for logging request times in grpc-node? I've been able to log response times using opentelemetry & jaegar (to display response times). I couldn't find any npm packages for this either, but just wanted to ask you guys if you did find any options for grpc-node.
You don't need a package to do it, you can do it using a simple gRPC client interceptor.
This is how you would do it in Golang. Just check how you can create a gRPC interceptor in Node. Sorry for not having a JS example, hope it helps some how.
func UnaryRequestTImeInterceptor() grpc.UnaryClientInterceptor {
return func(
ctx context.Context,
method string,
req interface{},
reply interface{},
cc *grpc.ClientConn,
invoker grpc.UnaryInvoker,
opts ...grpc.CallOption,
) error {
start := time.Now()
err := invoker(ctx, method, req, reply, cc, opts...)
reqTime := time.Since(start)
return err
}
}

How do you make a GET request in Rust?

I noticed that Rust doesn't have a builtin library to deal with HTTP, it only has a net module that deals with raw IP and TCP protocols.
I need to take a &str of the URL, make a HTTP GET request, and if successful return either a String or &str that corresponds to the HTML or JSON or other response in string form.
It would look something like:
use somelib::http;
let response = http::get(&"http://stackoverflow.com");
match response {
Some(suc) => suc,
None => panic!
}
The current best practice for this particular problem is to use the reqwest crate, as specified in the Rust Cookbook. This code is slightly adapted from the cookbook to run standalone:
extern crate reqwest; // 0.9.18
use std::io::Read;
fn run() -> Result<(), Box<dyn std::error::Error>> {
let mut res = reqwest::get("http://httpbin.org/get")?;
let mut body = String::new();
res.read_to_string(&mut body)?;
println!("Status: {}", res.status());
println!("Headers:\n{:#?}", res.headers());
println!("Body:\n{}", body);
Ok(())
}
As the cookbook mentions, this code will be executed synchronously.
See also:
How can I perform parallel asynchronous HTTP GET requests with reqwest?
Take a look at Hyper.
Sending a GET request is as simple as this.
let client = Client::new();
let res = client.get("http://example.domain").send().unwrap();
assert_eq!(res.status, hyper::Ok);
You can find more examples in the documentation.
Edit:
It seems that Hyper got a bit more complicated since they started to use Tokio. Here is updated version.
extern crate futures;
extern crate hyper;
extern crate tokio_core;
use std::io::{self, Write};
use futures::{Future, Stream};
use hyper::Client;
use tokio_core::reactor::Core;
fn main() {
let mut core = Core::new().unwrap();
let client = Client::new(&core.handle());
let uri = "http://httpbin.org/ip".parse().unwrap();
let work =
client.get(uri).and_then(|res| {
println!("Response: {}", res.status());
res.body().for_each(|chunk| {
io::stdout()
.write_all(&chunk)
.map_err(From::from)
})
});
core.run(work).unwrap();
}
And here are the required dependencies.
[dependencies]
futures = "0.1"
hyper = "0.11"
tokio-core = "0.1"
Try to go for reqwest:
extern crate reqwest;
fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut res = reqwest::get("https://httpbin.org/headers")?;
// copy the response body directly to stdout
std::io::copy(&mut res, &mut std::io::stdout())?;
Ok(())
}
Be aware that hyper has lots of dependencies and reqwest not only depends on hyper but also brings quite a few additional dependencies. This may or may not be an issue for you, but it's something you should be aware of.
An alternative you might want to consider is minreq. That crate has only 1 dependency by default (the log crate) and a few more that depend on optional features. For example, if you activate the https-native feature, it will also add native-tls as a dependency.

Resources