Rust's Lazy init (OnceCell crate) does not work everywhere - asynchronous

I'm writing a discord bot in Rust with Serenity.
In order to populate some embed headers, I need bot-related info (user & owner, hereafter named BOT) and I'm trying to use the Lazy struct from the once_cell crate to make that happen.
The catch is that this BOT data can only be queried using async functions with serenity's Http module and I'm struggling to make that work.
I set up a few slash-commands that each have their own handler function. When I dereference BOT (this is what's supposed to evaluate it (only once), right ?) from one of those handler functions, my Lazy closure blocks on the first .await it encounters.
On the other hand, it just works when I dereference BOT from main (tokio async).
The following code therefore prints "0,1,2,3" when dereferencing BOT from main but only "0,1" when doing so from a specific slash-command handler (BoxedFuture) :
use once_cell::sync::Lazy;
use serenity::{http::Http, model::user::User};
pub struct Bot {
pub user: User,
pub owner: User,
}
pub static BOT: Lazy<Bot> = Lazy::new(|| {
println!("0");
let http_client = Http::new(env!("DISCORD_BOT_TOKEN"));
futures::executor::block_on(async {
println!("1");
let user = http_client
.get_current_user()
.await
.expect("An HTTP client error occured")
.into();
println!("2");
let owner = http_client
.get_current_application_info()
.await
.expect("An HTTP client error occured")
.owner;
println!("3");
Bot { user, owner }
})
});
Can anyone explain why that is ?

You should never block inside an async runtime. Never, ever.
You can use tokio::sync::OnceCell instead of once_cell:
pub async fn get_bot() -> &'static Bot {
static BOT: tokio::sync::OnceCell<Bot> = tokio::sync::OnceCell::const_new();
BOT.get_or_init(|| async {
println!("0");
let http_client = Http::new(env!("DISCORD_BOT_TOKEN"));
println!("1");
let user = http_client
.get_current_user()
.await
.expect("An HTTP client error occured")
.into();
println!("2");
let owner = http_client
.get_current_application_info()
.await
.expect("An HTTP client error occured")
.owner;
println!("3");
Bot { user, owner }
})
.await
}

Related

Is there some way how to shutdown tokio::runtime::Runtime?

Problem is in a microservice with Tokio, where connect to db and other stuff async, but when some connection failed, microservice dont end work. Its a great when you need this case, but I need to end work of this microservice when connection lost... so could you help me how to safety shutdown process...?
src/main.rs
use tokio; // 1.0.0+
fn main() {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(workers_number)
.enable_all()
.build()
.unwrap();
rt.block_on(async {
// health cheacker connection
let health_checker = match HealthChecker::new(some_configuration).await?;
// some connection to db
// ...
// transport client connection
// ...
// so when connection failed or lost I need to
// end process like `std::process::abort()`
// but I cant use it, because its unsafe
let mut task_handler = vec![];
// create some task
join_all(task_handler).await;
});
}
anyone have some ideas?
You can call any of the Runtime shutdown methods shutdown_timeout or shutdown_background.
If it is needed some king of waiting, you could spawn a task with a tokio::sync::oneshot that will trigger the shutdown when signaled.
use core::time::Duration;
use crate::tokio::time::sleep;
use tokio;
fn main() {
let rt = tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()
.unwrap();
let handle = rt.handle().clone();
let (s, r) = tokio::sync::oneshot::channel();
rt.spawn(async move {
sleep(Duration::from_secs(1)).await;
s.send(0);
});
handle.block_on(async move {
r.await;
rt.shutdown_background();
});
}
Playground

Why is async TcpStream blocking?

I'm working on a project to implement a distributed key value store in rust. I've made the server side code using Tokio's asynchronous runtime. I'm running into an issue where it seems my asynchronous code is blocking so when I have multiple connections to the server only one TcpStream is processed. I'm new to implementing async code, both in general and on rust, but I thought that other streams would be accepted and processed if there was no activity on a given tcp stream.
Is my understanding of async wrong or am I using tokio incorrectly?
This is my entry point:
use std::error::Error;
use std::net::SocketAddr;
use std::path::{Path, PathBuf};
use std::str::FromStr;
use std::sync::{Arc, Mutex};
use env_logger;
use log::{debug, info};
use structopt::StructOpt;
use tokio::net::TcpListener;
extern crate blue;
use blue::ipc::message;
use blue::store::args;
use blue::store::cluster::{Cluster, NodeRole};
use blue::store::deserialize::deserialize_store;
use blue::store::handler::handle_stream;
use blue::store::wal::WriteAheadLog;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();
let opt = args::Opt::from_args();
let addr = SocketAddr::from_str(format!("{}:{}", opt.host, opt.port).as_str())?;
let role = NodeRole::from_str(opt.role.as_str()).unwrap();
let leader_addr = match role {
NodeRole::Leader => addr,
NodeRole::Follower => SocketAddr::from_str(opt.follow.unwrap().as_str())?,
};
let wal_name = addr.to_string().replace(".", "").replace(":", "");
let wal_full_name = format!("wal{}.log", wal_name);
let wal_path = PathBuf::from(wal_full_name);
let mut wal = match wal_path.exists() {
true => {
info!("Existing WAL found");
WriteAheadLog::open(&wal_path)?
}
false => {
info!("Creating WAL");
WriteAheadLog::new(&wal_path)?
}
};
debug!("WAL: {:?}", wal);
let store_name = addr.to_string().replace(".", "").replace(":", "");
let store_pth = format!("{}.pb", store_name);
let store_path = Path::new(&store_pth);
let mut store = match store_path.exists() {
true => deserialize_store(store_path)?,
false => message::Store::default(),
};
let listener = TcpListener::bind(addr).await?;
let cluster = Cluster::new(addr, &role, leader_addr, &mut wal, &mut store).await?;
let store_path = Arc::new(store_path);
let store = Arc::new(Mutex::new(store));
let wal = Arc::new(Mutex::new(wal));
let cluster = Arc::new(Mutex::new(cluster));
info!("Blue launched. Waiting for incoming connection");
loop {
let (stream, addr) = listener.accept().await?;
info!("Incoming request from {}", addr);
let store = Arc::clone(&store);
let store_path = Arc::clone(&store_path);
let wal = Arc::clone(&wal);
let cluster = Arc::clone(&cluster);
handle_stream(stream, store, store_path, wal, cluster, &role).await?;
}
}
Below is my handler (handle_stream from the above). I excluded all the handlers in match input as I didn't think they were necessary to prove the point (full code for that section is here: https://github.com/matthewmturner/Bradfield-Distributed-Systems/blob/main/blue/src/store/handler.rs if it actually helps).
Specifically the point that is blocking is the line let input = async_read_message::<message::Request>(&mut stream).await;
This is where the server is waiting for communication from either a client or another server in the cluster. The behavior I currently see is that after connecting to server with client the server doesn't receive any of the requests to add other nodes to the cluster - it only handles the client stream.
use std::io;
use std::net::{SocketAddr, TcpStream};
use std::path::Path;
use std::str::FromStr;
use std::sync::{Arc, Mutex};
use log::{debug, error, info};
use serde_json::json;
use tokio::io::AsyncWriteExt;
use tokio::net::TcpStream as asyncTcpStream;
use super::super::ipc::message;
use super::super::ipc::message::request::Command;
use super::super::ipc::receiver::async_read_message;
use super::super::ipc::sender::{async_send_message, send_message};
use super::cluster::{Cluster, NodeRole};
use super::serialize::persist_store;
use super::wal::WriteAheadLog;
// TODO: Why isnt async working? I.e. connecting servers after client is connected stays on client stream.
pub async fn handle_stream<'a>(
mut stream: asyncTcpStream,
store: Arc<Mutex<message::Store>>,
store_path: Arc<&Path>,
wal: Arc<Mutex<WriteAheadLog<'a>>>,
cluster: Arc<Mutex<Cluster>>,
role: &NodeRole,
) -> io::Result<()> {
loop {
info!("Handling stream: {:?}", stream);
let input = async_read_message::<message::Request>(&mut stream).await;
debug!("Input: {:?}", input);
match input {
...
}
}
}
This is the code for async_read_message
pub async fn async_read_message<M: Message + Default>(
stream: &mut asyncTcpStream,
) -> io::Result<M> {
let mut len_buf = [0u8; 4];
debug!("Reading message length");
stream.read_exact(&mut len_buf).await?;
let len = i32::from_le_bytes(len_buf);
let mut buf = vec![0u8; len as usize];
debug!("Reading message");
stream.read_exact(&mut buf).await?;
let user_input = M::decode(&mut buf.as_slice())?;
debug!("Received message: {:?}", user_input);
Ok(user_input)
}
Your problem lies with how you're handling messages after clients have connected:
handle_stream(stream, store, store_path, wal, cluster, &role).await?;
This .await means your listening loop will wait for handle_stream to return, but (making some assumptions) this function won't return until the client has disconnected. What you want is to tokio::spawn a new task that can run independently:
tokio::spawn(handle_stream(stream, store, store_path, wal, cluster, &role));
You may have to change some of your parameter types to avoid lifetimes; tokio::spawn requires 'static since the task's lifetime is decoupled from the scope where it was spawned.

Rust - How to use a synchronous and an asynchronous crate in one application

I began writing a program using the Druid crate and Crabler crate to make a webscraping application whose data I can explore. I only realized that merging synchronous and asynchronous programming was a bad idea long after I had spent a while building this program. What I am trying to do right now is have the scraper run while the application is open (preferably every hour).
Right now the scraper doesn't run until after the application is closed. I tried to use Tokio's spawn to make a separate thread that starts before the application opens, but this doesn't work because the Crabler future doesn't have the "Send" trait.
I tried to make a minimal functional program as shown below. The title_handler doesn't function as expected but otherwise it demonstrates the issue I'm having well.
Is it possible to allow the WebScraper to run while the application is open? If so, how?
EDIT: I tried using task::spawn_blocking() to run the application and it threw out a ton of errors, including that druid doesn't implement the trait Send.
use crabler::*;
use druid::widget::prelude::*;
use druid::widget::{Align, Flex, Label, TextBox};
use druid::{AppLauncher, Data, Lens, WindowDesc, WidgetExt};
const ENTRY_PREFIX: [&str; 1] = ["https://duckduckgo.com/?t=ffab&q=rust&ia=web"];
// Use WebScraper trait to get each item with the ".result__title" class
#[derive(WebScraper)]
#[on_response(response_handler)]
#[on_html(".result__title", title_handler)]
struct Scraper {}
impl Scraper {
// Print webpage status
async fn response_handler(&self, response: Response) -> Result<()> {
println!("Status {}", response.status);
Ok(())
}
async fn title_handler(&self, _: Response, el: Element) -> Result<()> {
// Get text of element
let title_data = el.children();
let title_text = title_data.first().unwrap().text().unwrap();
println!("Result is {}", title_text);
Ok(())
}
}
// Run scraper to get info from https://duckduckgo.com/?t=ffab&q=rust&ia=web
async fn one_scrape() -> Result<()> {
let scraper = Scraper {};
scraper.run(Opts::new().with_urls(ENTRY_PREFIX.to_vec()).with_threads(1)).await
}
#[derive(Clone, Data, Lens)]
struct Init {
tag: String,
}
fn build_ui() -> impl Widget<Init> {
// Search box
let l_search = Label::new("Search: ");
let tb_search = TextBox::new()
.with_placeholder("Enter tag to search")
.lens(Init::tag);
let search = Flex::row()
.with_child(l_search)
.with_child(tb_search);
// Describe layout of UI
let layout = Flex::column()
.with_child(search);
Align::centered(layout)
}
#[async_std::main]
async fn main() -> Result<()> {
// Describe the main window
let main_window = WindowDesc::new(build_ui())
.title("Title Tracker")
.window_size((400.0, 400.0));
// Create starting app state
let init_state = Init {
tag: String::from("#"),
};
// Start application
AppLauncher::with_window(main_window)
.launch(init_state)
.expect("Failed to launch application");
one_scrape().await
}

How to force logout firebase auth user from app remotely

I have a project which uses firebase auth with firebaseUI to authenticate users. I have enabled Google, Facebook and email providers. What I need is to remotely logout or disable some of the users.
I want the users to logout from the app on doing so. I tried disabling the user in the firebase console and also used the firebase admin SDK (https://firebase.google.com/docs/auth/admin/manage-sessions) to revoke the refresh tokens.
I waited for more than 2 days and still noticed that the user was logged in and could access the firestore data.
I have also gone through and tried
Firebase still retrieving authData after deletion
Can anyone point to what I am doing wrong ?
You also cannot remotely force a user to be signed out. Any sign out will have to happen from the device that the user is signed in on.
There is no way to revoke an access token once that is minted. This means that even if you disable the user's account, they may continue to have access for up to an hour.
If that is too long, the trick (as also mentioned in my answer to the question you linked) is to maintain a list of blocked users in your database (or elsewhere) and then check against that in your security rules (or other authorization layer).
For example in the realtime database, you could create a list of blocked user's UIDs:
banned_uids: {
"uid1": true
"uid2": true
}
And then check against that in your security rules with:
".read": "auth.uid !== null && !root.child('banned_uids').child(auth.uid).exists()"
You can send a message data with FCM to force to log out.
For example, if the users use android application.
Save the FCM token in a collection in firebase Realtime.
configure the Android client app, in the service. LINK You have to make when receive a message with especial string, force to log out.
make the trigger you need in cloud functions, to send the data LINK when you need the user log out.
SUCCESS!
As per your scenarios, i assume that you need to make user logout when user is disabled.
Use One global variable to store TokenNo (might be in shared preference or sqlite):
Add following code to your manifest:
<service android:name=".YourFirebaseMessagingService">
<intent-filter>
<action android:name="com.google.firebase.MESSAGING_EVENT" />
</intent-filter>
</service>
Add following code in your
public class LogoutOntokenchange extends FirebaseMessagingService{
#Override
public void onNewToken (String token){
if(TokenNo=>1){ //if tokenNo >=1 means he already logged in
TokenNo=0;
FirebaseAuth.getInstance().signOut(); //Then call signout method
}
else{
TokenNo=1; //store token no in db
}
}
}
What Happens here:
When user logged in first time onNewToken is called then It goes into else then TokenNo is updated to 1 from 0.
When You disable any user then automatically token is refreshed.Then OnNewToken is called then TokenNo>=1 so user will be logged out.
NOTE: When user log in for first time i.e if TokenNo variable is not stored then store it as 0.
For reference: https://firebase.google.com/docs/reference/android/com/google/firebase/messaging/FirebaseMessagingService
The only way I can think about is adding a if-else block in your starting activity.
Store the that status of user (verified/banned/deleted) in Firebase Real-time database. Then retrieve the status of user at start of application and add the code:
if (currentUserStatus.equals("banned"))
{
currentUser.logout();
}
What I've done is I created for each user upon registration a Firestore document with the UID as document ID. In this document I store an array which stores all fcm tokens the individual user receives when logging into a new device. That way I always keep track where the user is logged in. When the user logs out manually the fcm token will be deleted from the document in Firestore as well as on the device.
In order to be able to log out the user everywhere they are signed in I did the following. When starting the app and once the user is logged in I start a snapshot listener that listens to all changes in the users document. As soon as there is a change I retrieve the new array of fcm tokens, search inside the array for the local current device fcm token. If found, I do nothing. If the fcm token is no longer in the array I will call the local logout method and go back to the login screen.
Here are the methods I used in swift on iOS. The closures (passOnMethod) will just trigger an unwind segue to the login view controller.
import Foundation
import Firebase
class FB_Auth_Methods {
let db = Firestore.firestore()
var listener: ListenerRegistration?
func trackLoginStatus(passOnMethod: #escaping () -> () ) {
listener?.remove()
if let loggedInUserA_UID = Auth.auth().currentUser?.uid {
listener = db.collection(K.FStore.collectionOf_RegisteredUsers_Name)
.document(loggedInUserA_UID)
.addSnapshotListener { (snapshotDocument, error) in
if let error = error {
print(error)
} else {
if let document = snapshotDocument {
if let data = document.data() {
if let fcmTokens = data[K.FStore.Users.fcmTokens] as? [String] {
print("Found the following tokens: \(fcmTokens)")
self.compareTokensAgainstCurrentDeviceToken(fcmTokens: fcmTokens, passOnMethod: { () in
passOnMethod()
})
}
}
}
}
}
}
}
func compareTokensAgainstCurrentDeviceToken(fcmTokens: [String], passOnMethod: #escaping () -> () ) {
InstanceID.instanceID().instanceID { (result, error) in
if let error = error {
print(error)
} else if let result = result {
if fcmTokens.contains(result.token) {
print("Token found, doing nothing")
} else {
print("Token no longer found, logout user")
do {
try Auth.auth().signOut()
InstanceID.instanceID().deleteID { error in
if let error = error {
print(error)
} else {
passOnMethod()
}
}
} catch let signOutError as NSError {
print (signOutError)
}
}
}
}
}
}
And here is the method I use when logging out the user everywhere but at the current device.
func deleteAllFcmTokensExceptCurrent(loggedInUserA: User, passOnMethod: #escaping () -> () ) {
InstanceID.instanceID().instanceID { (result, error) in
if let error = error {
print(error)
} else if let result = result {
let batch = self.db.batch()
let deleteAllFcmRef = self.db.collection(K.FStore.collectionOf_RegisteredUsers_Name).document(loggedInUserA.uid)
batch.updateData([K.FStore.Users.fcmTokens: FieldValue.delete()], forDocument: deleteAllFcmRef)
let updateFcmTokenRef = self.db.collection(K.FStore.collectionOf_RegisteredUsers_Name).document(loggedInUserA.uid)
batch.updateData([K.FStore.Users.fcmTokens: FieldValue.arrayUnion([result.token])], forDocument: updateFcmTokenRef)
batch.commit { (error) in
if let error = error {
print(error)
} else {
passOnMethod()
}
}
}
}
}
Not tested yet, as our backend programmer, who is in charge of setting up Firestore rules was gone for the day, but in theory this should work: (and it's something I'll test tomorrow)
Having a FirebaseAuth.AuthStateListener in charge of serving UI based on the status of the user
This combined with rules in firestore
match /collection
allow read: if isAuth();
Where isAuth is:
function isAuth() {
return request.auth.uid != null;
}
If the user is then disabled, while being logged in, whenever the user tries to read data from the collection, he should be denied, and a signOut() call should be made.
The AuthStateListener will then detect it, and sign the user out.

How to create a global mutable bool status flag

Preface: I have done my research and know that it is really not a good idea/nor is it idiomatic Rust to have one. Completely open to suggestions of other ways to solve this issue.
Background: I have a console application that connects to a websocket and once connected successfully, the server sends a "Connected" message. I have the sender, and the receiver is separate threads and all is working great. After the connect() call a loop begins and places a prompt in the terminal, signaling that the application is ready to receive input from the user.
Problem: The issue is that the current flow of execution calls connect, and then immediately displays the prompt, and then the application receives the message from the server stating it is connected.
How I would solve this in higher level languages: Place a global bool (we'll call it ready) and once the application is "ready" then display the prompt.
How I think this might look in Rust:
//Possible global ready flag with 3 states (true, false, None)
let ready: Option<&mut bool> = None;
fn main(){
welcome_message(); //Displays a "Connecting..." message to the user
//These are special callback I created and basically when the
//message is received the `connected` is called.
//If there was an error getting the message (service is down)
//then `not_connected` is called. *This is working code*
let p = mylib::Promise::new(connected, not_connected);
//Call connect and start websocket send and receive threads
mylib::connect(p);
//Loop for user input
loop {
match ready {
Some(x) => {
if x == true { //If ready is true, display the prompt
match prompt_input() {
true => {},
false => break,
}
} else {
return; //If ready is false, quit the program
}
},
None => {} //Ready is None, so continue waiting
}
}
}
fn connected() -> &mut bool{
println!("Connected to Service! Please enter a command. (hint: help)\n\n");
true
}
fn not_connected() -> &mut bool{
println!("Connection to Service failed :(");
false
}
Question:
How would you solve this issue in Rust? I have tried passing it around to all the libraries method calls, but hit some major issues about borrowing an immutable object in a FnOnce() closure.
It really sounds like you want to have two threads that are communicating via channels. Check out this example:
use std::thread;
use std::sync::mpsc;
use std::time::Duration;
enum ConsoleEvent {
Connected,
Disconnected,
}
fn main() {
let (console_tx, console_rx) = mpsc::channel();
let socket = thread::spawn(move || {
println!("socket: started!");
// pretend we are taking time to connect
thread::sleep(Duration::from_millis(300));
println!("socket: connected!");
console_tx.send(ConsoleEvent::Connected).unwrap();
// pretend we are taking time to transfer
thread::sleep(Duration::from_millis(300));
println!("socket: disconnected!");
console_tx.send(ConsoleEvent::Disconnected).unwrap();
println!("socket: closed!");
});
let console = thread::spawn(move || {
println!("console: started!");
for msg in console_rx.iter() {
match msg {
ConsoleEvent::Connected => println!("console: I'm connected!"),
ConsoleEvent::Disconnected => {
println!("console: I'm disconnected!");
break;
}
}
}
});
socket.join().expect("Unable to join socket thread");
console.join().expect("Unable to join console thread");
}
Here, there are 3 threads at play:
The main thread.
A thread to read from the "socket".
A thread to interface with the user.
Each of these threads can maintain it's own non-shared state. This allows reasoning about each thread to be easier. The threads use a channel to send updates between them safely. The data that crosses threads is encapsulated in an enum.
When I run this, I get
socket: started!
console: started!
socket: connected!
console: I'm connected!
socket: disconnected!
socket: closed!
console: I'm disconnected!

Resources