WinRT ViewModel DataBind to async method - data-binding

I am deserializing a list of objects from an XML file, and would like to bind to the actual content of those objects in my View, passing over a ViewModel. The problem is that file operations are async and this bubbles all the way up to the ViewModel, where Property getters cannot be marked as such...
Problem
I deserialize all XML files in a folder to Profile objects and store them in a List<Profile>. This method (has to be) marked async.
public static async Task<List<Profile>> GetAllProfiles()
{
DataContractSerializer ser = new DataContractSerializer(typeof(Profile));
StorageFolder folder = await ApplicationData.Current.RoamingFolder.CreateFolderAsync("Profiles", CreationCollisionOption.OpenIfExists);
List<Profile> profiles = new List<Profile>();
foreach (var f in await folder.GetFilesAsync())
{
var fs = await f.OpenStreamForReadAsync();
profiles.Add((Profile)ser.ReadObject(fs));
fs.Dispose();
}
return profiles;
}
Ideal solution 1
The binding property in my ViewModel would then ideally call that static method like this
public async Task<ObservableCollection<string>> Lists
{
get
{
return new ObservableCollection<string>(GetAllProfiles().Select(p => p.Name));
}
}
BUT Properties cannot be marked async
Ideal solution 2
public ObservableCollection<string> Lists
{
get
{
return new ObservableCollection<string>((GetAllProfiles().Result).Select(p => p.Name));
}
}
BUT this never executes (it blocks in the await folder.GetFilesAsync() call for some reason)
Current solution
Calls an async Initialize() method that loads the result of the GetProfiles() function in a variable, and then makes a NotifyPropertyChanged("Lists") call:
public ViewModel()
{
Initialize();
}
public async void Initialize()
{
_profiles = await Profile.GetAllProfiles();
NotifyPropertyChanged("Lists");
}
private List<Profile> _profiles;
public ObservableCollection<string> Lists
{
get
{
if (_profiles != null)
return new ObservableCollection<string>(_profiles.Select(p => p.Name));
else
return null;
}
}
Question
Is there a better way?
Is there a pattern/method that I haven't yet discovered?
Edit
The root of the problem appears when doing non-UI code, and you cannot rely on the NotifyPropertyChanged to do some thread-synchronization stuff. -- The method Initialize has to be awaited and ctors cannot be async, so essentialy this is pattern is useless.
public MyClass()
{
Initialize();
}
public async void Initialize()
{
_profiles = await Profile.GetAllProfiles();
}
private ObservableCollection<Profile> _profiles;
public ObservableCollection<string> Lists
{
get
{
return _profiles; // this will always be null
}
}

Properties can't be async so this solution will not work as you mentioned. Task.Result waits for the task to complete, but this is blocking your UI thread where the I/O operation's async callback returns, so you are deadlocking your application, since the callback is never called. Your solution really is the best way. It could be improved though.
You should make the _profiles field an ObservableCollection, so you would not need to convert the List to the OC every time the list is accessed.
Since you are performing an I/O operation that can take arbitrary amount of time - you should enable some sort of a progress indicator while it is in progress.
In some cases you might want the Lists property to be lazier and only call the Init method the first time it is accessed.

Related

Unity async/await and IO operations with Firestore (Firebase)

I've developed a Unity App that uses Firebase as a BaaS and Firestore as a Database.
Firebase has a Client SDK to make calls that are usually called from client to server by an URL endpoint.
My concern is how my methods should be implemented to correctly work on client without blocking the user experience, cause if I made a heavy request, my Unity App is blocked, and no interaction is allowed to the user.
This is the code of my client DatabaseManager with the methods to retrieve a User from Firestore:
public class DatabaseManager
{
public DatabaseManager(FirebaseFirestore db)
{
this.db = db;
}
public async Task<User> GetUserByUIDAsync(string uid)
{
string documentID = uid;
return await AsyncGetDocumentFromCollection<User, User_FirestoreData>(COL_ID_USERS, documentID);
}
public async Task<PlainData> AsyncGetDocumentFromCollection<PlainData, FirestoreData>(string collectionID, string documentID) where PlainData : IConvertToFirestore<FirestoreData> where FirestoreData : IConvertToPlainData<PlainData>
{
try
{
DocumentReference docRef = db.Collection(collectionID).Document(documentID);
DocumentSnapshot documentSnapshot = await docRef.GetSnapshotAsync();
if (documentSnapshot.Exists)
{
Debug.Log("Get Document data for document:" + documentSnapshot.Id);
FirestoreData firestoreData = documentSnapshot.ConvertTo<FirestoreData>();
return firestoreData.ToPlainData();
}
else
{
Debug.Log($"Document {documentSnapshot.Id} does not exist!");
}
}
catch (Exception e)
{
Debug.Log(e);
}
return default(PlainData);
}
}
This is a simple call and when it's called from any MonoBehaviouryou couldn't notice the load difference when you call it like:
using UnityEngine.UI;
public class MyMono : MonoBehaviour
{
private void DatabaseManager db;
[SerializedField] private Button button = null;
private void Awake()
{
button.onClick.AddListener(async ()=> await CustomAwakeAsync(db));
}
private async Task CustomAwakeAsync(DatabaseManager db)
{
//if this Async method is heavier, this will block the main UI thread when the button is pressed
await db.GetUserByUIDAsync("xdfipñfajrfiñar");
}
}
But if instead of GetUserByUIDAsync I make a heavy call, or multiple recursive calls my application UI will freeze until it's finished...which is bad.
How should I build my code to avoid these case?
Note:
My easy way to test if it's blocking UI thread is having this class attached to a GameObject with Image component:
using UnityEngine;
public class InfiniteRotate : MonoBehaviour
{
public float speed = 1;
// Update is called once per frame
private void Update()
{
this.gameObject.transform.Rotate(0, 0, 1 * Time.deltaTime * speed);
}
}
If the image stop spinning, means that async/await is blocking the UI thread.
Your code as shown:
private void CustomAwake(DatabaseManager db)
{
await db.GetUserByUIDAsync("xdfipñfajrfiñar");
}
...should be producing the following error:
error CS4033: The 'await' operator can only be used within an async method. Consider marking this method with the 'async' modifier and changing its return type to 'Task'.
Even if somehow you managed to silence this error the method signature private void CustomAwake(DatabaseManager db) clearly indicates that this method is synchronous to the caller.
OP:
But if instead of GetUserByUIDAsync I make a heavy call, my application UI will freeze until it's finished...which is bad.
So if you are calling this from the same thread as Update, FixedUpdate etc (which by the looks of it you are) then you are going to block Unity and thus slow down your game.
If you are going to use async/await then you need to do so all the way back to the original caller.
Make it asynchronous
Change the method to:
private async Task<User> CustomAwake(DatabaseManager db) // Note the async Task
{
return await db.GetUserByUIDAsync("xdfipñfajrfiñar");
}
...and ensure that whatever calls it uses await in order to get the User.

Lazy CosmosDB Initialization takes longer when more tasks are waiting for it

Context
We have a service that is dependent on CosmosDB. We created a class, having a lazy container, that will be initialized on startup. In the startup class we do :
CreateDatabaseIfNotExist
CreateContainerIfNotExistsAsync
Problem
The first request to CosmosDB starts the initialization.
When we have multiple threads starting up before the initialization, waiting for this lazy intialization to finish, the intialization takes longer the more threads are waiting for it.
Expected
When multiple threads starting up, the threads that need to have this initialized container, should not impact the initialization duration, since this is in a locked context (lazy)
In the code example below, when changing the amount of threads to 5, the initialization is in a couple of seconds. the higher the count of threads, the higher the duration of the initialization.
code example:
using System;
using System.Diagnostics;
using System.Threading.Tasks;
using Microsoft.Azure.Cosmos;
namespace LazyCosmos.Anon
{
class Program
{
static void Main(string[] args)
{
new Do().Run().GetAwaiter().GetResult();
}
public class Do
{
private Lazy<Container> lazyContainer;
private Container Container => lazyContainer.Value;
public Do()
{
lazyContainer = new Lazy<Container>(() => InitializeContainer().GetAwaiter().GetResult());
}
public async Task Run()
{
try
{
var tasks = new Task[100];
for (int i = 0; i < 100; i++)
{
tasks[i] = Task.Run(() =>
ReadItemAsync<Item>("XXX", "XXX"));
}
await Task.WhenAll(tasks);
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
}
public async Task<T> ReadItemAsync<T>(string id, string partitionKey)
{
var itemResponse = await Container.ReadItemAsync<T>(id, new PartitionKey(partitionKey));
return itemResponse.Resource;
}
private async Task<Container> InitializeContainer()
{
var s = Stopwatch.StartNew();
Console.WriteLine($"Started {s.ElapsedMilliseconds}s");
var configuration = new CosmosDbServiceConfiguration("XXX", null, collectionId: "XXX",
"XXX", 400);
var _cosmosClient = new ColdStorageCosmosClient(new ActorColdStorageConfiguration("XXX", "XXX", "https://XXX.XX", "XXX"));
var database = await _cosmosClient
.CreateDatabaseIfNotExistsAsync(configuration.DatabaseId, configuration.DatabaseThroughput);
Console.WriteLine($"CreateDatabaseIfNotExistsAsync took {s.ElapsedMilliseconds}s");
var containerProperties = new ContainerProperties
{
Id = configuration.ContainerId,
PartitionKeyPath = $"/{configuration.PartitionKey}",
DefaultTimeToLive = configuration.DefaultTimeToLive
};
var db = (Database)database;
var containerIfNotExistsAsync = await db.CreateContainerIfNotExistsAsync(containerProperties, configuration.ContainerThroughput);
s.Stop();
Console.WriteLine($"CreateContainerIfNotExistsAsync took {s.ElapsedMilliseconds}s");
return containerIfNotExistsAsync;
}
}
}
public class CosmosDbServiceConfiguration
{
public CosmosDbServiceConfiguration(string databaseId, int? databaseThroughput, string collectionId, string partitionKey, int? containerThroughput = null)
{
DatabaseId = databaseId;
ContainerId = collectionId;
DatabaseThroughput = databaseThroughput;
ContainerThroughput = containerThroughput;
PartitionKey = partitionKey;
}
public string DatabaseId { get; }
public int? DatabaseThroughput { get; }
public string ContainerId { get; }
public int? ContainerThroughput { get; }
public string PartitionKey { get; }
public int? DefaultTimeToLive { get; set; }
}
public class ColdStorageCosmosClient : CosmosClient
{
public ColdStorageCosmosClient(ActorColdStorageConfiguration actorColdStorageConfiguration) : base(actorColdStorageConfiguration.EndpointUrl, actorColdStorageConfiguration.Key)
{
}
}
public class ActorColdStorageConfiguration
{
public ActorColdStorageConfiguration(string databaseName, string collectionName, string endpointUrl, string key)
{
DatabaseName = databaseName;
CollectionName = collectionName;
EndpointUrl = endpointUrl;
Key = key;
}
public string DatabaseName { get; }
public string CollectionName { get; }
public string EndpointUrl { get; }
public string Key { get; }
}
public class Item
{
public string id { get; set; }
}
}
You're experiencing thread pool exhaustion. There's a few different concepts that are conflicting to cause the exhaustion.
First, even though asynchronous code does not use a thread for the duration of the asynchronous operation, it often does need to very briefly borrow a thread pool thread in order to do housework when the asynchronous operation completes. As a result, most asynchronous code only runs efficiently if there is a free thread pool thread available, and if there are no thread pool threads available, then asynchronous code may be delayed.
Another part of the puzzle is that the thread pool has a limited thread injection rate. This is deliberate, so that the thread pool isn't constantly creating/destroying threads as its load varies. That would be very inefficient. Instead, a thread pool that has all of its threads busy (and still has more work to do) will only add a thread every few seconds.
The final concept to recognize is that Lazy<T> is blocking when using the default LazyThreadSafetyMode.ExecutionAndPublication behavior. The way this Lazy<T> works is that only one thread executes the delegate (() => InitializeContainer().GetAwaiter().GetResult()). All other threads block, waiting for that delegate to complete.
So now, putting it all together:
A large number of work items are placed onto the thread pool work queue (by Task.Run). The thread pool begins executing only as many work items as it has threads.
Each of these work items accesses the Container (i.e., Lazy<Container>.Value), so each one of these work items blocks a thread until the initialization is complete. Only the first work item accessing Container will run the initialization code.
The (asynchronous) initialization code attempts to make progress, but it needs a thread pool thread to be free in order to handle housekeeping when its awaits complete. So it is also queueing very small work items to the thread pool as necessary.
The thread pool has more work than it can handle, so it begins adding threads. Since it has a limited thread injection rate, it will only add a thread every few seconds.
The thread pool is overwhelmed with work, but it can't know which work items are the important ones. Most of its work items will just block on the Lazy<T>, which uses up another thread. The thread pool cannot know which work items are the ones from the asynchronous initialization code that will free up the other work items (and threads). So most of the threads added by the thread pool just end up blocking on other work that is having a hard time to complete since there are no thread pool threads available.
So, let's talk solutions.
IMO, the easiest solution is to remove (most of) the blocking. Allow the initialization to be asynchronous by changing the lazy type from Lazy<Container> to Lazy<Task<Container>>. The Lazy<Task<T>> pattern is "asynchronous lazy initialization", and it works by Lazy-initializing a task.
The Lazy<T> part of Lazy<Task<T>> ensures that only the first caller begins executing the asynchronous initialization code. As soon as that asynchronous code yields at an await (and thus returns a Task), the Lazy<T> part is done. So the blocking of other threads is very brief.
Then all the work items get the same Task<T>, and they can all await it. A single Task<T> can be safely awaited any number of times. Once the asynchronous initialization code is complete, the Task<T> gets a result, and all the awaiting work items can continue executing. Any future calls to the Lazy<Task<T>>.Value will immediately get a completed Task<T> which takes no time at all to await since it is already completed.
Once you wrap your head around Lazy<Task<T>>, it's pretty straightforward to use. The only awkward part is that the code for the work items now have to await the shared asynchronous initialization:
public class Do
{
private Lazy<Task<Container>> lazyContainer;
private Task<Container> ContainerTask => lazyContainer.Value;
public Do()
{
lazyContainer = new Lazy<Task<Container>>(InitializeContainer);
}
public async Task<T> ReadItemAsync<T>(string id, string partitionKey)
{
// This is the awkward part. Until you get used to it. :)
var container = await ContainerTask;
var itemResponse = await container.ReadItemAsync<T>(id, new PartitionKey(partitionKey));
return itemResponse.Resource;
}
// other methods are unchanged.
}
I have an AsyncLazy<T> type in my AsyncEx library, which is essentially the same as Lazy<Task<T>> with a few usability enhancements.
More information on this pattern:
Asynchronous lazy initialization blog post.
Recipe 14.1 "Initializing Shared Resources" in my book Concurrency in C# Cookbook, 2nd edition.
The Lazy<Task<T>> asynchronous lazy initialization pattern works great if you have a widely shared resource that may or may not need to be initialized. If you have a local resource (like a private member as in this example), and if you know you will always want it initialized, then you can make the code simpler by just using Task<T> instead of Lazy<Task<T>>:
public class Do
{
private Task<Container> ContainerTask;
public Do()
{
// Important semantic change:
// This begins initialization *immediately*.
// It does not wait for work items to request the container.
ContainerTask = InitializeContainer();
}
public async Task<T> ReadItemAsync<T>(string id, string partitionKey)
{
var container = await ContainerTask;
var itemResponse = await container.ReadItemAsync<T>(id, new PartitionKey(partitionKey));
return itemResponse.Resource;
}
// other methods are unchanged.
}

Application hangs while trying to perform asynchronous operations with SQLite in Xamarin

In a Xamarin app I'm having an intermittent issue, where it will close out of nowhere. After examining the code and output log, and googling around a bit, I figured that there was too much data load on the UI thread. Summarizing, this works fine:
public List<T> GetList(Expression<Func<T, bool>> expression)
{
return Connection.Table<T>().Where(expression).ToList();
}
However, the application hangs while trying to call an async version of the method:
public async Task<List<T>> GetList(Expression<Func<T, bool>> expression)
{
return await AsyncConnection.Table<T>().Where(expression).ToListAsync();
}
I also tried to be fancy, but the result was the same:
public async Task<List<T>> GetList(Expression<Func<T, bool>> expression)
{
return await Task.Factory.StartNew(() =>
{ return Connection.Table<T>().Where(expression).ToList(); });
}
It is used like so:
var lst = GetList(x => x.EntityName.Equals(entityMobile.EntityName)).Result
As per Stephen Cleary, this situation incurs a deadlock. His blog post goes into detail and presents two solutions. I opted for the latter in my case (just stick to async all the way):
public MyDAO
{
// ...
public async Task<List<T>> GetListAsync(Expression<Func<T, bool>> expression)
{
return await AsyncConnection.Table<T>().Where(expression).ToListAsync();
}
}
public MyViewModel
{
private MyDAO _myDAO;
//...
public async void DoAsyncStuff()
{
var myList = await _myDao.GetListAsync(x => x.EntityName.Equals(entityMobile.EntityName));
}
}
This is a simplification of what I actually have, but I suppose it will do as a broad example. More of these can be found in his article.

rxJava Observer.onNext not called second time

I am using rxJava to fetch data from the database and show it in a recyclerview. The relevant code is shown below
function updateUI(){
ContactsLab contactsLab = ContactsLab.get(getActivity());
Subscription sub = contactsLab.getContactList().subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.toList()
.subscribe(onContactsReceived());
mCompositeSubscription.add(sub);
}
ContactsLab is a singleton that returns an Observable of Contact objects.
onContactsReceived function is shown below
private Observer<List<Contact>> onContactsReceived(){
return new Observer<List<Contact>>() {
#Override
public void onCompleted() {}
#Override
public void onError(Throwable e) {}
#Override
public void onNext(List<Contact> contacts) {
if(mContactsAdapter == null) {
mContactsAdapter = new ContactsAdapter(contacts);
mRecyclerView.setAdapter(mContactsAdapter);
} else{
mContactsAdapter.setContactList(contacts);
mContactsAdapter.notifyDataSetChanged();
}
}
};
}
The updateUI function is called in my fragment onResume but the view is updated only the first time. If I come back to this fragment from any other fragment (having added more items to db), onResume is called, updateUI runs and onContactsReceived also runs but returns immediately without calling onNext or onComplete.
I think this has something to do with the way rxJava handles observables but no idea how to fix it (read about defer but couldn't understand much). Can somebody please help?
Edit:
The getContactList function look like this :
public rx.Observable<Contact> getContactList() {
List<Contact> contacts = new ArrayList<>();
ContactCursorWrapper cursorWrapper = queryContacts(null, null);
try{
cursorWrapper.moveToFirst();
while (!cursorWrapper.isAfterLast()){
contacts.add(cursorWrapper.getContact());
cursorWrapper.moveToNext();
}
} finally {
cursorWrapper.close();
}
return rx.Observable.from(contacts);
}
Basically it queries the database and maps the returned Cursor into my Contact class(which is a POJO). I added the rx.Observable.from to get an observable that was later collated using toList and updated into the adapter.
I used this approach avoid having to call notifyDataSetChanged after getting each item (and call it only once after getting all that).
What's the right approach to minimize the number of notifyDataSetChanged calls and also, refresh each time onResume is called?
Your observable contactsLab.getContactList().toList() has terminated.toList() collects all emissions from a source observable to a list and emits the entire list once the source Observable terminates (see the documentation). You aren't going to observe any more emissions from it.

Using "async" (even if it should complete) as part of a MVC route deadlocks the route; how can this be avoided?

Consider the following (based on the default MVC template), which is a simplified version of some "stuff" that happens in the background - it completes fine, and shows the expected result, 20:
public ActionResult Index()
{
var task = SlowDouble(10);
string result;
if (task.Wait(2000))
{
result = task.Result.ToString();
}
else
{
result = "timeout";
}
ViewBag.Message = result;
return View();
}
internal static Task<long> SlowDouble(long val)
{
TaskCompletionSource<long> result = new TaskCompletionSource<long>();
ThreadPool.QueueUserWorkItem(delegate
{
Thread.Sleep(50);
result.SetResult(val * 2);
});
return result.Task;
}
However, now if we add some async into the mix:
public static async Task<long> IndirectSlowDouble(long val)
{
long result = await SlowDouble(val);
return result;
}
and change the first line in the route to:
var task = IndirectSlowDouble(10);
then it does not work; it times out instead. If we add breakpoints, the return result; in the async method only happens after the route has already completed - basically, it looks like the system is unwilling to use any thread to resume the async operation until after the request has finished. Worse: if we had used .Wait() (or accessed .Result), then it will totally deadlock.
So: what is with that? The obvious workaround is "don't involve async", but that is not easy when consuming libraries etc. Ultimately, there is no functional difference between SlowDouble and IndirectSlowDouble (although there is obvious a structural difference).
Note: the exact same thing in a console / winform / etc will work fine.
It's to do with the way the synchronization context is implemented in ASP.NET (Pre .NET 4.5). There's tons of questions about this behavior:
Task.WaitAll hanging with multiple awaitable tasks in ASP.NET
Asp.net SynchronizationContext locks HttpApplication for async continuations?
In ASP.NET 4.5, there's a new implementation of the sync context that's described in this article.
http://blogs.msdn.com/b/webdev/archive/2012/11/19/all-about-httpruntime-targetframework.aspx
When you use .Result there is always a possibility of deadlock because .Result is blocking by nature. The way to avoid deadlocks is to not block on Tasks (you should use async and await all the way down). The subject is in details described here:
Don't Block on Async Code
One fix is to add ConfigureAwait:
public static async Task<long> IndirectSlowDouble(long val)
{
long result = await SlowDouble(val).ConfigureAwait(false);
return result;
}
Another fix is to use async/await throughout:
public async Task<ActionResult> Index()
{
var task = IndirectSlowDouble(10);
long result = await task;
ViewBag.Message = result.ToString();
return View();
}

Resources