I'm using Firestore database to store a list of objects. To retrieve them I use the Stream provided by the Firestore package, like this:
class FirestoreApi implements Api {
FirestoreApi._();
static final instance = FirestoreApi._();
#override
Stream<List<Job>> getJobList() {
final path = "users/myUserId/jobs";
final reference = Firestore.instance.collection(path);
final snapshots = reference.snapshots();
return snapshots.map((snapshot) => snapshot.documents.map(
(snapshot) => Job(
id: snapshot.data['uid'],
name: snapshot.data['name']
),
).toList());
}
}
It implements an abstract class:
abstract class Api {
Stream<List<Job>> getJobList();
}
In my Repository class I call it like this:
class Repository {
final FirestoreApi _firestoreApi = FirestoreApi.instance;
Stream<List<job>> getJobList() => _firestoreApi.getJobList();
}
Then in my BloC I call the Repository:
class JobBloc {
final _repository = new Repository();
Stream<List<Job>> getJobList() {
try {
return _repository.getJobList();
} catch (e) {
rethrow;
} finally {}
}
}
And finally here is how I use it in my Widget:
Widget _buildBody(BuildContext context) {
final JobBloc _jobBloc = Provider.of<JobBloc>(context);
return StreamBuilder<List<Job>>(
stream: _jobBloc.getJobList(),
builder: (BuildContext context, AsyncSnapshot<List<Job>> snapshot) {
if (snapshot.hasData) {
return RefreshIndicator(
child: JobList(snapshot.data),
onRefresh: () => _jobBloc.refreshJobList(),
);
} else {
if(snapshot.connectionState == ConnectionState.waiting) {
return Center(child: CircularProgressIndicator());
} else {
return Center(child: Text("No data"));
}
}
},
);
}
Until here everything works great and my Widget gets updated in real time when something is changed in the Firestore database.
But now I want to go one step further. Lets say that maybe in the future I need to change my api implementation and use a REST api instead of Firestore. I want that my code is prepared for that.
In that case, all the getJobList() methods should return a Future<List<Job>> since the API will not return a Stream (I don't know if that's possible).
I would have another API class like this that now returns Future<List<Job>>:
class RestApi implements Api {
RestApi._();
static final instance = RestApi._();
#override
Future<List<Job>> getJobList() {
//TODO: my rest api implementation
}
}
So the API abstract class would be modified like this:
abstract class Api {
Future<List<Job>> getJobList();
}
Here the updated Repository:
class Repository {
final RestApi _restApi = RestApi.instance;
Future<List<job>> getJobList() => _restApi.getJobList();
}
And finally in my BloC I would sink the list returned by the API in a StreamController like this:
class JobBloc {
final StreamController _jobController = StreamController<List<Job>>.broadcast();
// retrieve data from stream
Stream<List<Job>> get jobList => _jobController.stream;
Future<List<Job>> getJobList() async {
try {
_jobController.sink.add(await _repository.getJobList());
} catch (e) {
rethrow;
} finally {}
}
}
Now the question: I really like that Firestore returns a Stream, it makes my app to be updated in real time. But on the other hand, I would like that my architecture is consistent.
Since I cannot make my REST api to return a Stream, I think the only way possible would be converting the Firebase Stream to a Future but then I would loose the real-time update feature.
Something like this:
class FirestoreApi implements Api {
FirestoreApi._();
static final instance = FirestoreApi._();
#override
Future<List<Job>> getJobList() async {
final path = "users/myUserId/jobs";
final reference = Firestore.instance.collection(path);
final snapshots = reference.snapshots();
Stream<List<Job>> jobs = snapshots.map((snapshot) => snapshot.documents.map(
(snapshot) => Job(
id: snapshot.data['uid'],
name: snapshot.data['name'],
),
).toList());
List<Job> future = await jobs.first;
return future;
}
}
Until now what I've researched is that using the Future will return only one response, so I will lose the real-time functionality.
I would like to know if loosing the real-time feature would be worthy just to make the architecture consistent or if there is a better approach.
Thanks in advance, any ideas or suggestion will be appreciated.
EDIT: Thanks a lot for your comments, I really appreciate them. I actually don't know which one should be marked as accepted answer since all of them have helped me a lot so I decided to give a positive vote to all of you. If anyone doesn't agree with that or this is not the right behaviour in Stackoverflow please let me know
First of all, in my opinion, firebase is not designed to back up a mature project. In the end, you'll end up with a REST api backing up your app. It's true that, you might also end up using both but for different purposes. So i think you should think about firebase as a tool for MVP/proof of concept. I know that Firebase is cool and works well, etc. but the costs are not feasible for a final product.
Now, nobody says that you can't have a REST client implementation that will return a Stream. Check out this Stream.fromFuture(theFuture). You can think of the REST api like a stream that emits only one event (Rx equivalent: Single)
I would also advise to be careful with the real time update feature provided by Firebase, if you transition to a full REST api, you won't be able to do a real time update because REST doesn't work like that. Firebase is using Sockets for communication (if I remember correctly).
I recommended use the Future way, if you take a break and compare the two codes, with the Future way you need to write more, but the architecture is more clean, strong and scalable. In my experience, that's the right way to do good things. Great work
You can also include both methods in the api / repository, and either retrieve a Future or listen to the Stream in the bloc depending on what you want to do. I don't think you need to worry about violating the consistency of REST by also having a method that returns a stream. There is no better way to tap into the real-time functionality of Firestore than to use a stream like you described.
But to just return a Future, you don't have to go through a stream, you can just await a CollectionReference's getDocuments(), something like this:
class FirestoreApi implements Api {
FirestoreApi._();
static final instance = FirestoreApi._();
CollectionReference jobsReference = Firestore.instance.collection("users/myUserId/jobs");
#override
Future<List<Job>> getJobList() async {
QuerySnapshot query = await jobsReference.getDocuments();
List<Job> jobs = query.documents.map((document) => Job(
id: document.data['uid'],
name: document.data['name'],
)).toList();
return jobs;
}
}
It all depends on your app I think. If real time update is an important feature that effects user experience a lot, stick with the Firebase data streams. If real time updates are not a must, you can get data once using Futures. An alternative to Firebase for real time data updates could be GraphQL subscriptions. I would recommend you to check out Hasura for quick implementation of GraphQL API.
It's a good question.
Firestore vs REST API will result in different APIs (Stream vs Future).
Making the code generic won't work here. As you said:
Stream-based APIs will be realtime
Future-based APIs will not
Even the UX would be different.
In the Stream version, you don't need a refresh indicator.
In the Future version, you can reload the data with pull-to-refresh.
I would not recommend to future-proof your code in this case.
If Firestore works well for you, use Streams in all your APIs.
Only if/when you decide to move to a REST API, then you can convert all your APIs (and UX) to use Futures.
Giving up realtime capabilities upfront doesn't seem worth it.
Related
The documentation stresses that I should use a new EntityManager for each request and there's even a middleware for automatically generating it or alternatively I can use em.fork(). So far so good.
The EntityRepository is a great way to make the code readable. I could not find anything in the documentation about how they relate to EntityManager instances. The express-ts-example-app example uses single instances of repositories and the RequestContext middleware. This suggests that there is some under-the-hood magic that finds the correct EntityManager instances at least with the RequestContext. Is it really so?
Also, if I fork the EM manually can it still find the right one? Consider the following example:
(async () => {
DI.orm = await MikroORM.init();
DI.em = DI.orm.em;
DI.companyRepository = DI.orm.em.getRepository(Company);
DI.invoiceRepository = DI.orm.em.getRepository(Invoice);
...
fetchInvoices(em.fork());
}
async function fetchInvoices(em) {
for (const company of await DI.companyRepository.findAll()) {
fetchInvoicesOfACompany(company, em.fork())
}
}
async function fetchInvoicesOfACompany(company, em) {
let done = false;
while (!done) {
const invoice = await getNextInvoice(company.taxnumber, company.lastInvoice);
if ( invoice ) {
DI.invoiceRepository.persist(invoice);
company.lastInvoice = invoice.id;
em.flush();
} else {
done = true;
}
}
}
Does the DI.invoiceRepository.persist() in fetchInvoicesOfACompany() use the right EM instance? If not, what should I do?
Also, if I'm not mistaken, the em.flush() in fetchInvoicesOfACompany() does not update company, since that belongs to another EM - how should I handle situations like this?
First of all, repository is just a thin layer on top of EM (an extension point if you want), that bares the entity name so you don't have to pass it to the first parameter of EM method (e.g. em.find(Ent, ...) vs repo.find(...).
Then the contexts - you need a dedicated context for each request, so it has its own identity map. If you use RequestContext helper, the context is created and saved via domain API. Thanks to this, all the methods that are executed inside the domain handler will use the right instance automatically - this happens in the em.getContext() method, that first checks the RequestContext helper.
https://mikro-orm.io/docs/identity-map/#requestcontext-helper-for-di-containers
Check the tests for better understanding of how it works:
https://github.com/mikro-orm/mikro-orm/blob/master/tests/RequestContext.test.ts
So if you use repositories, with RequestContext helper it will work just fine as the singleton repository instance will use the singleton EM instance that will then use the right request based instance via em.getContext() where approapriate.
But if you use manual forking instead, you are responsible use the right repository instance - each EM fork will have its own one. So in this case you can't use a singleton, you need to do forkedEm.getRepository(Ent).
Btw alternatively you can also use AsyncLocalStorage which is faster (and not deprecated), if you are on node 12+. The RequestContext helper implementation will use ALS in v5, as node 12+ will be requried.
https://mikro-orm.io/docs/async-local-storage
Another thing you could do is to use the RequestContext helper manually instead of via middlewares - something like the following:
(async () => {
DI.orm = await MikroORM.init();
DI.em = DI.orm.em;
DI.companyRepository = DI.orm.em.getRepository(Company);
DI.invoiceRepository = DI.orm.em.getRepository(Invoice);
...
await RequestContext.createAsync(DI.em, async () => {
await fetchInvoices();
})
});
async function fetchInvoices() {
for (const company of await DI.companyRepository.findAll()) {
await fetchInvoicesOfACompany(company)
}
}
async function fetchInvoicesOfACompany(company) {
let done = false;
while (!done) {
const invoice = await getNextInvoice(company.taxnumber, company.lastInvoice);
if (invoice) {
company.lastInvoice = invoice; // passing entity instance, no need to persist as `company` is managed entity and this change will be cascaded
await DI.em.flush();
} else {
done = true;
}
}
}
Now this maybe good practice or a complete no no!!
I was trying to resist placing more StreamBuilder(s) under build(BuildContext context) and tried to use initState() instead. I am having trouble due to not using Future/async/await correctly. The String _leaseTenantName (first initState() Firestore.instance) would have correct value but Strings _leaseUnitName & _leaseUnitPropertyUid (second initState() Firestore.instance) usually would return as null. StreamBuilder<PropertyDetails> below build would give the error message 'Invalid document reference. Document references must have an even number of segments, but properties has 1, null)' but kept trying and eventually worked when _leaseUnitPropertyUid finally had a value.
I believe the solution is to somehow wrap the two initState() Firestore.instances in a Future/async/await but could not work out a way to do this. Any ideas?? Or should I just use yet more nested StreamBuilders?
class _LeaseTileState extends State<LeaseTile> {
String _leaseTenantName = '';
String _leaseUnitPropertyUid = '';
String _leaseUnitName = '';
String _leasePropertyName = '';
String _leasePropertyUnitName = '';
#override
void initState() {
super.initState();
Firestore.instance
.collection("companies")
.document(widget.leaseDetails.tenantUid)
.snapshots()
.listen((snapshot) {
_leaseTenantName = snapshot.data['companyName'];
});
Firestore.instance
.collection("units")
.document(widget.leaseDetails.unitUid)
.snapshots()
.listen((snapshot) {
_leaseUnitName = snapshot.data['unitName'];
_leaseUnitPropertyUid = snapshot.data['propertyUid'];
});
}
#override
Widget build(BuildContext context) {
final user = Provider.of<User>(context);
return StreamBuilder<PropertyDetails>(
stream: DatabaseServices(propertyUid: _leaseUnitPropertyUid)
.propertyByDocumentID,
builder: (context, userCompany) {
if (!userCompany.hasData) return Loading();
_leasePropertyName = userCompany.data.propertyName;
_leasePropertyUnitName = '$_leasePropertyName - $_leaseUnitName';
return Card(
That's a big no no.
Firstly, there's nothing wrong in using multiple StreamBuilder, StreamBuilder help you simplify the usage of Streams so you don't end up messing things up with their subscriptions... like you did in initState().
When you call listen() on snapshots() as you did on initState(), you created a subscription, that subscription should be canceled on dispose(), but you don't cancel it, so you are leaking memory right there, a StreamBuilder would saved you here as it manages this for you.
Another thing to keep in mind is that you are using _leaseUnitPropertyUid on build(), but you don't check if _leaseUnitPropertyUid is valid. _leaseUnitPropertyUid is only going to be set after the Firebase snapshot() Stream emits one value and build() could be called before that. Again, StreamBuilder would have saved you here as well as you could check if it has emitted a value or not.
Also you are hardcoding the Firebase.instance on your code, which makes it very hard to test. Take a look on Dependency Injection and try to inject the Firebase.instance onto your class, like a Repository pattern or something similar, so you can swap the Firebase.instace for a testing Mock and make your code more testable.
I'm working in ASP.NET 4.6 with EF 6 to a new Web Api 2 project, I've read a lot of articles that describes the async/await pattern and It's clear that for a lot of requests this pattern increase performance for a lot of reasons.
So I decided to use it for the first time.
In decided to create a Biz project and a DAL project here an example.
This is controller method:
[HttpGet]
public async Task<bool> CheckValueValidity(string value, string type)
{
return await _accountsBiz.CheckValueTypeValidity(value, type);
}
I instance biz object in controller constructor.
This is the BIZ:
public async Task<bool> CheckValueTypeValidity(string value, string type)
{
bool isValid = false;
switch (type.ToLower())
{
case "email":
isValid = await _accountsRepository.CheckEmailValidity(value);
break;
case "username":
isValid = await _accountsRepository.CheckUserNameValidity(value);
break;
}
return isValid;
}
I instance the repository in the BIZ constructor.
Finally this is the DAL method:
public async Task<bool> CheckEmailValidity(string email)
{
using(MyEntities db = new MyEntities())
{
return await db.AspNetUsers.CountAsync(u => u.Email == email) > 0 ? false : true;
}
}
And all it seems to work perfectly until I made two request at same time, one of these return the correct value the other say
A second operation started on this context before a previous asynchronous operation completed. Use 'await' to ensure that any asynchronous operations have completed before calling another method on this context. Any instance members are not guaranteed to be thread safe.
I made the requests from two different client so the context should be different. should....
I don't know how to fix it. I read a lot and my code seems to be correct.
The question is WHY throw this error? I can't understand
Thanks for help in advice.
Ok found the error in the execution flow there is a method that use DbContext without using block.... My mistake but now it's clear that context must be disposed each request!
How modify the following code to get article data and top articles asynchronously in hack ?
class ArticleController
{
public function viewAction()
{
// how get
$article = $this->getArticleData();
$topArticles = $this->getTopArticles();
}
private function getArticleData() : array
{
// return article data from db
}
private function getTopArticles() : array
{
// return top articles from db
}
}
The warning from the async documentation page is relevant here:
There is currently basic support for async. For example, you can
currently write basic async functions that call other async functions.
However, we are currently finalizing other foundations (e.g. async
database, scheduling, and memory handling APIs) which will be required
to reach the full potential of async in production. We felt, though,
it would be useful to introduce the concept and technology of async
(even with basic functionality) in order to get developers used to the
syntax and some of the technical details.
So, the raw database queries you need to actually make use of async functions are unfortunately not available yet. The documentation linked above talks some about how async functions work in general, and includes an example of coalesced fetching, something that you can do with async functions right now.
The DB API is coming eventually, but isn't available yet, sorry!
HHVM 3.6 and newer
async functions info
The two HHVM PHP language keywords that enable async functions are async and await. async declares a function as asynchronous. await suspends the execution of an async function until the result of the asynchronous operation represented by await is available. The return value of a function that await can be used upon is an object that implements Awaitable<T>.
You have an example in the documentation (1). There is a discussion about asynchronous functions in the language specification as well (2).
It actually took me some time to realize how to use and call the asynchronous functions, so I think you will find some more info useful.
We have these two functions: foo() and bar().
async function foo(): Awaitable<void> {
print "executed from foo";
}
async function bar(int $n): Awaitable<int> {
print "executed from bar";
return $n+1;
}
Let's experiment some ways to call these two functions:
foo(); // will print "executed from foo"
bar(15); // will print "executed from bar"
$no1 = bar(15); // will print "executed from bar"
print $no1; // will output error, because $number is not currently an `int`; it is a `WaitHandle`
$no2 = bar(15)->join(); // will print "executed from bar"
print $no2; // will print 16
AsyncMysqlClient tips
The connection to a MySQL database is made with AsyncMysqlClient::connect asynchronous function which returns an ExternalThreadEventWaitHandle to an AsyncMysqlConnection.
You can perform query or queryf on an AsyncMysqlConnection. Note: the data you send to a queryf is properly escaped by the function.
A query you perform on an AsyncMysqlConnection returns either an AsyncMysqlQueryResult (when the query performs ok) or AsyncMysqlQueryErrorResult (if the query goes wrong; then you can treat errors with the mysql_error(), mysql_errno() and failureType() members of this class). Both AsyncMysqlQueryResult and AsyncMysqlQueryErrorResult extend AsyncMysqlResult abstract class.
Below is a probable implementation of your class:
class ArticleController {
private AsyncMysqlConnection $connection;
public async function viewAction(int $articleId): Awaitable<void> {
$this->connection = await AsyncMysqlClient::connect( /* connection data */ );
$article = await $this->getArticleData($articleId);
}
public async function getArticleData(int $id): Awaitable<?Vector> {
$articleDataQuery = await $this->connection->queryf("SELECT * FROM articles WHERE id %=d", $id);
if($articleDataQuery instanceof AsyncMysqlQueryErrorResult) {
throw new Exception("Error on getting data: ".$articleDataQuery->mysql_error());
}
// Considering that $id represents a unique id in your database, then
// you are going to get only one row from your database query
// so you return the first (and only) row in the query result
if($articleDataQuery->numRows() == 1) {
return $articleDataQuery->mapRowsTyped()[0];
}
return null;
}
}
P.S. I hope it is not too late for this answer and I hope it helps you. If you consider this useful, please, accept it.
I need to call a method returning a Task from within
public override void OnActionExecuting(ActionExecutingContext filterContext)
It wont let me make this method async it throws the following
An asynchronous module or handler completed while an asynchronous
operation was still pending.
and when calling
entityStorage.GetCurrentUser().Result
I get a deadlock. How can I avoid this?
I have been playing around with it coming up with stuff like
entityStorage.GetCurrentUser().Result.ConfigureAwait(false).GetAwaiter().GetResult();
But this isn't working. How can I do it? My solution will need to work with ASP.NET 4 and the Async Targetting Pack, I can't use ASP.NET 4.5 as am deploying to Azure.
The cause of the deadlock is explained here. In short, don't block on async code. You should use ConfigureAwait(false) in your library async code and await the results (not use Result or Wait).
Update: Please vote here for the MVC team to add support for async action filters.
Since await is just syntax sugar for the compiler rewriting a continuation for you, the most 'direct' path would be to take whatever code was going to follow your await and make it a ContinueWith call.
So, something like:
entityStorage.GetCurrentUser().ContinueWith(t =>
{
// do your other stuff here
});
If you MUST convert asynch to synch.
public User GetCurrentUserSynch()
{
return Task.Run(() =>
{
var asyncResult = entityStorage.GetCurrentUser();
while (!asyncResult.IsCompleted)
{
Application.Current.TryFindResource(new object()); // This is for WPF but you can do some other nonsense action of your choosing
}
return asyncResult.Result;
}).Result;
}
Otherwise use #Stephen's answer.