we are building a system where changes in Firestore produce PubSub-message on which other components may act.
Firestore events are used to implement this. The system keeps a history of changes for each document as subcollection of the document. To reduce the size of the PubSub-messages we are only sending references to documents in this subcollection. One before and one after the change. To allow our function to not waste time with queries to figure out what the last state was we are using the timestamps on the change events as ids.
env.ts:
import * as functions from 'firebase-functions';
import {handler} from '#handlers/rootCollection';
//noinspection JSUnusedGlobalSymbols
export const handlerFunction = functions.firestore
.document('rootcollection/{documentID}')
.onWrite(handler.process.bind(handler));
rootCollection.ts:
import {DocumentSnapshot} from 'firebase-functions/lib/providers/firestore';
import {Change, EventContext} from 'firebase-functions';
interface MessageI extends Object {
eventID: string;
ref: string;
beforeID: string | null;
afterID: string | null;
fcfReceived: number | null;
}
class WriteHandler {
public name: string;
public constructor(name: string) {
this.name = name;
}
public async process(change: Change<DocumentSnapshot>, context: EventContext): MessageI {
const afterExists = change.after.exists;
const beforeExists = change.before.exists;
const afterID = afterExists ? change.after.updateTime.toMillis().toString() : null;
const beforeID = beforeExists ? change.before.updateTime.toMillis().toString() : null;
return {
'eventID': context.eventId,
'ref': change.before.ref.path,
'beforeID': beforeID,
'afterID': afterID,
'fcfReceived': Date.now()
};
}
}
export const itinerariesHandler = new WriteHandler(
'rootCollectionHandler'
);
Now we are pretty surprised that the data is correctly reflecting the before and after states but the timestamp seems to be the after.timestamp.
Published message 429345402459000 to topic. Message:
{
"eventID": "389fe158-115a-43ca-9fdd-ec9737af066f-0",
"ref": "collection/VMDcsHirbB2coyaXF5wZ",
"beforeID": "1552298315479",
"afterID": "1552298315479",
"fcfReceived": 1552298315575,
}
It seems counterintuitive that change.before.updateTime is containing the time when the change was done. But i cannot find a explicit mention in the documentation.
Is this behavior intended? Is there another way to find a unique id for the before state?
Regards,
Carsten
firebaser here
This is bug in the Firestore bindings for Cloud functions. It indeed sets the before.updateTime and after.updateTime to the same value.
We filed a bug internally to get this fixed. But there is no timeline for when this fix will be available.
Until that time the best I can think of is to add an lastUpdated field to the document data, and validate that is the current server-side timestamp in your security rules. That way you can get the before.data().lastUpdated and after.data().lastUpdated to do the comparison. This should at least allow you to implement the behavior, until the problem is fixed.
Related
I want to enable my users to set certain global colors when using the app. Therefor I have created a 'dynamicVariables.css' file:
:root {
--my-color: violet;
}
It is imported in 'global.scss' file:
#import "./theme/dynamicVariables.css";
Also, I've added a colorpicker on one page and I can set the --my-color variable fine from there.
onColorChange(data: any) {
document.documentElement.style.setProperty('--my-color', data);
}
Just when closing the app on my device (I've deployed it with ionic capacitor run android), it resets the css variable, because when I run it again the color is back to its default value.
I'm pretty sure, I have a general misconception here and would be grateful for some clarification. I'm generally new to web development and would be grateful for any help.
Thanks in advance.
just like how Mustafa explained in comments, you need to make these changes outside app "runtime" and in the device's memory, that would stay there even after the app (whether web or native) is closed. for example you can use ionic storage and save your data with keys and values same as your css keys, and load it up whenever the app opens.
Thanks to the responds, I was able to solve the problem with the help of Ionic Storage.
First, I created a Storage Service:
import { Storage } from '#ionic/storage-angular';
import { Injectable } from '#angular/core';
#Injectable({
providedIn: 'root'
})
export class StorageService {
private _storage: Storage | null = null;
constructor(private storage: Storage) {
}
async init() {
const storage = await this.storage.create();
this._storage = storage;
}
public set(key: string, value: any) {
this._storage?.set(key, value);
}
public get(key: string) {
return this._storage?.get(key);
}
}
When starting the app, I run the following code in the app.component.ts
async ngOnInit() {
await this.storageService.init();
let storedPathologicalColor = await this.storageService.get('--my-color');
if (storedPathologicalColor == null)
document.documentElement.style.setProperty('--my-color', getComputedStyle(document.documentElement).getPropertyValue('--my-color'))
else
document.documentElement.style.setProperty('--my-color', storedPathologicalColor);
}
It is important to init() the service from outside. When setting a new css variable, I also set a new key/value pair to the Storage.
Thanks again.
I'm new about this library and I'm trying to understand some basic concept/use cases.
The idea I made about this library is the ability to define a Model that can have custom method to collect the logic related to the data inside a single place.
Let's suppose I have a model of a book (the same that is in the README, reported below for clarity):
class Book extends Model {
toString() {
return `Book: ${this.name}`;
}
// Declare any static or instance methods you need.
}
Book.modelName = 'Book';
// Declare your related fields.
Book.fields = {
id: attr(), // non-relational field for any value; optional but highly recommended
name: attr(),
};
export default Book;
in my idea, in the component, I want to make something:
const bookSelector = createSelector(orm.Book);
function MyBookComponent({bookId}){
const book = useSelector(state => bookSelector(state, bookId))
return book.toString();
}
but this doesn't work because the bookSelector returns the ref of the book and not the instance model.
I've seen this is explicitly made at https://github.com/redux-orm/redux-orm/blob/master/src/selectors/FieldSelectorSpec.js#L29
Is there any use case that I miss / some simplification I can do, to access the model instance using that selector?
The solution I've found now is to use
const bookSelector = createSelector(
orm,
(state, bookId) => bookId,
(session, bookId) => return session.Book.withId(bookId),
);
but this approach looks like a bit verbose. Is it my use case a not so common usecase?
The documentation stresses that I should use a new EntityManager for each request and there's even a middleware for automatically generating it or alternatively I can use em.fork(). So far so good.
The EntityRepository is a great way to make the code readable. I could not find anything in the documentation about how they relate to EntityManager instances. The express-ts-example-app example uses single instances of repositories and the RequestContext middleware. This suggests that there is some under-the-hood magic that finds the correct EntityManager instances at least with the RequestContext. Is it really so?
Also, if I fork the EM manually can it still find the right one? Consider the following example:
(async () => {
DI.orm = await MikroORM.init();
DI.em = DI.orm.em;
DI.companyRepository = DI.orm.em.getRepository(Company);
DI.invoiceRepository = DI.orm.em.getRepository(Invoice);
...
fetchInvoices(em.fork());
}
async function fetchInvoices(em) {
for (const company of await DI.companyRepository.findAll()) {
fetchInvoicesOfACompany(company, em.fork())
}
}
async function fetchInvoicesOfACompany(company, em) {
let done = false;
while (!done) {
const invoice = await getNextInvoice(company.taxnumber, company.lastInvoice);
if ( invoice ) {
DI.invoiceRepository.persist(invoice);
company.lastInvoice = invoice.id;
em.flush();
} else {
done = true;
}
}
}
Does the DI.invoiceRepository.persist() in fetchInvoicesOfACompany() use the right EM instance? If not, what should I do?
Also, if I'm not mistaken, the em.flush() in fetchInvoicesOfACompany() does not update company, since that belongs to another EM - how should I handle situations like this?
First of all, repository is just a thin layer on top of EM (an extension point if you want), that bares the entity name so you don't have to pass it to the first parameter of EM method (e.g. em.find(Ent, ...) vs repo.find(...).
Then the contexts - you need a dedicated context for each request, so it has its own identity map. If you use RequestContext helper, the context is created and saved via domain API. Thanks to this, all the methods that are executed inside the domain handler will use the right instance automatically - this happens in the em.getContext() method, that first checks the RequestContext helper.
https://mikro-orm.io/docs/identity-map/#requestcontext-helper-for-di-containers
Check the tests for better understanding of how it works:
https://github.com/mikro-orm/mikro-orm/blob/master/tests/RequestContext.test.ts
So if you use repositories, with RequestContext helper it will work just fine as the singleton repository instance will use the singleton EM instance that will then use the right request based instance via em.getContext() where approapriate.
But if you use manual forking instead, you are responsible use the right repository instance - each EM fork will have its own one. So in this case you can't use a singleton, you need to do forkedEm.getRepository(Ent).
Btw alternatively you can also use AsyncLocalStorage which is faster (and not deprecated), if you are on node 12+. The RequestContext helper implementation will use ALS in v5, as node 12+ will be requried.
https://mikro-orm.io/docs/async-local-storage
Another thing you could do is to use the RequestContext helper manually instead of via middlewares - something like the following:
(async () => {
DI.orm = await MikroORM.init();
DI.em = DI.orm.em;
DI.companyRepository = DI.orm.em.getRepository(Company);
DI.invoiceRepository = DI.orm.em.getRepository(Invoice);
...
await RequestContext.createAsync(DI.em, async () => {
await fetchInvoices();
})
});
async function fetchInvoices() {
for (const company of await DI.companyRepository.findAll()) {
await fetchInvoicesOfACompany(company)
}
}
async function fetchInvoicesOfACompany(company) {
let done = false;
while (!done) {
const invoice = await getNextInvoice(company.taxnumber, company.lastInvoice);
if (invoice) {
company.lastInvoice = invoice; // passing entity instance, no need to persist as `company` is managed entity and this change will be cascaded
await DI.em.flush();
} else {
done = true;
}
}
}
I'm using Firestore database to store a list of objects. To retrieve them I use the Stream provided by the Firestore package, like this:
class FirestoreApi implements Api {
FirestoreApi._();
static final instance = FirestoreApi._();
#override
Stream<List<Job>> getJobList() {
final path = "users/myUserId/jobs";
final reference = Firestore.instance.collection(path);
final snapshots = reference.snapshots();
return snapshots.map((snapshot) => snapshot.documents.map(
(snapshot) => Job(
id: snapshot.data['uid'],
name: snapshot.data['name']
),
).toList());
}
}
It implements an abstract class:
abstract class Api {
Stream<List<Job>> getJobList();
}
In my Repository class I call it like this:
class Repository {
final FirestoreApi _firestoreApi = FirestoreApi.instance;
Stream<List<job>> getJobList() => _firestoreApi.getJobList();
}
Then in my BloC I call the Repository:
class JobBloc {
final _repository = new Repository();
Stream<List<Job>> getJobList() {
try {
return _repository.getJobList();
} catch (e) {
rethrow;
} finally {}
}
}
And finally here is how I use it in my Widget:
Widget _buildBody(BuildContext context) {
final JobBloc _jobBloc = Provider.of<JobBloc>(context);
return StreamBuilder<List<Job>>(
stream: _jobBloc.getJobList(),
builder: (BuildContext context, AsyncSnapshot<List<Job>> snapshot) {
if (snapshot.hasData) {
return RefreshIndicator(
child: JobList(snapshot.data),
onRefresh: () => _jobBloc.refreshJobList(),
);
} else {
if(snapshot.connectionState == ConnectionState.waiting) {
return Center(child: CircularProgressIndicator());
} else {
return Center(child: Text("No data"));
}
}
},
);
}
Until here everything works great and my Widget gets updated in real time when something is changed in the Firestore database.
But now I want to go one step further. Lets say that maybe in the future I need to change my api implementation and use a REST api instead of Firestore. I want that my code is prepared for that.
In that case, all the getJobList() methods should return a Future<List<Job>> since the API will not return a Stream (I don't know if that's possible).
I would have another API class like this that now returns Future<List<Job>>:
class RestApi implements Api {
RestApi._();
static final instance = RestApi._();
#override
Future<List<Job>> getJobList() {
//TODO: my rest api implementation
}
}
So the API abstract class would be modified like this:
abstract class Api {
Future<List<Job>> getJobList();
}
Here the updated Repository:
class Repository {
final RestApi _restApi = RestApi.instance;
Future<List<job>> getJobList() => _restApi.getJobList();
}
And finally in my BloC I would sink the list returned by the API in a StreamController like this:
class JobBloc {
final StreamController _jobController = StreamController<List<Job>>.broadcast();
// retrieve data from stream
Stream<List<Job>> get jobList => _jobController.stream;
Future<List<Job>> getJobList() async {
try {
_jobController.sink.add(await _repository.getJobList());
} catch (e) {
rethrow;
} finally {}
}
}
Now the question: I really like that Firestore returns a Stream, it makes my app to be updated in real time. But on the other hand, I would like that my architecture is consistent.
Since I cannot make my REST api to return a Stream, I think the only way possible would be converting the Firebase Stream to a Future but then I would loose the real-time update feature.
Something like this:
class FirestoreApi implements Api {
FirestoreApi._();
static final instance = FirestoreApi._();
#override
Future<List<Job>> getJobList() async {
final path = "users/myUserId/jobs";
final reference = Firestore.instance.collection(path);
final snapshots = reference.snapshots();
Stream<List<Job>> jobs = snapshots.map((snapshot) => snapshot.documents.map(
(snapshot) => Job(
id: snapshot.data['uid'],
name: snapshot.data['name'],
),
).toList());
List<Job> future = await jobs.first;
return future;
}
}
Until now what I've researched is that using the Future will return only one response, so I will lose the real-time functionality.
I would like to know if loosing the real-time feature would be worthy just to make the architecture consistent or if there is a better approach.
Thanks in advance, any ideas or suggestion will be appreciated.
EDIT: Thanks a lot for your comments, I really appreciate them. I actually don't know which one should be marked as accepted answer since all of them have helped me a lot so I decided to give a positive vote to all of you. If anyone doesn't agree with that or this is not the right behaviour in Stackoverflow please let me know
First of all, in my opinion, firebase is not designed to back up a mature project. In the end, you'll end up with a REST api backing up your app. It's true that, you might also end up using both but for different purposes. So i think you should think about firebase as a tool for MVP/proof of concept. I know that Firebase is cool and works well, etc. but the costs are not feasible for a final product.
Now, nobody says that you can't have a REST client implementation that will return a Stream. Check out this Stream.fromFuture(theFuture). You can think of the REST api like a stream that emits only one event (Rx equivalent: Single)
I would also advise to be careful with the real time update feature provided by Firebase, if you transition to a full REST api, you won't be able to do a real time update because REST doesn't work like that. Firebase is using Sockets for communication (if I remember correctly).
I recommended use the Future way, if you take a break and compare the two codes, with the Future way you need to write more, but the architecture is more clean, strong and scalable. In my experience, that's the right way to do good things. Great work
You can also include both methods in the api / repository, and either retrieve a Future or listen to the Stream in the bloc depending on what you want to do. I don't think you need to worry about violating the consistency of REST by also having a method that returns a stream. There is no better way to tap into the real-time functionality of Firestore than to use a stream like you described.
But to just return a Future, you don't have to go through a stream, you can just await a CollectionReference's getDocuments(), something like this:
class FirestoreApi implements Api {
FirestoreApi._();
static final instance = FirestoreApi._();
CollectionReference jobsReference = Firestore.instance.collection("users/myUserId/jobs");
#override
Future<List<Job>> getJobList() async {
QuerySnapshot query = await jobsReference.getDocuments();
List<Job> jobs = query.documents.map((document) => Job(
id: document.data['uid'],
name: document.data['name'],
)).toList();
return jobs;
}
}
It all depends on your app I think. If real time update is an important feature that effects user experience a lot, stick with the Firebase data streams. If real time updates are not a must, you can get data once using Futures. An alternative to Firebase for real time data updates could be GraphQL subscriptions. I would recommend you to check out Hasura for quick implementation of GraphQL API.
It's a good question.
Firestore vs REST API will result in different APIs (Stream vs Future).
Making the code generic won't work here. As you said:
Stream-based APIs will be realtime
Future-based APIs will not
Even the UX would be different.
In the Stream version, you don't need a refresh indicator.
In the Future version, you can reload the data with pull-to-refresh.
I would not recommend to future-proof your code in this case.
If Firestore works well for you, use Streams in all your APIs.
Only if/when you decide to move to a REST API, then you can convert all your APIs (and UX) to use Futures.
Giving up realtime capabilities upfront doesn't seem worth it.
How modify the following code to get article data and top articles asynchronously in hack ?
class ArticleController
{
public function viewAction()
{
// how get
$article = $this->getArticleData();
$topArticles = $this->getTopArticles();
}
private function getArticleData() : array
{
// return article data from db
}
private function getTopArticles() : array
{
// return top articles from db
}
}
The warning from the async documentation page is relevant here:
There is currently basic support for async. For example, you can
currently write basic async functions that call other async functions.
However, we are currently finalizing other foundations (e.g. async
database, scheduling, and memory handling APIs) which will be required
to reach the full potential of async in production. We felt, though,
it would be useful to introduce the concept and technology of async
(even with basic functionality) in order to get developers used to the
syntax and some of the technical details.
So, the raw database queries you need to actually make use of async functions are unfortunately not available yet. The documentation linked above talks some about how async functions work in general, and includes an example of coalesced fetching, something that you can do with async functions right now.
The DB API is coming eventually, but isn't available yet, sorry!
HHVM 3.6 and newer
async functions info
The two HHVM PHP language keywords that enable async functions are async and await. async declares a function as asynchronous. await suspends the execution of an async function until the result of the asynchronous operation represented by await is available. The return value of a function that await can be used upon is an object that implements Awaitable<T>.
You have an example in the documentation (1). There is a discussion about asynchronous functions in the language specification as well (2).
It actually took me some time to realize how to use and call the asynchronous functions, so I think you will find some more info useful.
We have these two functions: foo() and bar().
async function foo(): Awaitable<void> {
print "executed from foo";
}
async function bar(int $n): Awaitable<int> {
print "executed from bar";
return $n+1;
}
Let's experiment some ways to call these two functions:
foo(); // will print "executed from foo"
bar(15); // will print "executed from bar"
$no1 = bar(15); // will print "executed from bar"
print $no1; // will output error, because $number is not currently an `int`; it is a `WaitHandle`
$no2 = bar(15)->join(); // will print "executed from bar"
print $no2; // will print 16
AsyncMysqlClient tips
The connection to a MySQL database is made with AsyncMysqlClient::connect asynchronous function which returns an ExternalThreadEventWaitHandle to an AsyncMysqlConnection.
You can perform query or queryf on an AsyncMysqlConnection. Note: the data you send to a queryf is properly escaped by the function.
A query you perform on an AsyncMysqlConnection returns either an AsyncMysqlQueryResult (when the query performs ok) or AsyncMysqlQueryErrorResult (if the query goes wrong; then you can treat errors with the mysql_error(), mysql_errno() and failureType() members of this class). Both AsyncMysqlQueryResult and AsyncMysqlQueryErrorResult extend AsyncMysqlResult abstract class.
Below is a probable implementation of your class:
class ArticleController {
private AsyncMysqlConnection $connection;
public async function viewAction(int $articleId): Awaitable<void> {
$this->connection = await AsyncMysqlClient::connect( /* connection data */ );
$article = await $this->getArticleData($articleId);
}
public async function getArticleData(int $id): Awaitable<?Vector> {
$articleDataQuery = await $this->connection->queryf("SELECT * FROM articles WHERE id %=d", $id);
if($articleDataQuery instanceof AsyncMysqlQueryErrorResult) {
throw new Exception("Error on getting data: ".$articleDataQuery->mysql_error());
}
// Considering that $id represents a unique id in your database, then
// you are going to get only one row from your database query
// so you return the first (and only) row in the query result
if($articleDataQuery->numRows() == 1) {
return $articleDataQuery->mapRowsTyped()[0];
}
return null;
}
}
P.S. I hope it is not too late for this answer and I hope it helps you. If you consider this useful, please, accept it.