I tried to sign up for the "Analytical Storage" preview feature in the Azure Portal for my Cosmos DB account in North Europe. However it has been stuck in "Pending" state for two weeks now. Is this feature still available and if so how can I get it enabled?
Also how is the analytical storage accessed? According to https://learn.microsoft.com/en-us/azure/cosmos-db/globally-distributed-transactional-analytical-storage it's “Apache Spark”. Does this refer to the Cosmos DB Spark Connector or how do you query the underlying Parquet files? Also will it be possible to move the Parquet files to external storage (e.g. ADLS) for consumption by other BI tools?
Analytical Storage is currently a gated preview. This should go to public preview sometime this Summer.
The storage itself is fully managed so you will not be able to access the parquet files or storage directly. The only way to access this will be through the Synapse Analytics service.
thanks.
In Build session #BRK3060 Mark Russinovich demos some code that uses both the SQL and Graph APIs on the same database (starts at 45:27):
https://www.youtube.com/watch?v=S2zguwKvlQk
Does anyone have any insight into these read/write using multiple APIs on the same database?
I'm teaching myself to use Flutter and I'm making an app that queries The Movie Database API. Currently, I'm having the client query the API on launch but I'm thinking this is not the most efficient way of doing it, and I would rather have the client query a backend service like Firebase to get the same data.
I would appreciate some guidance into where to start in order to setup a periodical process to query the API and use the results as entries into a Firestore DB. I've looked online but I might be using suboptimal keywords since I haven't found a good tutorial or example for this.
Thanks.
You can use Firebase Cloud Functions to build code that runs on Firebase servers to fill your Firebase database, but you can only make HTTP requests to non-Google addresses if you use a paid plan.
https://firebase.googleblog.com/2017/03/how-to-schedule-cron-jobs-with-cloud.html explains how to invoke periodic tasks with Cloud Functions. It utilizes Google AppEngine for that because Cloud Functions doesn't provide that out of the box.
I have built an app using firestore as we are interested in the realtime updates portion of things. However,we are not building a website that has CRM component where a lot of reports will be generated. The contents of that CRM are all new. There is only one report that would need firebase data as well as the new data (you can say 1 report out of 20).
I was thinking of building the CRM backend off mysql DB? Do you recommend to go with this approach or shall I do the CRM in the same firebase/firestore db?
Thanks
If you are looking for a real-time backend database for your CRM, then the Firebase RTDB / Cloud Firestore would be ideal for this. I'm not sure why you'd want to add a mySQL component, unless you are going to create some reports that require complex joins. However, if your data is modelled correctly, this also shouldn't be an issue.
Take a look at this video to get a better understanding: What is a NoSQL Database? How is Cloud Firestore structured? - Get to Know Cloud Firestore Ep.1
Google announced Firestore, the new document datastore on the block.
I have been developing an application using Google Cloud Datastore for over six months now and after reading the blog, I feel Firestore seems to be a better choice.
The concept of the alternate collection-document-subcollection looks excellent to me because while designing schema for datastore I was aware I will be unable to query nested fields. Now with firestore subcollections, I get full query capabilities which is a game changer for me (I can get maximum data with minimum queries).
As a counter argument, the flowchart suggests me to use datastore because I do not have any mobile clients.
Will it be a good idea to use Firestore just like Datastore ?
(I will conveniently ignore the mobile client/realtime updates/syncing features!)
Update 2 (01/31/19)
As of today, Cloud Firestore is no longer in Beta and is Generally Available:
https://cloud.google.com/blog/products/databases/announcing-cloud-firestore-general-availability-and-updates
This means that Cloud Datastore is no longer an option for new projects (you can keep using it on existing projects). New projects that want to use the Datastore API can use Cloud Firestore in Datastore mode.
Update 1
As you we have noticed, we've expanded Cloud Firestore since this question was posted.
This means Cloud Firestore now has 2 modes:
The original launch was 'Native mode'
The new launch adds 'Datastore mode'
'Datastore mode' is the 3rd gen of Cloud Datastore. 1st was called Master/Slave Datastore, 2nd was High-Replication Datastore (HRD) that was rebranded as Cloud Datastore in 2013.
The below answer is still largely relevant since both modes are currently mutually exclusive, so you need to pick one or the other.
The main differences are the improves of Cloud Firestore in Datastore mode over Cloud Datastore. The biggest ones are:
Write through-put per entity group now unbounded (was 1 write/second)
Transactions no longer limited to 25 entity groups
All queries now strongly consistent.
Also note Cloud Firestore regardless of mode is beta, so the new Service-Level Agreement (SLA) doesn't go into effect until the product reaches General Availability (GA).
Original Answer
Cloud Datastore (CD) and Cloud Firestore (CF) are similar, however different in significant ways.
CF is mobile-centric with direct from mobile client functionality with the Firebase SDKs and Rules functionality. CD is server-centric with a wider range of server client libraries, as well as some mature frameworks on App Engine Standard that bundle in memcache functionality.
CF has a newer storage layer that is strongly consistency in the same way as Cloud Spanner, however, it's still in beta without an SLA. CD's storage layer is only strongly consistent within entity-groups and eventually consistent across entity-groups, however, it is GA with a 99.95% SLA for the Multi-Region locations.
CF is only available in the US Multi-Region at this time. CD is available Cloud across a dozen locations including places in the Americas, Europe, Asia, and Australia.
CF during beta has a guideline limit of 2500 writes/second while we build experience monitoring and tuning the system prior to GA, whereas CD will happily handle >1M writes/second (please reach out to your account rep first though).
CF and CD's set of query capabilities are overlapping but not the same. Overall CD has a broader set of query capabilities we haven't built in CF yet, so you'd have more flexibility in CD.
Overall, I'd consider this list to see if any of the differences make or break what you're trying to build then pick the DB that fits closest to your needs.
Firestore is the 3rd generation architecture and replacement for Datastore, essentially available in 2 modes: Native mode and Datastore mode.
Documentation regarding the choices: https://cloud.google.com/datastore/docs/firestore-or-datastore
Video overview: https://www.youtube.com/watch?v=SYG-BgXoJFQ
I'd say that Datastore is now a subset of Firestore:
Cloud Firestore is the next major version of Cloud Datastore and a re-branding of the product.
See Choosing between Cloud Firestore and Cloud Datastore
Cloud Firestore can operate in "Datastore mode", making it backwards- compatible with Cloud Datastore. Some time after Cloud Firestore is released for general availability, Google will begin contacting owners of existing Cloud Datastore databases to schedule an automatic upgrade to Cloud Firestore in Datastore mode. See auto upgrade
Google documentation says:
Firestore is the new version of Datastore and removes several
Datastore limitations.
I think cloud firestore also has nodejs client and its not centric to mobile. Actually thats the difference between Firebase realtime database which was mobile-centri vs Cloud Firestore which is anything centric.