I tried to sign up for the "Analytical Storage" preview feature in the Azure Portal for my Cosmos DB account in North Europe. However it has been stuck in "Pending" state for two weeks now. Is this feature still available and if so how can I get it enabled?
Also how is the analytical storage accessed? According to https://learn.microsoft.com/en-us/azure/cosmos-db/globally-distributed-transactional-analytical-storage it's “Apache Spark”. Does this refer to the Cosmos DB Spark Connector or how do you query the underlying Parquet files? Also will it be possible to move the Parquet files to external storage (e.g. ADLS) for consumption by other BI tools?
Analytical Storage is currently a gated preview. This should go to public preview sometime this Summer.
The storage itself is fully managed so you will not be able to access the parquet files or storage directly. The only way to access this will be through the Synapse Analytics service.
thanks.
Related
I created Azure CosmosDb database and container for my documents.
I use NiFi as a main data ingestion tool and want to feed my container with documents from NiFi flow files.
Can anybody please share a way to post flowfile content to Azure Cosmos Db from NiFi?
Thanks in advance
UPDATE(2019.05.26):
In the end I used Python script and called it from NiFi to post messages. I passed a message as a parameter. The reason I chose python is because it has some examples on official Microsoft site with all the required connection settings and libraries, so it was easy to connect to Cosmos.
I tried Mongo component, but couldn't connect to Azure (security config didn't work), didn't really go too far with it as Python script worked just fine.
Azure CosmosDB exposes MongoDB API so you can use the following MongoDB processors which are available in NiFi to read/query/write to & from Azure CosmosDB using Apache NiFi.
DeleteMongo
GetMongo
PutMongo
PutMongoRecord
RunMonogAggregation
Useful Links
https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-introduction
https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-feature-support
Valeria. According to the components list supported by Apache Nifi related to Azure, you could only get Azure Blob Storage, Queue Storage, Event Hub etc,not including Cosmos DB.
So,I suggest you using PutAzureBlobStorage to feed azure blob container with documents from NiFi flow files. Then please create a copy activity pipeline in Azure Data Factory to transfer data from Azure Blob Storage into Azure Cosmos DB.
I am using firebase database in my Android app. The download limit for the database is 10 GB/month. If I upgraded my app to Blaze plan and made another database will each one of them will be able to download up to10 GB?
If I split the data between them will the data be able to be downloaded up to 20GB/month for free?
firebaser here
The free quota applies to a project, not to a database. Even if you switch to the Blaze plan and create multiple databases, only the first 10GB of outbound traffic will be free.
Multiple Database in Firebase is only available in Blaze plan. And in Blaze plan, there is no limit so no need to worry about the 10GB/month. See their Pricing page for more details.
Is there any tool to monitor the data transfer to my real time database in firebase? To see how many user connect at live time and how much data is transferred?
There's a usage tab in your Firebase Database console that has most of that information.
We also just released a tool that allows you to profile your database usage. See that tool's blog post and reference documentation.
I learnt about real-time data storage and hosting storage from this post Difference between Data Storage and Hosting Storage?
But i am still not clear about real time Database and and newly introduced file storage.
Does anybody have some brief explanation about it ?
Thanks in advance .
(As per the concern about duplicated with Difference between Data Storage and Hosting Storage? what the problem solves and what i am asking are two different things and hosting storage and file storage are different in case of google firebase )
Firebase now offers these places to store your data:
Realtime Database
Remote Config
Hosting
Storage
Cloud Firestore
The best place to store your data, depends on the type of data you want to store and the way you want to consume it.
The Firebase documentation says this about it:
The Firebase Realtime Database stores JSON application data, like game state or chat messages, and synchronizes changes instantly across all connected devices.
Firebase Remote Config stores developer-specified key-value pairs to change the behavior and appearance of your app without requiring users to download an update.
Firebase Hosting hosts the HTML, CSS, and JavaScript for your website as well as other developer-provided assets like graphics, fonts, and icons.
Firebase Storage stores files such as images, videos, and audio as well as other user-generated content.
On choosing between Cloud Firestore and the Firebase Realtime Database, the Firebase documentation says:
Firebase offers two cloud-based, client-accessible database solutions that support realtime data syncing:
Realtime Database is Firebase's original database. It's an efficient, low-latency solution for mobile apps that require synced states across clients in realtime.
Cloud Firestore is Firebase's new flagship database for mobile app development. It improves on the successes of the Realtime Database with a new, more intuitive data model. Cloud Firestore also features richer, faster queries and scales better than the Realtime Database.
*Realtime database store data only json format and it is specially used in app where data is synchronized concurrently like ola app(user location),sensex(Nifty) app where data not persist .
*Firebase Storage just only store data like memory card.It is specially used for store backend data of app.
I learnt about real-time data storage and hosting storage from this post Difference between Data Storage and Hosting Storage?
But i am still not clear about real time Database and and newly introduced file storage.
Does anybody have some brief explanation about it ?
Thanks in advance .
(As per the concern about duplicated with Difference between Data Storage and Hosting Storage? what the problem solves and what i am asking are two different things and hosting storage and file storage are different in case of google firebase )
Firebase now offers these places to store your data:
Realtime Database
Remote Config
Hosting
Storage
Cloud Firestore
The best place to store your data, depends on the type of data you want to store and the way you want to consume it.
The Firebase documentation says this about it:
The Firebase Realtime Database stores JSON application data, like game state or chat messages, and synchronizes changes instantly across all connected devices.
Firebase Remote Config stores developer-specified key-value pairs to change the behavior and appearance of your app without requiring users to download an update.
Firebase Hosting hosts the HTML, CSS, and JavaScript for your website as well as other developer-provided assets like graphics, fonts, and icons.
Firebase Storage stores files such as images, videos, and audio as well as other user-generated content.
On choosing between Cloud Firestore and the Firebase Realtime Database, the Firebase documentation says:
Firebase offers two cloud-based, client-accessible database solutions that support realtime data syncing:
Realtime Database is Firebase's original database. It's an efficient, low-latency solution for mobile apps that require synced states across clients in realtime.
Cloud Firestore is Firebase's new flagship database for mobile app development. It improves on the successes of the Realtime Database with a new, more intuitive data model. Cloud Firestore also features richer, faster queries and scales better than the Realtime Database.
*Realtime database store data only json format and it is specially used in app where data is synchronized concurrently like ola app(user location),sensex(Nifty) app where data not persist .
*Firebase Storage just only store data like memory card.It is specially used for store backend data of app.