I tested setup Azure Purview and CosmosDB with Provisioned throughput capacity/ serverless capacity mode SQL API with few collections. After Cosmos Data source registration, I initiated scan and provided key vault with secret pointing to CosmosDB account key.
For CosmosDB with Provisioned throughput capacity mode, data assets were discovered successfully.
For capacity mode - serverless CosmosDB SQL API, collection items were not identified as data assets after scan completion.
For CosmosDB with Provisioned throughput capacity mode, data assets were discovered successfully.
For capacity mode - serverless CosmosDB SQL API, collection items were not identified as data assets after scan completion.
Related
I am trying to review if there is a way to track throughput changes for Azure Cosmos DB when using manual throughput. Please if you can provide any pointers.
Doesn't Azure support Geo-Redundancy and Multi-region Writes options for serverless CosmosDB Core(SQL) account? But, those options are listed for Provisioned throughput capacity.
Yes you are right, It's a limitation with Serverless Accounts, According to docs,
A serverless account can only run in a single Azure region. It is not
possible to add additional Azure regions to a serverless account after
you create it.
There is some work being done in this area, but there is no ETA yet.
Does cosmosdb cassandra api uses same memtable , commit log, systable compaction write path to provide good performance for heavy write operations or is it same as the write operations in cosmos sql api.
Will cosmos cassndra api is suppose to work better then write same no. of write operations in cosmos sql api?
I created Azure CosmosDb database and container for my documents.
I use NiFi as a main data ingestion tool and want to feed my container with documents from NiFi flow files.
Can anybody please share a way to post flowfile content to Azure Cosmos Db from NiFi?
Thanks in advance
UPDATE(2019.05.26):
In the end I used Python script and called it from NiFi to post messages. I passed a message as a parameter. The reason I chose python is because it has some examples on official Microsoft site with all the required connection settings and libraries, so it was easy to connect to Cosmos.
I tried Mongo component, but couldn't connect to Azure (security config didn't work), didn't really go too far with it as Python script worked just fine.
Azure CosmosDB exposes MongoDB API so you can use the following MongoDB processors which are available in NiFi to read/query/write to & from Azure CosmosDB using Apache NiFi.
DeleteMongo
GetMongo
PutMongo
PutMongoRecord
RunMonogAggregation
Useful Links
https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-introduction
https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-feature-support
Valeria. According to the components list supported by Apache Nifi related to Azure, you could only get Azure Blob Storage, Queue Storage, Event Hub etc,not including Cosmos DB.
So,I suggest you using PutAzureBlobStorage to feed azure blob container with documents from NiFi flow files. Then please create a copy activity pipeline in Azure Data Factory to transfer data from Azure Blob Storage into Azure Cosmos DB.
I am using Azure Cosmos DB as my database. Just wondering how many database instance I can create under my Azure account? It is saying that the starting instance number is 200 per account. I may need around 1000 database instance per my tenant.
Thanks advanced.
Here is a feeback about the limitation of total databases number per Cosmos DB account. Also, I checked this issue on my side, and this limitation for 100 databases per Cosmos DB account still remains. For Azure Cosmos DB account number limit per subscription, I checked DocumentDB limits, but find nothing useful information. I have 20+ Cosmos DB account under my subscription. I assumed that you could send email to askdocdb#microsoft.com for more details about limitation of Azure Cosmos DB.
Currently the Azure Cosmos DB service quotas documentation states that you can have unlimited number of databases per account, so if there has been any limitations before they seem to be lifted. Just beware, it can get costly.