In Build session #BRK3060 Mark Russinovich demos some code that uses both the SQL and Graph APIs on the same database (starts at 45:27):
https://www.youtube.com/watch?v=S2zguwKvlQk
Does anyone have any insight into these read/write using multiple APIs on the same database?
Related
I created Azure CosmosDb database and container for my documents.
I use NiFi as a main data ingestion tool and want to feed my container with documents from NiFi flow files.
Can anybody please share a way to post flowfile content to Azure Cosmos Db from NiFi?
Thanks in advance
UPDATE(2019.05.26):
In the end I used Python script and called it from NiFi to post messages. I passed a message as a parameter. The reason I chose python is because it has some examples on official Microsoft site with all the required connection settings and libraries, so it was easy to connect to Cosmos.
I tried Mongo component, but couldn't connect to Azure (security config didn't work), didn't really go too far with it as Python script worked just fine.
Azure CosmosDB exposes MongoDB API so you can use the following MongoDB processors which are available in NiFi to read/query/write to & from Azure CosmosDB using Apache NiFi.
DeleteMongo
GetMongo
PutMongo
PutMongoRecord
RunMonogAggregation
Useful Links
https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-introduction
https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-feature-support
Valeria. According to the components list supported by Apache Nifi related to Azure, you could only get Azure Blob Storage, Queue Storage, Event Hub etc,not including Cosmos DB.
So,I suggest you using PutAzureBlobStorage to feed azure blob container with documents from NiFi flow files. Then please create a copy activity pipeline in Azure Data Factory to transfer data from Azure Blob Storage into Azure Cosmos DB.
Azure CosmosDB - MongoDB provides keys for Read-Write and Read Only at account level.
CosmosDB SDK and API are there through which users can be created and access can be define at database and document level.
• But What I need to do is to create a pair of username and password with restricted access to a MongoDB database similar to one provided by installable MongoDB.
• How a user can connect only CosmosDB MongoDB database using RoboMongo.
Highly appreciate any help.
Amit -
Today, Cosmos DB access are provided by using two keys, Master Key and Read Only key. However, if you want to restrict user access per collection, per document etc, you have to use Resource Tokens. You can read more about it here and please take a look at CH9 video to see the implementation details. Resource Token service can be implemented as an Azure Function. Here is code to get you started.
But if you are using RoboMongo you have to Use the keys as define in this document. At this time you cannot define different users and Keys for a Database.
I was wondering if MongoDB is fully supported in Azure Cosmos DB through the MongoDB API https://learn.microsoft.com/es-es/azure/cosmos-db/mongodb-introduction
I have read that the aggregation pipeline, map-reduce and the full-text indexes is not fully integrated. Does anyone have further information about it? Would you use MongoDB in Azure Cosmos DB considering its current status?
Cosmos DB implements MongoDB wire protocol and many customers already use MongoDB API in production. Aggregation pipeline is in private preview and you can enable it by emailing askcosmosmongoapi#microsoft.com. Map-reduce functionality is mostly covered by aggregation pipiline. Full-text search is partially available through Azure Search, which can index MongoDB collections and $regex operator within MongoDB API covers less complex text search. You can find some other feature requests and their status at https://feedback.azure.com/forums/263030-azure-cosmos-db/category/321994-mongodb-api
Cosmos DB's MongoDB layer implements a large subset of native MongoDB functionality. Specifics of supported features are published here.
You mentioned aggregation pipeline: As of November 2017, this is now supported.
Regarding "current status" of the Cosmos DB MongoDB API: It's a production database with SLA. You'll need to make your own decision on whether to use it, based on feature set and your app's needs.
You can activate aggregation pipeline through Azure portal by going to Preview Features menu.
I have a dataset of 6 million entries. Each entry has one-to-many relationship with other entries. Previously this data has been stored in a Neo4J instance.
Does Google Cloud provide a product that can store Graph Databases? Or is there a way to adapt an existing Google Cloud Database product to work as a graph databases engine?
I am trying to avoid running a Neo4J instance on a Google compute instance.
JanusGraph is an open source graph database solution which can use Google Cloud Bigtable as a storage backend; here's a guide for deploying JanusGraph with Cloud Bigtable on GCP.
Some of the folks from Google even help maintain the project. So that might be close to what you looking for.
I have an ionic application that uses SQLite to store data. The data is retrieved from an HTTP REST Service, which then connects to a neo4j database. I need to be able to sync the changing data or insert any new data into my sqlite database as the data changes on the neo4j server. What is the best way to do this? Are there any existing frameworks? I am aware of PouchDB, but that doesn't really fit with what I am doing. I can't use local storage or any other in-memory storage as there could be a lot of data.
You may find these neo4j data integration articles helpful:
Import Data Into Neo4j
Data Migration between MySQL and Neo4j