I use java version 4 SDK for azure cosmos db. I want to create database inside azure cosmos db account with service principal, not with masterkey.
I assigned to service principal DocumentDB Account Contributor and Cosmos DB Operator built-in-role definitions, according to this documentation:
https://learn.microsoft.com/pl-pl/azure/role-based-access-control/built-in-roles#cosmos-db-operator
I was not able to create CosmosAsyncClient, until I added new custom role, which just contains reading metadata. Above mentioned built-in-role definitions do not contain it...
TokenCredential ServicePrincipal = new ClientSecretCredentialBuilder()
.authorityHost("https://login.microsoftonline.com")
.tenantId(tenant_here)
.clientId(clientid_here)
.clientSecret(secret_from_above_client)
.build();
client = new CosmosClientBuilder()
.endpoint(AccountSettings.HOST)
.credential(ServicePrincipal)
.buildAsyncClient();
After I added this role, client was created, but I am not able to create database instance and also container inside it as next step. In access control I can see that roles are assigned so service principal is correct here.
What is more, when firstly I create database and container with master key and then I want to read/write data using service principal, it works (obviously after adding custom role for writting also).
Then I do not know why DocumentDB Account Contributor and Cosmos DB Operator does not work for creation database.
Looks it is a bug in java SDK, the DocumentDB Account Contributor role is enough to create the database and container as it has the Microsoft.DocumentDb/databaseAccounts/* permission(* is a wildcard, it also includes the Microsoft.DocumentDB/databaseAccounts/readMetadata you mentioned).
When I test to use a service principal with this role to create the database with the powershell New-AzCosmosDBSqlDatabase, it works fine. When using the service principal to run this command, it essentially uses the Azure AD client credential flow to get the token, then uses the token to call the REST API - PUT https://management.azure.com/subscriptions/xxxx/resourceGroups/xxxx/providers/Microsoft.DocumentDB/databaseAccounts/xxxx/sqlDatabases/testdb1?api-version=2020-04-01 to create the database, the java SDK essentially also does the same thing, so it should also work.
Related
I am attempting to setup Synapse to access a Cosmos Db that has firewall rules set to only allow whitelisted IPs.
After a bit of research, I came across this article:
Securing Azure Synapse Workspaces? Beware of One Inescapable Networking Blocker | by Moussa Taifi PhD | Towards Dev
According to that post, the only option is to whitelist the entire range of IPs that might be used by the pool. Can someone let me know if this is indeed the case? I started looking at private endpoints as that seems like a perfect solution, but I can't get it to work. I tried the following multiple times:
Create new CosmosDb with Azure Synapse Link enabled
Restrict to Selected networks
Create a new DB and Container
Verify that I can’t add a new item
Add my IP
Add new item
Create a new Synapse Workspace, choosing Managed VNet
After creation, verify that the Integration Runtime is in the Managed VNet.
Create two new private endpoints for my Cosmos db. One for type Sql, and one for Analytical (I’m not sure which I need yet)
Go to the Private Link center and approve both end points
Data > Connect to External Data
Ensure that my runtime is in the Manage VNet
Select my DB
I waited 10 min, but the managed endpoint list is stuck at “Refreshing.” I continued to save anyway, but when I try to make a SQL call (after creating the credential), I get:
Resolving CosmosDB path has failed with error 'Access to the database account '*******' is forbidden.'.
The endpoints are permanently "Refreshing" in both the properties of the connection and also in the Manage Private Endpoints. The end point links are "approved" and show as such in Cosmos DB.
Can anyone let me know:
Are Private endpoints a method that I can use to connect my Synapse Workspace to my locked down Cosmos DB?
If so, what might I be doing wrong?
Thanks!
You should consider opening a support case in the case of misconfiguration in your settings.
There should be an option to allow access to Azure IPs. This is contained in documentation located here:
Add a managed private endpoint for Azure Cosmos DB analytical store
Sign into the Azure portal.
From the Azure portal, navigate to your Synapse Analytics workspace and open the Overview pane.
Launch Synapse Studio by navigating to Getting Started pane and select Open under Open Synapse Studio.
In the Synapse Studio, open the Manage tab.
Navigate to Managed private endpoints and select New
Create a new private endpoint for analytical store.
Select Azure Cosmos DB(SQL API) account type > Continue.
Select Azure Cosmos DB SQL API to create a private endpoint.
Fill out the New managed private endpoint form with the following details:
Name - Name for your managed private endpoint. This name cannot be updated after it's created.
Description - Provide a friendly description to identify your private endpoint.
Azure subscription - Select an Azure Cosmos DB account from the list of available accounts in your Azure subscriptions.
Azure Cosmos DB account name - Select an existing Azure Cosmos DB account of type SQL or MongoDB.
Target sub-resouce - Select one of the following options: Analytical: If you want to add the private endpoint for Azure Cosmos DB analytical store. Sql (or MongoDB): If you want to add OLTP or transactional account endpoint.
Note
You can add both transactional store and analytical store private endpoints to the same Azure Cosmos DB account in an Azure Synapse Analytics workspace. If you only want to run analytical queries, you may only want to map the analytical private endpoint.
Choose analytical for the target subresource.
After creating, go to the private endpoint name and select Manage approvals in Azure portal.
Navigate to your Azure Cosmos DB account, select the private endpoint, and select Approve.
Navigate back to Synapse Analytics workspace and click Refresh on the Managed private endpoints pane. Verify that private endpoint is in Approved state.
How to copy data, one collection, from one Cosmos DB API for MongoDB account to another Cosmos DB API for MongoDB account, in another subscription, placed in another Azure region.
Preferably do it periodically.
You can use Azure Data Factory to easily copy a collection from one Cosmos DB API for MongoDB account to another Cosmos DB API for MongoDB account, in any other subscription, placed in any other Azure region simply using Azure Portal.
You need to deploy some required components like Linked Services, Datasets and Pipeline with Copy data activity in order to accomplish this task.
Use Azure Cosmos DB (MongoDB API) Linked Service to connect the Azure Data Factory with your Cosmos DB Mongo API account. Refer Create a linked service to Azure Cosmos DB's API for MongoDB using UI for more details and step to deploy.
Note: You need to deploy two Azure Cosmos DB (MongoDB API) Linked Service, one for source account from where you need to copy the collection, and another for destination account where the data will be copied.
Create Datasets by using Linked service created in above step. Your dataset will connect you to the collection. Again you need to deploy two datasets, one for source collection and another for destination collection. It will look like as shown below.
Now create a pipeline using Copy data activity
In Source and Sink tab in copy data activity settings, select the source dataset and sink dataset respectively which you have created in step 2.
Now just Publish the changes and click on Debug option to run the pipeline once. The pipeline will run and collection will be copied at destination.
If you want to run the pipeline periodically, you can create Trigger based on event or any specific time. Check Create a trigger that runs a pipeline on a schedule for more details.
The examples given in the python SDK to access a cosmosdb account only use the masterkey; is it possible access tables or SQL collections using from msrestazure.azure_active_directory import MSIAuthentication or something similar (which for example allows for MSI token auth into keyvault) for auth into cosmos?
(obviously python could manually implement REST API for token auth, just wondering if SDK support is there)
No, Azure Cosmos doesn't support AAD based auth yet, so none of the SDKs support this. A common pattern I've seen to avoid having it in a config file/env variable is to use Azure KeyVault to store the master key, then use MSI to access that key in-memory and pass it into the client constructor.
This is supported on Cosmos DB now. You can try it across all SDKs now, You can check the samples for different SDKS.
Azure Cosmos DB exposes a built-in role-based access control (RBAC) system
which lets you to Authenticate your data requests with an Azure Active Directory (Azure AD) identity.
Azure CosmosDB - MongoDB provides keys for Read-Write and Read Only at account level.
CosmosDB SDK and API are there through which users can be created and access can be define at database and document level.
• But What I need to do is to create a pair of username and password with restricted access to a MongoDB database similar to one provided by installable MongoDB.
• How a user can connect only CosmosDB MongoDB database using RoboMongo.
Highly appreciate any help.
Amit -
Today, Cosmos DB access are provided by using two keys, Master Key and Read Only key. However, if you want to restrict user access per collection, per document etc, you have to use Resource Tokens. You can read more about it here and please take a look at CH9 video to see the implementation details. Resource Token service can be implemented as an Azure Function. Here is code to get you started.
But if you are using RoboMongo you have to Use the keys as define in this document. At this time you cannot define different users and Keys for a Database.
I have an existing CosmosDB Account which was originally set up for the SQL api. I would like to create a graph but keep/manage it under the same database account.
I would like to use the Gremlin.Net sdk which (if I understand properly) requires the 'https://my-account.gremlin.cosmosdb.azure.com:443/' endpoint. Do all collections have this endpoint, or only collections created with an account targeting the gremlin api?
A better way of doing this would be
Create a new Graph account ( this will create the required gremlin server endpoint)
Migrate existing data using graph bulk executor.
Jayanta