Infrastructure description: I have a dynamo db table in one AWS account (Say A1) and an application hosted in EC2 in another account (say A2) /VPC-private subnet. This app (in account A2) reads/writes that dynamo db table in account A1. Both accounts are under same organization and the table and app are in same AWS region. I created a VPC endpoint (say VPC-E1) for the dynamo db in the application's account (A2) and the route table is correctly populated with the VPC endpoint targets. The app authorizes itself using AssumeRole method. I created an role policy to the same IAM account that the EC2 uses to allow connecting to the DynamoDB only if the source VPC endpoint is the one I created (VPC-E1). NOTE: the EC2 has internet connectivity via NAT gateway.
IAM policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "dynamodb:*",
"Effect": "Deny",
"Resource": [
"My DynamoDb table ARN"
],
"Condition": {
"StringNotEquals": {
"aws:SourceVpce": "VPC-E1"
}
}
}
]
}
The Reachability analyzer says that the traffic from Ec2 to the VPC endpoint will work fine.
The problem: The traffic is denied by the policy because either 1. the traffic is not taking place via that VPC endpoint or I assume that the traffic is taking place via Nat gateway/internet. When I remove this policy it works fine; because the traffic might take place via NAT gateway.
Have anyone configured such setup successfully? i.e. Accessing DynamoDb across accounts via AWS private network (VPC endpoint). My aim is to send the traffic via AWS private network from one account/VPC to another account that the dyanamo db table belongs to.
Yes. DynamoDb is a SaaS and is not hosted inside your VPC. I removed the condition in the IAM policy. I created a cloudtrail logs at account (A2 - where dynamodb belongs to) to capture the data events from specific dynamodb table . The VPC endpoint created in the consumer account (A2) appears in the cloudtrail logs (Data events from specific DynamoDb table/index) in the target AWS account (A1). Hence this works.
Related
I have a GKE cluster where my application is running. My application is built using springboot. And I have google Datastore database which is running in a separate GCP Project.
It throws the error "DatastoreException: Unauthenticated" when it tries to connect to datastore database during application start-up.
The connection happens through "Service account" permissions, application service account having necessary(Datastore user) permission to the datastore database. But it still fails.
Below two are the similar scenarios where it works:
1.
With similar set-up in a lower environment, it works.
2.
I have another application running inside same GKE cluster which is using the same service account but connecting to another datastore database with similar role to the service account, it works fine here.
I am attempting to setup Synapse to access a Cosmos Db that has firewall rules set to only allow whitelisted IPs.
After a bit of research, I came across this article:
Securing Azure Synapse Workspaces? Beware of One Inescapable Networking Blocker | by Moussa Taifi PhD | Towards Dev
According to that post, the only option is to whitelist the entire range of IPs that might be used by the pool. Can someone let me know if this is indeed the case? I started looking at private endpoints as that seems like a perfect solution, but I can't get it to work. I tried the following multiple times:
Create new CosmosDb with Azure Synapse Link enabled
Restrict to Selected networks
Create a new DB and Container
Verify that I can’t add a new item
Add my IP
Add new item
Create a new Synapse Workspace, choosing Managed VNet
After creation, verify that the Integration Runtime is in the Managed VNet.
Create two new private endpoints for my Cosmos db. One for type Sql, and one for Analytical (I’m not sure which I need yet)
Go to the Private Link center and approve both end points
Data > Connect to External Data
Ensure that my runtime is in the Manage VNet
Select my DB
I waited 10 min, but the managed endpoint list is stuck at “Refreshing.” I continued to save anyway, but when I try to make a SQL call (after creating the credential), I get:
Resolving CosmosDB path has failed with error 'Access to the database account '*******' is forbidden.'.
The endpoints are permanently "Refreshing" in both the properties of the connection and also in the Manage Private Endpoints. The end point links are "approved" and show as such in Cosmos DB.
Can anyone let me know:
Are Private endpoints a method that I can use to connect my Synapse Workspace to my locked down Cosmos DB?
If so, what might I be doing wrong?
Thanks!
You should consider opening a support case in the case of misconfiguration in your settings.
There should be an option to allow access to Azure IPs. This is contained in documentation located here:
Add a managed private endpoint for Azure Cosmos DB analytical store
Sign into the Azure portal.
From the Azure portal, navigate to your Synapse Analytics workspace and open the Overview pane.
Launch Synapse Studio by navigating to Getting Started pane and select Open under Open Synapse Studio.
In the Synapse Studio, open the Manage tab.
Navigate to Managed private endpoints and select New
Create a new private endpoint for analytical store.
Select Azure Cosmos DB(SQL API) account type > Continue.
Select Azure Cosmos DB SQL API to create a private endpoint.
Fill out the New managed private endpoint form with the following details:
Name - Name for your managed private endpoint. This name cannot be updated after it's created.
Description - Provide a friendly description to identify your private endpoint.
Azure subscription - Select an Azure Cosmos DB account from the list of available accounts in your Azure subscriptions.
Azure Cosmos DB account name - Select an existing Azure Cosmos DB account of type SQL or MongoDB.
Target sub-resouce - Select one of the following options: Analytical: If you want to add the private endpoint for Azure Cosmos DB analytical store. Sql (or MongoDB): If you want to add OLTP or transactional account endpoint.
Note
You can add both transactional store and analytical store private endpoints to the same Azure Cosmos DB account in an Azure Synapse Analytics workspace. If you only want to run analytical queries, you may only want to map the analytical private endpoint.
Choose analytical for the target subresource.
After creating, go to the private endpoint name and select Manage approvals in Azure portal.
Navigate to your Azure Cosmos DB account, select the private endpoint, and select Approve.
Navigate back to Synapse Analytics workspace and click Refresh on the Managed private endpoints pane. Verify that private endpoint is in Approved state.
AppMaker gives this error: "The default Google Cloud SQL instance is not setup properly. Please ask a G Suite administrator to check the Google Cloud SQL configuration for your domain. (Reason: App Maker is unable to verify the default Google Cloud SQL instance. The instance must be a 2nd generation SQL database.)"
But the SQL database is a 2nd generation
ALF-experiments instances:
Instance ID Type High availability Location Labels
sgialfmysql MySQL 2nd Gen 5.7 us-central1-a
Connected as the default in admin console
Google Cloud SQL instance setting
Enter the Google Cloud SQL instance connection name to use with App Maker:
alf-experiments:us-central1:sgialfmysql
I have made multiple Cloud instances under multiple project but none seem to be able to connect. I have reviewed the documentation several times to verify that GCP and Cloud SQL are set correctly and they are.
when trying to connect to my org SQL cloud from app maker, defined to use only private IP it fails:
The default Google Cloud SQL instance is not setup properly. Please ask a G Suite administrator to check the Google Cloud SQL configuration for your domain. Cannot create new database in the default Google Cloud SQL instance.
any specific service needs to be given to appmaker-maestro to make this work ?
if I connect to a custom instance which has public IP enabled it works all well.
thanks for any guidance you might have.
I have an Azure Web App hosting an API (ASP.NET MVC project) that interacts with a CosmosDB database and collections to get subscriptions and other information.
The CosmosDB database is accessed R/W by the Web App middle-ware uses through the nuget package "Microsoft.Azure.DocumentDB" SDK v1.19.1.
I am trying to set up the CosmosDB IP Firewall through the Azure Portal. I allowed the Azure Portal to have access to the db and then I needed to also allow the web app (also hosted on Azure) to have access. To do this, I copied the Virtual IP Address of the Web App from the Properties tab in the Azure Portal.
But this was not enough. I waited more than 10 minutes trying my web app but all the calls to the CosmosDB were rejected with error 404, which as the documentation states it is the proper behavior for SDK Calls (security reasons).
Then I added, all the Outbound IP Addresses stated at the same Properties Tab of the Web App. Waited for more than 20 mins and still 404 error.
What are the correct steps to achieve the requested task?
For example in SQL On Azure, the IP Filtering allowed for an option, to allow access from any Azure App/ VM / Service. How can we achieve the equivalent in CosmosDB?
Thanks in advance
Since Azure App Service is PaaS, and following this article, please try adding the IP 0.0.0.0.
On the Azure Portal, this can also be set by switching on Allow access to Azure Services.