We have a client that has data in Snowflake, but also limits IP connectivity to the data warehouse. If we are going to use the default Azure Data Factory Snowflake connector, what IP's do we have to give the client to whitelist? Is it the entire range from the datacenter location? I understand we may be able to run the ADF in a separate vnet, but don't want to add that to the deployment.
If you are using SHIR for connectivity then you can whitelist your SHIR machine IP address.
In case if you would want to use a custom Azure Intergration Runtime specific to a region then you need to whiltelist the complete Azure IR IP range of that region.
In case if you are using default Azure IR then you will have to whitelist the complete Azure IR IP range.
You can get an IP range list of service tags from the service tags IP range download link. For example, if the Azure region is AustraliaEast, you can get an IP range list from DataFactory.AustraliaEast. For more info please refer here - Azure Integration Runtime IP addresses: Specific regions
Related
Can someone let me know if its possible to connect or PING a Databricks Cluster via its public ip address?
For example I have issued the command ping --all-ip-addresses and I get the ip address 10.172.226.115.
I would like to be able to PING that ip address(10.172.226.115) from my on-premise PC (or connect to the cluster with an application using the ip address?
Can someone let me know if that is possible?
That public IP is not guaranteed to be your cluster; unless somehow you've installed Databricks into your own cloud provider account, where you fully control the network routes, it would be connecting to Databricks managed infrastructure where the public ip would likely be an API gateway or router that serves traffic for more than one account
Note: just because you can ping Google DNS with outbound traffic doesn't mean inbound traffic from the internet is even allowed through the firewall
connect to the cluster with an application
I'd suggest using other Databricks support channels (i.e their community forum) to see if that's even possible, but I thought you're just supposed to upload and run code within their ecosystem. At least, for the community plans
Specifically, they have a REST API to submit a remote job from your local system, but if you want to be able to send data back to your local machine, I think you'd have to write and download from DBFS or other cloud filesystem
Networking newbie here. From the Documentation it feels like both NSG and Routing tables(UDR) are doing the same thing - capable of defining ACLs at multiple levels (Vnet, Subnet, VM)
https://learn.microsoft.com/en-us/azure/virtual-network/network-security-group-how-it-works
https://learn.microsoft.com/en-us/azure/virtual-network/virtual-networks-udr-overview
So how are they different and when is each used?
thanks.
Azure automatically creates a route table for each subnet within an Azure virtual network and adds system default routes to the table. The route table is like a networking map that tells the traffic from one place to another place via the next hop. This generates the "path" but does not filter traffic.
The Azure network security group is used to filter network traffic to and from Azure resources in an Azure virtual network. It contains security rules that allow or deny inbound network traffic to, or outbound network traffic from, several types of Azure resources. If there is no route to one place from a subnet, you even do not need to configure the security rules because there is no path. So when you consider the NSG it should have a successful network route.
For example, usually, we can access Azure VM in Azure virtual network via SSH or RDP over the Internet but it has a less secure way to expose the port 22 or 3389. We can restrict access to your Azure VM via specifying the source IP address in the NSG. This setting allows traffic only from a specific IP address or range of IP addresses to connect to the VM. Read more details here. In this scenario, we need to ensure that there is a route to the internet from your Azure virtual network and vice versa.
I am using Firebase cloud functions as a backend for my app and I want to set up a dedicated IP address using a VPC for my cloud functions since I also need to interact with a Mongo Atlas DB and want to whitelist a single IP address from which it can receive requests as a security measure. According to Firebase docs (https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip), it seems this is possible using a workaround of using a VPC.
This is a very murky and uncertain area of programming for me so I am progressing using what I am thinking the docs are telling me.
Set up a VPC. I did this using manual setup. When you set up a VPC, and you create a new subnet, it requires you to put in an IP address range that must be valid. According to this document (https://cloud.google.com/vpc/docs/vpc#manually_created_subnet_ip_ranges#subnet-ranges), the IP address range 10.0.0.0/8 is a valid IP range. I used this range for my VPC subnet.
NOTE: I tried the other IP addresses in that document but they were invalid and threw an error
THIS IS WHERE I AM STUCK vvvvvvv
2) The next step is to set up a VPC serverless access connector. I started to do this. This too requires me to define an IP range. According to this document (https://cloud.google.com/vpc/docs/configure-serverless-vpc-access?&_ga=2.204931472.-1046973627.1608007278#creating_a_connector), the range 10.8.0.0 (/28) "will work in most new projects." However, when I use this range and create the VPC serverless access connector, I get an error that says: "Connector is in a bad state, manual deletion recommended" (see below).
Again, my end goal is to have a single IP address from which I can connect to my Mongo instance. I think I am going about this correctly, but could be wrong. How can I proceed from this step and silence the error I am getting? Am I doing something wrong in the initial setup? Again, the end goal is to get a single IP address from which I can connect Firebase cloud functions to MongoDB so I can whitelist that IP address on Mongo as a security measure. Thank you.
UPDATE
Screenshot of my VPC and subnet
The reason you were not able to create a Serverless VPC connector is because the CIDR of your VPC and the CIDR of the connector cannot overlap.
This is well documented here:
[IP_RANGE] is an unreserved internal IP network, and a '/28' of unallocated space is required. The value supplied is the network in CIDR notation (10.8.0.0/28). This IP range must not overlap with any existing IP address reservations in your VPC network. For example, 10.8.0.0/28 works in most new projects.
In your VPC Serverless connection creation you omitted the most important part
This IP range must not overlap with any existing IP address reservations in your VPC network.
"The range 10.8.0.0 (/28) will work in most new projects." which is true but "most" does not mean "all" so you should check your settings always.
Since you had 10.0.0.0/8 in your VPC, the CIDR 10.8.0.0/28 overlaps with the VPC CIDR, for this reason and as suggested by #guillaume-blaquiere in the comments, the CIDR 192.168.0.0/28 will work.
I am getting a netflow traffic feed, I need to identify the application associated with the IP.
Is there any kind of global database that I can use map the global IP with the application it is used to serve?
Ex: IP A for Amazon Prime
IP B for Amazon WebService
IP C for Amazon Shopping
All IP A,B and C are owned by Amazon.
There is no public DB that has that info in that I am aware of. AS number gets you close, but not the detail on which exact service it is contacting. There maybe a way to build a list based on additional netflow intel, like proto or port. If you do discover a more detailed list or wanted to create one manually or part automated, you could create a TrafficGroup for group of IP's you have id'd. Here is the view I get on my home network with Amazon ASN filter
enter image description here
I am trying to setup a consul server in an openstack cluster. I have the server provisioned and have associated an IP with the server that is accessible from vagrants on developer machines.
I am able to join the server from a local vagrant if I use the -advertise flag on the consul agent -server command and use the floating ip I set. However, I am provisioning the server with salt and need to the machine to be able to determine that IP automatically.
By default, the server is using its bind address which is set to its 10.x.x.x local IP. That local IP is the only one I seem to be able to easily determine.
Is there a way to get an instance's floating ip(s)?
Bonus points: Is there a way to get an instances name?
The information you are looking for is available to an instance using the Openstack metadata service. It is basically a REST API that an instance can hit to get information specific to this instance. See more information here:
http://docs.openstack.org/grizzly/openstack-compute/admin/content/metadata-service.html
You should be able to get both the instance name and its floating ip (look for "public-ipv4")