I have successfully deployed my spring boot app to Compute Engine on ubuntu 18.04, it is behind Nginx proxy but currently Nginx is listening to 80 port, which is http. I need to set up secure connection. I have question about few details, im new to this, all i've done so far is write spring boot/react js apps on windows, in IDE.
Is it necessary to buy a domain for my compute engine or I can just make SSL for the external ip of compute engine ? On compute
engine only the back end rest api is deployed, the front end is on
Netlify and it's already working. I don't need a good sounding
domain name for back end because user won't see it, only front end
app will use the external ip of my compute engine to fetch data
from back end.
I have seen guides that set up SSL in the setting of Nginx, why is that ? Isn't the request first coming to the compute engine's external ip and only after that to Nginx ? Isn't it the job of compute engine to do secure connection by sending public key&certificate to front end and only then relay the request to Nginx ? Or does compute engine simply relay the https request that comes to it to Nginx right away, without securing it/doing any key&certificate sending ?
You can give some advice if you have any, i'm just trying to make a secure connection to my back end spring boot app which is behind Nginx on google compute engine, which currently works only with HTTP but not HTTPs.
1. Technically you're able to have SSL certificate for public IP, but it's rarely used. More details you can find in RFC 5280 and in this question.
Keep in mind that if your IP address changes your SSL certificate become useless.
I've checked a few SSL providers and found that you should be the owner of the IP to obtain such SSL certificate:
accordingly to the article Using an IP Address in an SSL Certificate posted by geocerts:
If you decide that you really need an IP in your cert there are
specific stipulations, conditions, and limitations to consider. The
biggest hurdle for most folks is that the IP address must be
specifically assigned to your company or organization (not your ISP or
hosting provider) as verified by an IP WHOIS lookup.
accordingly to the article Issuing SSL certificate for an IP address by LeaderSSL:
Quite frequent question: is it possible to issue an SSL certificate
for an IP address (and not for a domain name)? Yes, it is possible.
However, there are several requirements:
Only OV SSL certificates can be issued;
The company must own IP address (validation based on WHOIS information of IP-addresses).
same in the article WHAT IS AN IP ADDRESS SSL CERTIFICATE?:
An IP address SSL certificate secures connections directly with the IP
address submitted. Whereas typically an SSL certificate is issued to a
Fully Qualified Domain Name (FQDN), some organisations may need to
secure an IP address.
Only public IP addresses may be used and you must be the owner of the
IP address according to the records at RIPE.
As result, practically, it's almost not possible in case of GCE VM instance and it's easier to proceed with domain certificate.
2. In GCE all the connections to the external IP of VM instance passed through directly to the VM instance. GCE isn't able to secure connections on it's own. You should configure SSL certificate on VM instance. More details you can find in the documentation VPC network overview and IP Addresses.
In addition, you're able to use Google-managed SSL certificates or own SSL certificates on external HTTP(S) load balancers.
Related
I try to use IP directly access a URL by HTTP protocol, there are two methods here:
Use the URL which the domain replace with its IP
Config the IP and domain pair in local host of system
When the http request arrived the server, from the point of the server view, what's the different between the both request?
There would be no difference to the server. It is up to the client to resolve domain names to their IP addresses and there are a few ways to do that. One is to use a DNS server, another is to use the hosts file.
To add on, the domain is most certainly sent in the HTTP request. The browser (or other HTTP client) is responsible for that. If you use the IP, then no domain will arrive at the server, meaning it will serve up whatever is configured as the default content for the IP. The domain info is what allows the server to server multiple domains on a single IP.
Consider a Client connects to a Server.
First thing your computer does, is to analize the address you input.
If you wrote an IP, nothing to do now. If you wrote a domain, it will 'resolve' domain.
Resolving domain (DNS) means to 'translate' your URL into an IP. When computer resolves the domain, it will only use the new IP it got.
Client is sending packages with meta-data, that includes where this package is from (client's ip), where it may reach (server's ip), destin port etc.
Domain is not included on network packages.
Only IPs are included.
That means the server could never know if user connected to it using a domain or its ip directly.
I am trying to get the big picture although my primary domain is not networking.
Some question's narrowed down for which I'm not getting enough/proper answers online
Is the IP that is resolved by the DNS server when I hit www.google.com is same as any of the Google router's Gateway IP?
Do bigger companies like Amazon do port forwarding?
If point 2 is true, I suppose they must be port forwarding with only 443 (https) port which means, to use multiple static IP across different data centers, they need to have that many routers. So, if they have N static IP address which resolves to a website, then they must be having N routers right? Is this a fair assumption?
A gateway IP refers to a device on a network which sends local
network traffic to other networks. it sits between you and internet,or other network . its like a watchman.
Question 1 : google.com has multiple ip addresses lets say then , Yes, that is possible, and will need to be two A records. This is called Round-Robin DNS. Clients will semi-randomly use one of the two addresses.
question2: yes port forwarding happens more often than we think. ALL VPC's (virtual private clouds like AWS , GCP , Azure etc) use this as they dont want to expose servers/internal resources to the internet.
depending on the port number , particular service is exposed to requesting client. lets say we want to make a website public , then we explicitly expose port 80(http) 443(https) so that web crawlers and users can see them.
Port forwarding, sometimes called port mapping, allows computers or
services in private networks to connect over the internet with other
public or private computers or services.
google https://www.google.com:444/ wont work because they did not expose port 444 on their cloud router
but https://www.google.com:443/ will work because the server corresponding to google.com has explicitly left it open.
How IP is resolved:
Step 1 - Send a Request to Resolve a Domain Name
When you type www.google.com into a browser, in order to load the webpage, your computer asks for the IP address. Computers do not know in advance where they can find the necessary information, so they try searching through the DNS cache and for available external source. proceed from lower level caches to root/main servers.
Step 2+3 - Try to resolve an IP Locally
Before going externally, your computer loads the local DNS cache database to see if you already requested the IP for that domain name. Every computer has a temporary cache with the most recent DNS requests and attempts to connect to online sources. if required record is present locally its called "CACHE HIT" and query stops.
However A computer’s local DNS cache database does not always contain the necessary data to resolve a domain name this is called a "CACHE MISS" . In that case, the request goes further to your Internet Service Provider (ISP) and its DNS server.
Step 4 - ISPs Ask Outside DNS Servers to Provide an IP Address iff Cache miss
ISP DNS resolvers are configured to ask other DNS servers for correct IP address mapping until they can provide data back to the requester. These are iterative DNS queries.
When a DNS client sends such a request, the first responding server does not provide the needed IP address. Instead, it directs the request to another server that is lower in the DNS hierarchy, and that one to another until the IP address is fully resolved. There are a few stops in this process.
hierarchy looks like this (just for reference):
Root domain nameservers. Root servers themselves do not map IP addresses to domain names. Instead, they hold the information about all top-level domain (TLD) nameservers and point to their location. TLD is the rightmost section of a domain name... Root servers are critical since they are the first stop for all DNS lookup requests.
TLD nameservers. These servers contain the data for second-level domains, such as ‘phoenixnap’ in phoenixnap.com. Previously, the root server pointed to the location of the TLD server. Then, the TLD server needs to direct the request toward the server that contains the necessary data for the website we are trying to reach.
Authoritative nameserver. Authoritative servers are the final destination for DNS lookup requests. They provide the website’s IP address back to the recursive DNS servers. If the site has subdomains, the local DNS server will keep sending requests to the authoritative server until it finally resolves the IP address.
Step 5 - Receive the IP Address
Once the ISP’s recursive DNS server obtains the IP address by sending multiple iterative DNS queries, it finally returns it to your computer. The record for this request now stays cached on the hard drive. The browser can then fetch this IP from the cache and connect it to the website’s server.
ALL this happens in less than 1 second, most of the times. if you just registered a new domain it might take few hours to propagate this DNS cache globally hence newly registered websites do not show up sometimes.
About companies owning multiple IPs
Big companies have pool of IPs reserved for example 123.234.xxx.xxx which means a company has reserved 255*255 ips. they are mapped on a VPC(virtual private cloud)
and accessible vis a subnet masking and CIDR feature, like your EC2 instances on AWS
Is the IP that is resolved by the DNS server when I hit www.google.com is same as any of the Google router's Gateway IP?
For sure it should, but it is mostly a Google management question that only they will be able to answer right. The thing is that we must understand how DNS query's work for this.
Let's take a look of it:
Device A requests the IP address through a DNS query of the device B.
To do this, it uses the network port 53 (Domain) on which it will ask, depending on which DNS server is being used at the time, which is usually the home router. Then the router will ask the ISP's DNS server, which will respond with a cached response, or the query with another server on top of it if it does not have one; All this process is followed until a reliable cache response is reached or until the authoritative response server is reached, that is, the name server that manages the domain in question.
Only the authoritative response server contains the reliable information of which IP of the domain which is going to be reached.
I suppose that within Google's servers and its network they use Google's own DNS servers, which are 8.8.8.8 and 8.8.4.4 where the DNS records are obtained and consulted by caching from many sites.
In general terms Google's IP will change depending on where you are, I made a DIG query to Google's authoritative servers, however, I received a result based on location to improve the route and loading time of the site which was 142.250.73.238.
Do bigger companies like Amazon do port forwarding?
Yes, they do. To handle queries with load balancers or similar and even for caching dns requests.
If point 2 is true, I suppose they must be port forwarding with only 443 (https) port which means, to use multiple static IP across different data centers, they need to have that many routers. So, if they have N static IP address which resolves to a website, then they must be having N routers right? Is this a fair assumption?
This has multiple answers. By the way, they actually can do a secure DNS query.
if they have N static IP address which resolves to a website, then they must be having N routers right?
They don't have to, but if they want to they can.
"Is this a fair assumption?"
No, the IP's doesn't depend on a router, the router only routes to a computer/server which can have multiple IP's. By the other hand, each thing (computer, server, etc... must have an IP which can be also a WAN IP).
I'm a little confused about IP addresses.
I know that every web domain has an ip address.
Does the IP address represent the physical machine / host the website files are stored on?
Therefore when DNS lookup is performed, the domain's IP address is returned to the client. The client then uses this ip to contact the server that the web files reside on.
Is my understanding correct?
Many thanks
You are correct.
It is kind of like how some companies may say Dial PIZZAHUT instead of saying, Dial 74992488. PIZZAHUT is easier to remember, but you actually are dialling the number.
You're talking about HTTP protocol. Yes, FQDN (web domain) is resolved to IP address by DNS server. Client will connect to server IP address. Since you're probably using HTTP 1.1, HTTP request will contain also FQDN. This information is used by web server to perform several checks, like SSL certificate validation or Virtualhost management (several domains on a single IP address).
I use to develop my project on my localhost, on apache in ubuntu machine.
Sometimes i need to show progress to my costumer.
Is it possible to access to localhost from remote machine?
You can use a service that provides a tunnel to your local service, such as localtunnel, pagekite or ngrok. These services simplify setting up remote demos, mobile testing and some provide request inspection as well.
I find ngrok useful because it provides a https address, which is needed to test things like webcam access.
Terms used in this answer:
Host = machine with site on it
Client = machine you are trying to access the host from
If the host and client are on the same network, you can access the host from the client by entering
http://(hostname or ip address)
in your client's browser. If the site is not running on port 80 (for http) or port 443 (for https), add the post as so (this example is for if your server is on 8080, a common alternate port):
http://(hostname or ip address):8080
If the host and client are not on the same network, and you need to reach across the internet from the client to see the host, you will need to make your host available on the internet for the client to access.
This can be extremely dangerous for your information security if you're not sure what you're doing and I'd recommend getting a cheap-o hosting account (can get them for like $10/month at places like 1:1 hosting).
There are many methods to do this - the difference is security, easiness of the configuration and cost of the solution.
Following I am typing some methods with some analyses
Port Forwarding (with Dynamic DNS and SSL encryption)
This requires router configuration (to forward your routers public port to loclhoat port), however this requires you to have fixed ip address. In case your ip address is not fixed (in most cases) you need to use Dynamic DNS services to be able to use domain name instead ip address (there are lot of available free services). Here we still have security question open. To solve security question i.e. setup ssl certificate we can use Let’s Encrypt service ( https://letsencrypt.org/ ) to get free certificate, however we should configure local server to use the certificate or we should setup reverse proxy (in most cases nginx or apache) and configure proxy to use certificate.
Conclusion – Hard to setup if we want to have secure connection (can be done for free)
VPN
For this scenario we should use VPN services. We should connect our local machine to VPN then in other side we should connect our client's machine to VPN that will allow us to access to localhost by local IP address. We can set up our own VPN server however this requires knowledge to do it right.
Conclusion – Easy, Paid, Secure, Bad User Experience (connecting to VPN every time you need to connect to localhost)
Tunneling
For this scenario we can use free tunneling services (i.e. https://tunnelin.com/). The process is very straight forward i.e. Register a User, Connect your device to service (by running one line command on device), use Web interface to open/close secure tunnels to the device.
Conclusion – Free, Secure, Easy
Yes, if you have a public and static IP. Usually, ISPs offer static ips during a session (i.e. until you disconnect and connect again)
I need to use my computer as a server but my ISP blocks port 80, 21, 23 etc. I can use other ports and some dynamic dns service but I don't want:
(HTTP) Users have to type http://mydynamicdnsaddress:#port#
(HTTP) Users be redirected from http://mydynamicdnsaddress to http://mydynamicdnsaddress:#port#
(HTTP) Some kind of service that gets HTTP response and change it before resending to users. No-ip and GoDaddy do that. They change some parts of html - eg: title.
Users have to type ftp://mydinamicdnsaddress:#port#
I believe that I need some kind of dynamic dns service that points to a router that forwards TCP packets to another address changing ports. Do you know any online service like that?
Many "dynamic DNS companies use HTTP redirection to send the browser from port 80 to a different port. When you ask a dynamic DNS company to point your domain to a port other than 80, what they actually do is point the domain to their own web-server IP address (in DNS), and then on their web-server (running on port 80) they have a simple server side script which redirects the browser to the your web-server on whatever port you specified - optionally "cloaked" so the visitor won't notice." Can I specify a TCP/IP port number for my web-server in DNS? (Other than the standard port 80)
Here's a reference article for a redirection script: Redirect Script.
What you are asking for is a tunnel or proxy. You'd set up a server which receives communications via port (e.g.) 80 and proxies that request to your home server on port-whatever. You'd probably need to get a dedicated host (or VM like linode) in order to do this. At that point, you might as well move your webserver to the unblocked host.
Also, to be clear, this is impossible with pure DNS. DNS, "Domain Name System", resolves names to IP addresses, NOT to IP address/port pairs.
Most dynamic DNS service providers also provide free web redirect or port forwarding such as dynu.com.
Please note that the cloak works by loading the page in a frame of sort and it does not work with all browsers. For example, Chrome does not support cloak.
As far as I know, you cannot specify the port number in the DNS unless the web server which performs redirection is clever enough to read out the TXT record and use it for redirection. Any web server doing that would be really nice though.