I am integrating my application with some country-detection module.
Overall logic is:
detect client IP
lookup IP in Geo IP DB (for simplicity let's
assume it is MaxMind DB)
identify country code based on IP
Some other business logic based off that country code
That works very well, but I am having problems writing automated integration tests.
There is a way to override (force) certain test IPs, but the problem is that all IPs are periodically changing, and time to time I am getting my tests failed because of that.
Any ideas how to stabilize such tests?
One thought I had is if there was a directory of main ISPs across the world, but could not find it.
Thanks!
You can override the GeoIP lookup service in your test environment to return fixed values for certain test IPs. But this only tests #4. To test #1,2,3, you need to use real IP addresses.
If your needs are limited to only a few countries, you can purchase private proxies with fixed IPs and run your tests through a proxied client with a fixed IP.
To automate at scale, you can look at https://www.geoscreenshot.com, it provides an API for visually testing many locations simultaneously.
You might need to use VPN to test applications. Some providers offer free servers in fixed countries.
If it is a web application, you can use http://www.locabrowser.com to test page view from different servers in multiple countries.
Also, you can use the CSV file of IPs that MaxMind use for their unit tests (see https://stackoverflow.com/a/23616234/354709)
Related
I have multiple applications making use of Application Insights for Production Data. I'm trying to use the City telemetry field to map our current users. This data appears to be tracked very inconsistently and in most cases (> 75%) is just unavailable.
I understand some customers will be using VPNs which could affect the results, but not to the extent I'm seeing.
Here is the info from the Azure FAQ:
How are City, Country and other geo location data calculated? We look
up the IP address (IPv4 or IPv6) of the web client using GeoLite2.
Browser telemetry: We collect the sender's IP address.
Server telemetry: The Application Insights module collects the client IP
address. It is not collected if X-Forwarded-For is set.
You can configure the ClientIpHeaderTelemetryInitializer to take the IP
address from a different header. In some systems, for example, it is
moved by a proxy, load balancer, or CDN to X-Originating-IP.
Does anyone know how to improve geolocating user cities for App Insights?
IP Geolocation is not 100% accurate and you need to live with it. City accuracy is quite low because the information is guessed from multiple data that change frequently. One way to improve accuracy is to use a service that aggregates data from multiple sources and does it continuously, multiple times a day.
A second manner to enhance the results is to filter based on whether the IP is associated with a proxy by using threat data.
For both purposes, I recommend looking at Ipregistry, a service I work for:
https://api.ipregistry.co/?key=tryout
It would be great if MSFT could provide an example of manually setting the location in Browser telemetry. I understand privacy concerns, but our use-case is for internal enterprise apps used by our field service teams. Since Browsers can access the Geolocation APIs, it's probably straightforward to add that info. It's just a matter of knowing the right way to do it so it's picked up consistently.
Supposed I have two services that need to share and / or exchange data. Both instances are separate from each other, and they shall not know anything about where the other part is located.
Now in order for them to be able to share and / or exchange data, they need to connect to each other.
How do they find each other without the need to configure the IP addresses explicitly? In other words: How could they detect each other automatically?
Basically, I have two ideas:
Pull: You need to have a central service where they register. Then you can ask that service for the address of a service, and that service then returns those data. While this works, it has the drawback that it only shifts the problem to the next level: What if I have multiple instances of that service, and I don't want them to know each other in advance?
Push: Each service broadcasts its own address, so that other services get it to know. Each service repeats this from time to time. Drawback: This does hardly work in the internet.
Any idea of how I could solve this in an intelligent way?
PS: If you want to say so, I'm looking for a way to handle dynamic IPs without the need for a central DNS server.
The usual way is to have some fault-tolerant server where services register and can then look for other services - Curator framework implements that over zookeeper.
If you want autodiscovery then you should probably implement some sort of gossip protocol so that the servers would know which other servers are out there in a reliable way. You should keep in mind that getting gossip protocols right is tricky (e.g. some of past Amazon cloud failures where due to problems in their implementation)
"broadcast packets are not forwarded everywhere on a network, but only to devices within a broadcast domain."
If your devices are on different broadcast domains then broadcasting is not going to work.
You are probably going to have to implement your own central service, unless you can use one of the free dynamic dns servers, for example: Free
Is it possible to simulate a load test on a HTTP endpoint from different geo-locations. e.g. I want to simulate requests to http endpoint from US, Canada, Mexico, China...
Yes. You'll either need to employ a testing service for this (either full-service or self-service) or obtain your own computing resources to do this. One of the cloud providers can give you the short-term resources you need a minimal cost. For example, Amazon EC2 has datacenters in 7 (8?) parts of the world. We use it in our testing services and it is integrated into our load testing software and it works great. The testing software you choose (and your site's performance requirements) will determine the kind and quantity of resources you'll need.
Are you interested on having the actual requests coming from the different geo-locations or your interest is related to the latency associated with having requests coming form those locations?
If you real need is the second one, then you have a couple of options:
Software based network emulation: E.g. Visual Studio load testing allows you to simulate latency during load tests.
Hardware based network emulation: solutions to simulate latency, also called WAN Emulators. E.g. SHUNRA
I hope this helps.
I saw that it's possible to specify geographical location in Load Testing Cloud which is compatible with JMeter. Use parameter 'Load Origin Location' when create load test.
I have a web service where i do different things according to where ones IP is from. I have a simple test application where i open a WebClient and makes it call the web service. I would like to be able to change the IP on the test application so that it "seems" that it comes from different countries (this will help me test goals in google analytics too) - is this possible - to change/simulate that my application is located in another country (France, Germany, Belgium, England, US, etc...)
It's possible to use a Proxy or an VPN Tunnel, but you'll need an End-Point in the country you want. But, there are also plenty of lists around the web for this.
The other answers more accurately provide a solution, but you could always fake it. Utilise your own small private network and provide a facade to handle IP locationing for DEBUG vs. PRODUCTION mode. All of this of course wouldn't trick Google ;-) but it would help solidify your application.
Sorry for possibly being redundant.
The obvious solution is to "bounce" through a proxy ser ver in each of the countries you wish to test for. I've had good luck in the past with sites such as proxy2free or publicproxyservers in the past.
Other solutions would involve running a client from a host in one of these countries, by way of a VPN / RDP / RAdmin-type session, but that implies owning assets or knowing people in these countries who would trust you with using their hosts in this fashion.
Another solution involves a bit of a program change in your application. By detection of a particular trigger (could be one of several different IPs but from the same country where you reside, could be some added parameter on the url such as &ctrytest=Spain etc.) your application would substitute the IP with one of several foreign IP (from the desired countries) at the level of the country detection logic in your code, but otherwise using the real IP from your client request to actually serve the application.
You probably realize it based on the previous answers, but just to be sure: IP addresses are not a certain indicator of the country a user is in. For example, I once worked in the US for a UK-based company, and we used IP addresses allocated to a UK-based ISP.
Ultrasurf may help: http://ultrasurf.en.softonic.com/
I don't think you can specify though, exactly where in the world your request is sent from.
I am interested to know how the DNS requests to political sites differ in different countries.
I need to know how I can send a DNS query to a remote computer, let say, in China. Then, I want to compare the results to US. The goal of the experiment is to get a hand-on experience on the concept about DNS poison. I feel my lectures so theoretical.
How can you compare DNS requests between China and US, such that I can investigate DNS poisoning?
This depends a bit on how the queries are being altered. If the server is giving different results based on your locality, then asking it directly will not be of any use. If you're queries are being poisoned by a caching server in between, these methods might help.
If you have shell accounts in different parts of the world you can perform a simple test.
I'm using 'dig', which is available on most *nix systems. If you're running Windows you might want to search for an alternative in this list of DNS tools
To find the responsible DNS servers
dig ns domain-in-question.com #the.dns.server.you.want.to.use
To get the IP addres for the hostname
dig a host.domain-in-question.com #the.dns.server.you.want.to.use
(You can skip the #.. part to run with your current server)
I recommend trying both of these from different parts of the world to see if the server itself is giving different results or if the caching servers on the way there are being poisoned.
Also, searching for 'how to poison dns' gave me a number of practical results.
You can just use nslookup (the server command lets you specify the DNS server to ask)
Try this web tool:
http://www.kloth.net/services/dig.php
As for learning about DNS poisoning, every computer has settings for which DNS server to trust, and so on. If one of them in a chain is compromised, every computer downstream will receive bad information.
If the remote servers are correctly configured, they won't let you interrogate them.
Any recursive resolver should be configured to only provide answers to the clients its intended to serve.