It appears that certain information is tied to an IP address, which is being taken advantage of by certain analytics packages such as LeadForensics and OpenTracker to identify anonymous visitors.
Where are companies like this getting information like this from? Is it contained in a WHOIS lookup against the allocation body (ARIN, APNIC, RIPE) or something else?
The above tools are pretty accurate.
Related
I am looking for as reliable and accurate / quick means possible to add in some htaccess code to block visits to a website from countries / IPs which are not in the white listed list of countries I want to allow access for. I have looked at https://www.ip2location.com/free/visitor-blocker which seems to offer a solution - for the 4 allowed countries I want to allow access - it has created a 4.1MB htaccess file! Will this mean slow access when someone attempts to view the site? I guess using a free service like this means the data is likely nowhere near comprehensive?
Does anyone have any suggestions on a good way to allow just visitors from a few countries access to a website?
It sounds like the service you used basically tried tried to brute force the blacklist. If you look into the htaccess file I'm sure you will be a long list of hard coded IP blocks.
In my opinion this is a terrible way to handle a geographic blacklist. To your original question - there is no "most reliable, most accurate, and quickest" method. Those are separate categories and you will need to preference one over the next.
For performance you could consider blacklisting at the routing level / dns server / proxy. This obviously isn't going to be the quickest way in terms of performance. There are Apache Modules that exist that allow you to use a local database to compare the incoming IP address with a list of known IP blocks from the blacklisted country. One of the main issues with this is that you need to constantly update your database to take in new IP blocks.
In my opinion the "best" method to do this is a simple redirect at the application layer using server side code. There exists several geographic API's where you can send in the IP or Hostname and get back a country of origin. An example:
$xml= new SimpleXMLElement(file_get_contents('http://www.freegeoip.net/xml/{IP_or_hostname}'));
if($xml->CountryCode == "US") {
header('Location: http://www.google.com');
}
There are two ways to block a visitor in web server. One is using firewall (.htaccess etc) and another one is using server-side scripting (PHP etc).
If you are concern of the performance of the firewall option, then you can download the IP2Location LITE database from http://lite.ip2location.com and implement the database in your local server. For every connection, you query the visitor IP address and find their country. You can redirect or block them using the PHP codes. Please find the complete steps in https://www.ip2location.com/tutorials/redirect-web-visitors-by-country-using-php-and-mysql-database
There is also another option to use remote geolocation API. However, we do not suggest this method because of network latency. It will slow down all user experience due to API queries.
I have a huge issue that has to do with whitelisting. I have been doing C++ for about 6 months now and I can't seem to figure out how to pinpoint my targets to limit who can open and use my application with a whitelist.
For example, if the user is not on the whitelist the program would tell them by the way it loads. I would like to see this done with ID's if specific ID matches with the whitelist then that person can use my program.
I have tried doing target drawbacks such as getting IP's, but doing this is so vulnerable if the IP is changed. Also, multiple programs could be opened up on different IDs on that IP, which I don't want.
Sorry if this is very confusing I have just been STRUGGLING with this whitelist I have less hair than I did before I started making the whitelist.
Thanks if you can help, tried to explain the best I could! :)
The general strategy is pretty simple.
First, specify what criteria a user should meet to be on the whitelist.
Second, specify how data about users on the whitelist will be stored.
Third, when the program starts, gather information about the user - when the program starts - that can be compared against the criteria on the whitelist.
Fourth, when comparing data about the user with stored whitelist data, start by assuming the user is NOT on the whitelist and only permit access if a match is found. If there are multiple criteria, you need to decide how to combine them to find a match (e.g. restrict a user to a specific IP, allow a user only if using an IP in a range - which will prevent a user starting the program from home, etc etc)
Fifth, take steps to ensure your program can access the stored whitelist data, but users cannot modify it.
There are many ways to target specific users. First, I need some extract information.. How can you identify a single user ? Your program should be a connection toward any server ? In that case, your user should provide an id and a password or it's a anonymous connection ?
can we trace the user geographical location in asp.net ,
I know we can get the user IP address or ServerVariables("REMOTE_ADDR")
but my need is to check the visitors by city name so can my client use this data for his business
As I said in my comment:
I am not .net guy, I am actually a PHP dev but this is quite universal
across board. Yes, you can get A country and A city, but as to whether
it is your users COUNTRY and CITY....well it is unlikely to be your
users city since most ISPs do not hold their exchanges in the same
city as the user...but you have a good chance with country.
But I thought I would elaborate a bit.
The country, especially if they are like me and using an encrypted proxy, won't be of any help to you, however services like cloudflare will give you the users IP country in HTTP headers, and it's free. Better than spending £200 a month on getting nothing but false positives from dedicated services.
A similar question was asked on asp.net forum and it was answered using a good example.
Reference : Link
How do you expect to do that? My IP is assigned by an ISP located 300
miles from me. My web host is in North Carolina, a 16 hour drive
away. My cellular link in my notebook has an IP issued from New
Jersey. It's been two decades since the last time I was in New
Jersey. 1: http://forums.asp.net/post/3072924.aspx
I am probably really late answering this, but I recently worked on this project because I needed it myself. so maybe it might help someone out there.
http://iplookup.tk/service/iplookup/{YOUR IP ADDRESS}
NB: depending on the content-type of your request the result can either be in Json or XML. by default its xml
I have a DB with user accounts information.
I've scheduled a CRON job which updates the DB with every new user data it fetches from their accounts.
I was thinking that this may cause a problem since all requests are coming from the same IP address and the server may block requests from that IP address.
Is this the case?
If so, how do I avoid being banned? should I be using a proxy?
Thanks
You get banned for suspicious (or malicious) activity.
If you are running a normal business application inside a normal company intranet you are unlikely to get banned.
Since you have access to user accounts information, you already have a lot of access to the system. The best thing to do is to ask your systems administrator, since he/she defines what constitutes suspicious/malicious activity. The systems administrator might also want to help you ensure that your database is at least as secure as the original information.
should I be using a proxy?
A proxy might disguise what you are doing - but you are still doing it. So this isn't the most ethical way of solving the problem.
Is the cron job that fetches data from this "database" on the same server? Are you fetching data for a user from a remote server using screen scraping or something?
If this is the case, you may want to set up a few different cron jobs and do it in batches. That way you reduce the amount of load on the remote server and lower the chance of wherever you are getting this data from, blocking your access.
Edit
Okay, so if you have not got permission to do scraping, obviously you are going to want to do it responsibly (no matter the site). Try gather as much data as you can from as little requests as possible, and spread them out over the course of the whole day, or even during times that a likely to be low load. I wouldn't try and use a proxy, that wouldn't really help the remote server, but it would be a pain in the ass to you.
I'm no iPhone programmer, and this might not be possible, but you could try have the individual iPhones grab the data so all the source traffic isn't from the same IP. Just an idea, otherwise just try to be a bit discrete.
Here are some tips from Jeff regarding the scraping of Stack Overflow, but I'd imagine that the rules are similar for any site.
Use GZIP requests. This is important! For example, one scraper used 120 megabytes of bandwidth in only 3,310 hits which is substantial. With basic gzip support (baked into HTTP since the 90s, and universally supported) it would have been 20 megabytes or less.
Identify yourself. Add something useful to the user-agent (ideally, a link to an URL, or something informational) so we can see your bot as something other than "generic unknown anonymous scraper."
Use the right formats. Don't scrape HTML when there is a JSON or RSS feed you could use instead. Heck, why scrape at all when you can download our cc-wiki data dump??
Be considerate. Pulling data more than every 15 minutes is questionable. If you need something more timely than that ... why not ask permission first, and make your case as to why this is a benefit to the SO community and should be allowed? Our email is linked at the bottom of every single page on every SO family site. We don't bite... hard.
Yes, you want an API. We get it. Don't rage against the machine by doing naughty things until we build it. It's in the queue.
What I really want is to limit 1 vote per person but the next best thing i can think of is limit 1 vote per IP address to prevent malicious users/hackers from severely tempering with my company's voting system. I was thinking of using a database to keep track of the IP addresses.
Update:
Sorry about not being clear in the first time aruond. What i wanted to know if limiting 1 vote per IP address was a good strategy to limiting 1 vote per person. Basically, i wanted to know if 1 unique IP address is roughly equal to 1 person. People have already mentioned that proxies and routers re-use ip addresses so unfortunately, many people can be using the same ip address.
Thanks. I think, for my case, it'll be best to NOT limit 1 vote per ip address.
I would suggest not going with the IP approach. When I looked at this before some of your large ISPs reuse IPs a lot (AOL...), but if you do use IP addresses, use a database to track them. A fast way to do it is to make it a unique key and to catch the exception as "already voted".
One good thing to add is not to show a user that their vote was not counted, just show the results, or thank them for voting. By not giving that specific error, it is harder and sometimes not even noticed by your problem users.
If you use IP addresses then you'll be limiting most companies to only one vote because they route all outbound internet traffic through a firewall or proxy server. We did this a couple of years ago and found that all AOL traffic came from only 5 ip addresses.
Generally, yes, what you would do is have a database table for the votes, and simply store choice+ip address - then when inserting, do a DB query to see if an entry already exists with the given IP.
The ideal solution would be to tie votes to user accounts which are in turn linked to more concrete presence (such as a credit card, cell phone, or other less-easily-multiplied identity source).
What exactly is the question you're asking?
The way I have always done it is to concat the user agent and ip address into an MD5 hash (in some cases this will allow people from the same IP to vote, long as they are using different browsers), and store that as a "fingerprint" for the vote the the database and add a unique key to it. As IPX Ares said, from there you can catch the duplicate key exception, and you should be good.
If you wanted to allow people to vote once a day, you could also append the Ymd to that "fingerprint", or other variations to allow x amount an hour or x amount per day.
Yes, use database. Don't rely on cookies, they can be easily deleted.
IMO, so far, IP based voiting limitation is the best option.
IP address has its limitations as we have noted from above, but there are many other characteristics a browser has which can damper mischeivious voters. BrowserID, for example, is different for just about every browser. You could use a combination of BrowserID and IP address to create a unique ID.
Another way to 'help' avoid cheating is to provide a 1 time use hash into the form then check if that's is valid before you count the vote.
For example:
When you create the voting form you make a random hash and store it in the database and put it in the form as a hidden field.
(might want to add a date field to the hash database to you can clean up the unused hashes)
Then when you get a vote POST request you can check if the supplied hash is in the database and remove it from the database so it cant be used again.
CONS:
Might load the database with high IO if the voting page has high traffic.
Can't cache the page as plan html so it puts more stress on the web app.