I would like to duplicate the functionality of some web filtering software, however I don't want the user to have to configure their browser. Some other products on the market do this without any apparent configuration in the browser settings.
The user would be installing this for themselves, so air-tight filter security is not a priority. But ease of installation and the ability to apply to an arbitrary browser would be important.
Since the vision is standalone desktop software, inserting a filter on another upstream machine is not really an option.
You will need software that runs on a network node that all internet flows through, and it will have to intercept HTTP requests and redirect them accordingly.
Some routers have this sort of capability, it can also be accomplished with linux routers using iptables and a squid proxy.
Install your program as a proxy for all HTTP traffic.
Windows Filtering Platform
Windows Filtering Platform (WFP) is a set of API and system services that provide a platform for creating network filtering applications. The WFP API allows developers to write code that interacts with the packet processing that takes place at several layers in the networking stack of the operating system. Network data can be filtered and also modified before it reaches its destination.
http://msdn.microsoft.com/en-us/library/aa366510%28VS.85%29.aspx
Related
I belong to a team of several people, divided throughout the world.
We are building software for a certain platform, lets call it "Platform S". To develop for this platform there are 2 resources that are necessary. One is the SDK, which can be installed only using certain installer, that connects directly with the Platform S centrail server, and install it in out machine. The other resource is the developer website, where people post their questions and doubts about the SDK and hardware it operates on itself.
The problem comes that to connect to these resources, both the forums and to download/update the SDK, I need to have always the same IP address.
To solve this problem, I initially created a server with a fixed IP and installed proxy software in it, so that we could configure our local machines to connect to the proxy, and all have the same IP address.
Of course, to avoid that this proxy were used with nefarious purposes by hackers, and others, I protected the proxy with a password. When accessing the forums, this was no problem, as the browser opened a small dialogue window, to ask me for the user and password. But the installer that is in charge of installing/updating the SDK does not offer me this dialogue window. The last time, I disabled the password for a while, since SDK updating is not a task that one does that often, but after just a couple hours, I already got a notice from my server provider to warn me that the server was being used for malicious purposes. So that meant that this solution was not appropiate.
What approach could I take to solve my problem? Is the proxy idea the wrong way to go?
I had Fiddler open recently with no filters on, and I was running a program from my command prompt that communicates with a server (it's a simple in-house proprietary program).
Interestingly, Fiddler was not picking up this communication, which is mostly made up of REST API calls. I did some research and found this quote, from here
With regard to why you're not seeing this traffic in Fiddler--
assuming you don't have any filters set, this suggests that whatever
mechanism you're using to send the HTTP request isn't adopting the
system's proxy settings. This means, for instance, that your code
would fail if run on a corporate computer that requires a proxy server
to reach the Internet.
However, I'm wondering why this would be the case. As far as I understand, my computer still needs to send data from my network card to the proxy server's network card. The traffic isn't bypassing my network card - it simply carries a different address. Why isn't Fiddler able to see this?
The reason is that Fiddler works not by sniffing your network card but by installing itself as the system proxy. If you click start and type "Internet Options" and choose the tab "Connections" and the button "LAN Settings" you'll see that localhost:8888 is your system proxy. Now, most well-behaved clients (e.g. IE and Chromer) respect and use the system proxy but some don't. In particular Java programs have their own ideas about which proxy they will use and you would have to set that separately. For other programs which are just hard-coded to make their own direct HTTP request you cannot AFAIK monitor them with Fiddler.
I have a website hosted in Azure Websites as a Basic tier website.
I'm currently in the development stage, yet the site is live and accessible by the outside world (at least at a basic level), so I wanted to better understand the monitoring features in the Azure management portal.
When I looked at the monitoring tab inside the portal, I see an odd pattern for HTTP success. Looking at the past 60 minutes (which I personally have not been active on), the HTTP successes are very cyclic, with 80 connections, then 0, then 40, then 0, then repeat.
Does anyone have any pointers how I can figure out what the 80 and 40 connections are. I certainly don't have any timed events in my code, so there shouldn't be any calls being made unless a person is actually hitting the site.
UPDATE:
I setup a staging server and blocked all incoming traffic except my own IP. So the same code running, just without access from the outside world. And the HTTP success appears only when I hit the server myself (as expected). This suggests that my site is being hit by an outside bot maybe? Does anyone know how to protect against this? Or at least diagnose if the requests are not legitimate, etc?
I'd say it's this setting that causes the traffic:
Always On. By default, websites are unloaded if they are idle for some period of time. This lets the system conserve resources. In Basic or Standard mode, you can enable Always On to keep the site loaded all the time. If your site runs continuous web jobs, you should enable Always On, or the web jobs may not run reliably
http://azure.microsoft.com/en-us/documentation/articles/web-sites-configure/
It's just a keep alive to avoid cold starts every time you or someone else visit your site.
Here's another reference that describes this behavior:
What the always-on feature does is simply ping your site every now and
then, to keep the application pool up and running.
And Scott Gu says:
One of the other useful Web Site features that we are introducing
today is a feature we call “Always On”. When Always On is enabled on a
site, Windows Azure will automatically ping your Web Site regularly to
ensure that the Web Site is always active and in a warm/running state.
This is useful to ensure that a site is always responsive (and that
the app domain or worker process has not paged out due to lack of
external HTTP requests).
About the traffic in general: First of all, the requests could really only come from Microsoft, since any traffic pattern like this will quickly be automatically detected and blocked when using Azure Websites - you cannot set up a keep alive like this yourself. Second, no modern bot whatsoever would regularily ping a specific page with that kind of regularity since it's all to obvious. Any modern datacenter security appliance would catch that kind of traffic and block/ignore/nullroute it.
As for your question regarding protection and security: Microsoft cannot protect your code from yourself. However, everything at the perimeter is managed and handled by Microsoft. That's one of the USP features of Azure - Firewall, Load Balancing, Spoofing, Anti-bot and DDOS protection etc. There will of course always be security concerns regarding any publicly exposed service but you can stay focused on your application while Microsoft manages the rest.
When running Azure Websites, you're in the hands of Microsoft regarding security outside of your application scope. That's a great thing, but if you really like to be able to use other security measures you'll have to set up a virtual machine instead and run your site from there.
You may want to first understand what are these requests. Enable web server logging for the website on Azure Management portal and download IIS logs for your website after seeing this pattern. Then check those to understand the URL, client ip addresses for the requests and user agent field to identify if the requests are really from search bots. Based on the observation, you can either disable some IP statically, use dynamic ip restrictions or configure URLREWRITE to block requests with specific patterns in request or request headers
EDIT
This is how you can block search bots - http://moz.com/ugc/blocking-bots-based-on-useragent
You can configure the URLREWRITE locally on an IIS server in the way described in the above article and then copy the configuration generated in the web.config or connect to the azure website directly using IIS manager as described in http://azure.microsoft.com/blog/2014/02/28/remote-administration-of-windows-azure-websites-using-iis-manager/ and configure urlrewrite rule
I'm wondering if there is a way to capture some SAML POST tokens/data in the network traffic without using 3rd party software such as Fiddler 2, and without having admin rights to the computer to upgrade web browsers or install anything? I myself would need to remote into this persons computer, and try to capture the data that I need to look at for an issue that is presenting itself. But the persons computer I would remote into does not have admin rights to install any software of any kind, or even do updates for that matter. They are running IE8. Is there a way to capture network traffic from their computer without admin rights or 3rd party software?
Honoring your request to not consider 3rd party software...
Depending on user permissions available, you could try setting IE HTTP proxy settings to use a remote IP you control - one presumably running a proxy/debug tool of choice. For example, you could run a small VM in a cloud such as Amazon EC2, run a tool such as Fiddler, Burp Suite, Charles, etc., and inspect traffic on the user's behalf. Most HTTP debuggers like this do support configuration to allow remote computers to use them as proxy.
I have 50 machines in a LAN and each of these have internet access. Can a program be developed using vc++ which will tell what are all the websites which is being opened by users in each machine?
You can easily accomplish this by writing an application which captures packets outbound on port 80 (and the associated DNS information). The problem is that this application must run on every client computer which you want to trace. The easier method, as stated by others, is to take advantage of your network architecture and tunnel all traffic through a central proxy which can record the same information.
There are many-many enterprise tools suited for just this task in the latter instance.
Route your internet traffic through a centralized proxy and monitor the traffic from proxy say using Fiddler, or something else. In case proxying is not possible, use Fiddler to generate data at known location and then collate it at required intervals.
Install a firewall, if you don't already have one, and use it to log connections.