how to detect sharepoint crawlers in custom httpmodule - asp.net

I have written a custom http module which I successfully deployed to sharepoint. The purpose behind this module was to track if the users to the sharepoint site had accepted an EULA (represented as a cookie in the request context) and then simply redirect them to another website (running independantly) to accept our EULA etc. The problem that I am facing at the moment, is that while users with their web-browsers using the site are fine, everything works as it should, but the trouble occurs with things like the SPFarms serch failing etc when trying to index the site. My quesiton is basically, how/what should I be filtering to ensure that my module only executes my logic for a request coming from a webbrowser and how to detect any of sharepoints crawlers etc such as the search service etc? I realize i can hardcode it to check for the username that the service is running on and check the filepath extensions and filter on that, but that seems like horrible design. please advise if you know of a better way to do this please

Try filtering based on the User-Agent string, in Request.UserAgent -- just don't rely on User-Agent for security purposes, since it can be faked.

Related

can you bypass the login to a asp.net site using only a unique url?

I'm just curious if you can bypass the login into a asp.net website, which to let yall know, I have no control of using a unique url?
I have login credentials to the site, and tried using those to do this but to no avail.
So is this possible? Only thing I could think of was
http://username:password.awesomesauce.com/login/login.aspx
FYI I can log in fine, I just need this to login to the site via a 3rd party app. It is a major pain to login everytime with the app and sometimes many of the functions fail because its screwy with keeping an authenticated login
No, this is not normally possible.
This would be a very specific custom case - there is no standard for this as all logins are different, and authentication methods with databases etc. are all different.
FTP can work like this, but that is because the authentication is part of the protocol whereas in a web form it is not.
I used a firefox add on called Live HTTP Headers 0.17 to follow the url actions as it logins into the site. After turning on the capture option I logged in and it gave me this
Then I took the contents from the Content-Length catagory and appended it to the url like so
https://www.TROLLFACE.com/login/login.aspx?__LASTFOCUS=&__EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE=%2FwEPDwUJMTI2OTI3NTE1D2QWAmYPZBYCAgMPZBYMAiAPDxYCHgdWaXNpYmxlaGRkAiEPDxYCHwBoZGQCJg8WAh4LXyFJdGVtQ291bnQCGhYCAhkPZBYCAgEPFgIeBXN0eWxlBQtib3JkZXI6MHB4O2QCKA8WAh8AaGQCKg8PFgIeC05hdmlnYXRlVXJsBRZqYXZhc2NyaXB0Om9wZW5DaGF0KCk7ZGQCOQ8PFgIeBFRleHQFKUNvcHlyaWdodCAmY29weTsgMjAxMiBCIE8gWCBQYXJ0bmVycywgTExDZGQYAQUeX19Db250cm9sc1JlcXVpcmVQb3N0QmFja0tleV9fFgEFGGN0bDAwJGJ0bkNoZWNrSW52ZW50b3J5MdYH4EkMQWCgv%2FEOSMLJPNZ21rMa&ctl00%24cphMain%24txtUsername=USERNAME-GOES-HERE&ctl00%24cphMain%24txtPassword=PASSWORD-GOES-HERE8&ctl00%24cphMain%24btnLogin=Log+In
Please note the USERNAME-GOES-HERE and the PASSWORD-GOES-HERE
So far it works every single time, effectively skipping the login.

windows authentication using ajax

I want to create an internal website, preferably in asp.net, that uses ajax calls for performing all operations.
The requirements are that the site should use windows domain authentication (to act as a single-sign-on solution) for verifying which user is sending the request. What is the best way to accomplish this in a secure fashion? I am open to a solution using https or cookies or anything else feasible.
( I would prefer regular ajax using jquery since I havent used asp.net ajax but if that solves some issues more easily, then let me know)
The current way I am doing this (not through ajax) is disabling anonymous access in iis and then getting the logged in username from asp.net, but this requires the site to perform postbacks, etc which i want to avoid.
An Ajax request is not different from any other HTTP request when it comes to authentication.
Your user will most likely be authenticated when they access the index of your site. Any subsequent request, Ajax or not, will be authenticated. There is nothing special to do, and your jQuery code will look just like what it normally looks.
Windows authentication is checked on every request.
How this is checked depends which authentication method /provider is enabled and supported on the server (kerberos, ntlm, custom, etc)
Credentials are automatically sent over under certain rules (I believe intranet trusted for IE, I dont recall what else outside of that).
Firefox will prompt you if I recall every time.
Check out:
http://www.winserverkb.com/Uwe/Forum.aspx/iis-security/5707/IIS-6-0-Windows-Authentication-401-Every-Request
The important thing to note is the browser and server will take care of this and prompt the user when necessary.
Also to note is I highly highly suggest ALL traffic is under an SSL connection at this point to prevent sniffing and credential stealing/token stealking (such as is extremely easy for basic auth)
You can enable this protection inside of IIS and use all file permissions or you can control this in your web applications web.config by enabling windows authentication. How this is accomplished though depends on your server config (kerberos support, etc)
You should absolutely:
Use an https connection to transmit username and password
Authenticate on the server side, with asp.net
Leverage the asp.net session to as great an extent as possible.
If you're not familiar with ASP.Net ... well, consider this a good opportunity to learn :)
jQuery is great, but it's just a tool. In this case, it's probably not the best tool for this particular job.
IMHO...

ASP.NET Cross-Domain Cookies & Facebook Connect

I have a website which i integrate with Facebook (via FBML - JavaScript API).
I have setup the application on Facebook as per normal, specifying the "Connect URL" to be the domain of my website.
However, my application has multiple bindings in IIS for the same website.
Such as:
www.bar.com.au
foo.com.au
The domains are completely different, no relationship in the name whatsoever - so a regex-style rule is not possible (ie base domain). The domains were made different due to a combination of localization and marketing. Keep in mind these domains are baked in to an already live website, in other words i cannot change this architecture.
Is there a way i can specify BOTH of these domain's in the ONE Facebook Application settings for the "Connect URL"? Or will i have to create multiple applications?
Of course i cannot use the "Base Domain" setting as the bindings are not on the same sub-domain.
I actually have around 7 bindings in my website - so i'd rather not have to create 7 seperate Facebook applications - because this means maintaining 7 sets of API key/secret pairs in my application.
alt text http://www.freeimagehosting.net/uploads/268b234e2f.png
What's happening of course is when im on foobar.com.au, the Facebook cookies are not available to the domain.
For the meantime, i will try and create multiple ApiKey's - but i think i might run into issues. I'm going to have to go: "If domain is this, use this ApiKey", then the same logic in every single call to the Graph API. Messy stuff.
So i guess my problem/question isn't really caused by Facebook Connect, its the nature of HTTP Cookies by design.
How can i easily access these cookies cross domain? Will i need to setup a third website and direct all cookie logic to there?
If you want *.foobar.com.au to be allowed, then setup your Base Domain to foobar.com.au.
Could you set one of them up as your "Facebook Authentication" site, and direct all FB Auth-related traffic there, and then use one of a large number of cross-site communication tricks to send the token over to the original site?
In other words, regardless of the site they come in from, you'll use .foobar.com.au (for example) as the redirect URI. Then when they come in to that site with you having noted that they came from .foo.bar.com.au, you'll redirect them back where they came from passing along the access token in some cross-domain fashion (querystring, post vars, etc.)
In my current situation, timeframes have stopped me from doing the proper solution, which is what #Yuliy has highlighted.
For now, i have created multiple Facebook applications. But to keep it DRY, i have abstracted all that away behind exposed properties:
private static string _ApiToken_Site1, _ApiToken_Site2;
public static string ApiToken
{
get
{
if (Site1) return _ApiToken_Site1;
else if (Site2) return _ApiToken_Site2;
}
}
Not exactly clean, but the main thing is i did not have to touch my existing code at all, the smarts to work out which Api Key to use is in that property.
For our next project release, i'll be scrapping this and most likely implementing a WCF/ASMX web service which handles authentication from the one place (ie seperate web service on seperate domain).

Integrating Drupal + Moodle + MediaWiki with OpenID

I'd like to be able to use these "best of breed" opensource solutions, with the only requirement of some sort of single-sign-on between the different sites. I don't want my users having to log-in in 3 different places, so I though it could be possible with OpenId.
Has anyone tried something similar?
OpenID will not avoid the problem of having to sign in 3 separate times. It was allow the user to share the same login credentials between the sites, but they will have to actually log in to each of the three systems. If that is not a problem, go with OpenID. If it is, you have two options:
Use an LDAP server to authenticate on all three sites. I think all three software packages have modules/plugins for LDAP (Drupal, Moodle, MediaWiki). Once you have the LDAP server running, the rest should be easy.
Write custom modules/plugins for each platform that authenticate against a single database. Maybe you could use the Drupal database as the primary one, and have MediaWiki and Moodle authenticate with that. So, effectively, the user will only have an account on the Drupal site, but will get access to all three. This is basically the same idea as an LDAP server, but might save you some overhead and complication.
There is also the Moodle Integration module for Drupal that attempts to accomplish the same thing, only without MediaWiki in the mix. I would check that out.
Good luck!
here are three possible solutions: (1) sigle sign-in site, (2) inject login/register forms into all sites using server site includes - SSI and (3) - ajax.
Single sign-in site.
suppose you have site1.domain.com and site2.domain.com and you want to login/register at both simultaneously. Probably the easiest way to do it will be to create another domain e.g. login.domain.com that will do the job. Your login/register application will need access to databases for site1 and site2 and/or their api's. Since login status usually resides in the cookies, your login application will need to set those login cookies to both sites simultaneously (on successful login/registration) and delete on logout.
To set cookies for all sites from login.domain.com - all of the must sit on .domain.com and cookie domain parameter must be .domain.com
If your solution needs both api access (to the other applications) and access to the same database by several applications - you may need to deal with database transactions. This is because new registrations won't be visible on other sites until transaction is committed - so for example - you can not call api from within login code to retrieve cookies before committing the transaction with the new registration.
One important detail. If you already have users separately registered at site1 and/or site2 but not on both your signon site will either have to handle those cases or you'll need to sync registrations manually yourself upon deployment of your new registration system. Manual fix won't be possible when extra user input is required to complete the cross-site registration. This point also becomes important when you add new sites requiring some new user input for the registration.
Finally, carefully choose domain name handling OpenID. To the best of my knowledge it is impossible to transfer openid endorsements across subdomains without users consent - please correct me if I am wrong. You don't want to ask users to re-register just because you decide to rename the sub-domain.
server side include (ssi) method
Another solution is to inject those forms via sever-side includes into all sites. This may be considerably harder and will depend on the type of webserver in use and will work slower.
A pre-requisite here is that all your applications run on the same subdomain - so that openid works for all of them.
I've once built common user registration for MW (php) and cnprog (python/django).
My solution was to display the same exact registration form on the wiki and the forum site, while generating and processing this form with django. I did it this way because wiki and forum "skins" are so different that I did not want to surprise visitors with the dramatic change of site appearance when they go to the registration page. This is complicated and I will not do it again :) and instead would go with single sign-in method.
in order to display django output through mediawiki I've created a wiki extension printing apache "include virtual" call to glue django-generated content with the wiki output. This comes with problems.
Apache include virtual on my installation cannot POST to subrequests and cannot pass cookies from subrequests and cannot pass redirect responses (all http headers will be thrown out) to the upstream user requests.
So I've added "was_posted=true" to mark the posts for django and a secret code to prevent cross-site forgery. To get the cookies out - had them printed with cookie_morsel.output_js() in python. So javascript must run on the client for this to work. Any redirects will have to be done with javascript too. Extra work will still be needed to upload files (like avatar picture).
So single sign-on may be the best solution.
ajax may be a neat way around - just build forms in all of your sites with javascript and submit them via ajax. Will work fast and will not break appearance of your various sites,
but this won't please the folks allergic to javascript.
Actually, the only method that does not require any javascript is single sign-in site.
Posted this because I've spent enough time building this thing for MW and django - an hour of typing did not make a difference :).

How to tell where the user is coming from?

Is there a reliable way to determine where a user is coming from in an ASP.NET application? We have a web application that is linked to from two different locations. The two links are on separate domains, and they need to dictate certain user permissions within this app. Here's what I have tried so far...
Using Request.UrlReferrer (which is the Referer HTTP header). This always returned an empty string. I believe this is because the hyperlinks use Javascript to launch a popup window. Based on my research, the user agent provides this HTTP header on standard hyperlinks. Javascript popups are a different story (evidently).
A simple query string to indicate the referrer. This is not really an option because we need something that is not so easy to bypass (more secure).
Any ideas? I understand that in the grand scheme of things, this could have a better overall design/structure. Please don't post an answer suggesting I re-design everything, because that is not an option.
There's no a reliable way to tell where an user is coming from and this is not only an ASP.NET limitation, but all web applications in general. The url referrer can be easily spoofed so it is not reliable. I think the best option could be some encrypted url parameter, or cookie if you prefer.
So both pages should agree on common private keys.
Page1 will use the key to encrypt its address and pass it to Page2
Page2 will check for the presence of this parameter and try to decrypt it with the same private key used to encrypt
If this succeeds it means that Page2 will be capable to determine who called it, if not, the data has been tampered
Without the browser passing a referrer or using the querystring like you describe, there is no way to know.
Another option is to have two different landing pages on the ASP.NET application. The landing pages can set the security options and then redirect to a common homepage. This is a little more secure than the querystring option.
Or, you could place a 1x1 pixel image on the referring sites that is pulled from your ASP.NET application site. The referrer should be passed to the script and you could then set a cookie on the users machine that you can then reference when they hit the app homepage.

Resources