This is my first time doing a project in ASP so this is my first time running into the dreaded A potentially dangerous Request.Form value was detected from the client... error.
I've looked around the web and stackoverflow, and can't seem to find a "best practice" way to deal with this. I don't exactly want to turn off all validations, because that would mean that in every single text box where the user can input data, I need to have to encode/decode the text myself right? Which leads me to think that this is a very easy way to leave security holes in your site.
Another seemingly common fix is to have the validation break gracefully and lead to a nicer looking error page for the user, which isn't a bad fix, but it can't be the best right?
I've noticed on most sites, if you enter the HTML encoding for a apostrophe for example, it automatically turns into a apostrophe character. As I type this now, if I try to type in ' ; (take away the space between 9 and ; and this is the HTML encoding for apostrophe), it automatically gets converted into a '. There has to be a way in ASP.net 4.0 to do the same thing?
Btw, I'm using c# as my backend server side code.
Related
I have a standard ASP.Net WebForms application running on IIS 7.0 with an Integrated Managed Pipeline. Many of the images on our site have spaces in their files names (e.g. './baseball drawing.gif'). When we place these images into our html pages we url encode the paths so that our html img tags look like this <img src='./baseball%20drawing.gif' />
Now, the problem comes in when certain search engines and webcrawlers try to index our site. When they scrape our pages they will html encode our already html-encoded paths getting image links like this './baseball%2520drawing.gif' where %25 is the url encoding for '%'. This causes two problems:
When users get results from these search engines they receive broken links.
When users attempt to navigate to these broken links it throws errors in our system.
As you can see this is a lose lose situation. Users get broken links, and we get noise in our error logs.
I've been trying to figure out how to correct this problem with no luck. Here is what I've tried:
Set <requestFiltering allowDoubleEscaping='true'> in web.config to prevent the "404.11 URL Double Escaped error". This fixed the first error but caused a new one, "a potentially dangerous Request.Path was found".
Removed the '%' from the <httpRuntime requestPathInvalidChars> to prevent the "potentially dangerous Request.Path" error. This fixed the second error but now we have a third one, "Resource can't be found".
I placed a break in my code to watch Request.Path. It looks like it is right with a value of 'Ball Image.gif' instead of 'Ball%2520Image.gif'. With this being the case I'm not sure why it isn't working.
I feel like I have a super hack where I am having to disable everything without really understanding why nothing is working. So I guess my question is three fold
Why did solution attempt 1 not take care of the problem?
Why did solution 2 not take care of the problem?
Why does my Request.Path look right in step 3 but it still doesn't work?
Any help anyone can provide would be greatly appreciated.
OK, after much searching of the internets and plenty of experimentation I think I finally understand what is going on. My main problem was a case of extreme confirmation bias. Everything I read said what I wanted to hear rather than what it actually said. I am going to summarize greatly the key points I needed to understand in order to answer my question.
First, I needed to understand that IIS and ASP.Net are two different applications. What IIS does in a nutshell is receive a request, route that request to an application that handles it, gets the output from the handling application, and then sends the output from the application back to the requester. What ASP.Net does is receive the request from IIS, handle it, and then pass the response back to IIS. This is a huge over-generalization of the whole process but for my purposes here it is good enough.1
Incoming ASP.Net requests have to pass through two gatekeepers. The IIS7 RequestFiltering module(configured in system.webserver/requestFiltering2), and then the ASP.Net HttpRuntime request filters(configured in system.web/httpRuntime3).
The IIS RequestFiltering module is the only one that normalizes incoming requests and it only applies normalization ONE time. Again I repeat it only applies it ONE time. Even if <requestFiltering allowDoubleEscaping="true" /> it will still only apply normalization once. So that means '%2520' will be normalized to '%20'. At this point if allowDoubleEscaping is false IIS will not let the request through since '%20' could still be normalized. If, however, allowDoubleEscaping is set to true then IIS7 will pass off the request '%20' to the next gatekeeper, ASP.Net. This was the cause of the first error.
The Asp.net filter is where the requestPathInvalidCharacters are checked. So now our '%20' is invalid because by default '%' is a part of requestPathInvalidCharacters. If we remove the '%' from that list we will make it through the second gatekeeper and ASP.Net will try to handle our request. This was the cause of the second error.
Now ASP.net will try to convert our virtual path into a physical one on the server. Unfortunately, we still have a '%20' in our path instead of the ' ' we want so ASP.Net isn't able to find the resource we want and throws a "resource can't be found error". The reason the path looked right to me when I broke in my code is because I placed a watch on the Request.Url property. This property tries to be helpful by applying its own normalization in its ToString() method thus making our %20 look like the ' ' we want even though it isn't. This was the cause of the final error.
To make this work we could write our own custom module that receives the request after the first two gatekeepers and fully normalizes it before handing it off to ASP.Net. Doing this though would allow any character to come through as long as it was URL encoded. For example, we normally don't want to allow a '<' or a '>' in our paths since these can be used to insert tags into our code. As things work right now the < and > will not get past the ASP.Net filter since they are part of the requestPathInvalidCharacters. However, encoded as a %253C and a %253E they can if we open the first two gates and then normalize the request in our own custom module before handing it off to ASP.Net.
In conclusion, allowing %2520 to be fully normalized can't be done without creating a large security hole. If it were possible to tell the RequestFiltering module to fully normalize every request it receives before testing that request against the first two gatekeepers then it would be much safer but right now that functionality isn't available.
If I got anything wrong let me know and I hope this helps somebody.
If you want to allow double-escaping, you can follow the instructions at
http://www.iis.net/ConfigReference/system.webServer/security/requestFiltering
It worked for me on IIS 7.0 with no other configuration required. Double-escaping has no impact for the code of the web site I implemented it on; I don't know what potential security implications there could be for other sites.
My colleagues and I have been debating how to best protect ourselves
from XSS attacks but still preserve HTML characters that get entered
into fields in our software.
To me, the ideal solution is to accept the data (turn off ASP .NET
request validation) as the user enters it, throw it in the database
exactly as they entered it. Then, whenever you display the data on the
web, HTML-encode it. The problem with this approach is that there's a
high likelihood that a developer somewhere someday will forget to
HTML-encode the display of a value somewhere. Bam! XSS vulnerability.
Another solution that was proposed was to turn request validation off
and strip out any HTML users enter before it is stored in the database
using a regex. Devs will still have to HTML-encode things for display,
but since you've stripped out any HTML tags, even if a dev forgets, we
think it would be safe. The drawback to this is that users can't enter
HTML tags into descriptions and fields and things, even if they
explicitly want to, or they may accidentally paste in an email address
surrounded by < > and the regex doesn't pick it up...whatever. It
screws with the data, and it's not ideal.
The other issue we have to keep in mind is that the system has been
built in the fear of commitment to any one strategy around this. And
at one point, some devs wrote some pages to HTML encode data before it
gets entered into the database. So some data may be already HTML
encoded in the database, some data is not - it's a mess. We can't
really trust any data that comes from the database as safe for display
in a browser.
My question is: What would be the ideal solution if you were
building an ASP .NET web app from the ground up, and what would be a good
approach for us, given our situation?
Assuming you go ahead and store the HTML directly in the database, in ASP.NET/MVC Razor, HTML-encoding is done automatically, so your negligent developer would have to really go above and beyond the call of duty to introduce the XSS. With standard webforms (or the webform view engine), you can force developers to use the <%: syntax, which will accomplish the same thing. (albeit with more risk that the developer will be negligent)
Furthermore, you could consider only selectively disabling request validation. Do you really need to support it for every request? The vast majority of requests, presumably, would not need to preserve (or allow) the HTML.
Using a regex to strip html is fairly easy to defeat and very difficult to get correct. If you want to clean HTML input it's better to use an actual parser to enforce strict XML compliance.
What I would do in this situation is store two fields in the database: clean and raw for the data. When the user wants to edit their content, you send them the raw data. When they submit changes, you sanitize it and store it in the clean field. Developers then only ever use the clean field when outputting the content to the page. I would even go so far as to name the raw field dangerousRawContent so it's obvious that care must be taken when referencing that field.
The added benefit of this technique is that you can re-sanitize the raw data with improved parsers at a later date without every loosing the originally intended content.
i am using file upload on the user front. is there any way if some exception occurs the html file control is able to persist its value on post back(i know it can't) but any work around?. i am using server side validations.
This could get messy.
Just a couple of points,
If an exception occurs do you really think it is wise to save the file?
By its nature an exception is telling you an error has occurred and processing should not continue. I would think the only time you should save the file is when you can recover or handle the exception without any side affects.
In which case you have to ask yourself, can you not just prevent the exception in the first place?
Always, always do validation on BOTH client and server side.
Why? Well to prevent this sort of thing, at least as much as possible. Let the user know there is a problem before posting, much better experience for them.
It terms of the problem, the best thing you could do is potentially keep the file path, and if an error occurs put that value back into the input box.
This wont be pretty, the input type for files is one of those special cases where the browser is responsible for a lot of the user interaction.
Remember in most cases what is happening is the browser is collection the byte information from the file and passing that around as part of the request, this is not something you are going to be able to manipulate easily.
We got a long-running website where XSS lurks. The problem comes from that some developers directly - without using HtmlEncode/Decode() - retrieve Request["sth"] to do the process, putting on the web.
I wonder if there is any mechanism like HTTPModule to help us HtmlEncode() all the items in a Http request to avoid XSS to some extent.
Appreciate for any suggestion.
Rgds,
Ricky
The problem is not retrieving Request data without HTML-encoding. In fact that's perfectly correct. You should not encode any text until the final output stage when you spit it into an HTML page.
Trying to blanket-encode incoming parameters, whether that's HTML-encoding or SQL-encoding, is totally the wrong thing. It may hide XSS holes in your app but it does not fix them. You will still have a hole if you output content that hasn't come from parameters, or has been processed since then. Meanwhile the automatic encoding will fill your database with multiply-escaped &amp;amp;amp;amp;amp; crud.
You need to fix the output stage, that's where the problem lies.
Like bobince said, this is an output problem, not an input problem. If you can isolate where this data is being output on the page, you could create a Filter and add it to the Response object. This filter would isolate the areas that are common output and then HtmlEncode them.
I'm creating an Asp.Net program UI where users can browse and change information in a database. For this reason, they need to be able to use all forms of chars, but I still need to keep the program HTML and SQL itself secure. For that reason, I'm using a self-built method that replaces dangerous chars such as '<' etc with their html-codes while they're being handled outside of a textbox (issued on page-load so they have no functionality in there).
Now my dilemma: To be able to do this, I have to disable the Validaterequest parameter as per the topic, the program will issue a complaint. What are the possible consequences of setting it to False?
The SQL query is parametirized already, and I filter out the following marks only:
& # < > " ’ % # =
Question: am I leaving the program open for threats even if I handle the chars above? Basically this is an intranet application where only a few people will be able to access the program. Nevertheless, the information it accesses is fairly important so even unintentional mishaps should be prevented. I literally have no idea what the Validaterequest thing even does.
Edit: Alright, thx for the answers. I'll just go with this then as initially planned.
The main things Validate Request is looking for are < and > characters, to stop you opening your site up to malicious users posting script and or HTML to your site.
If you're happy with the code you've got stripping out HTML mark-up, or you are not displaying the saved data back to the website without processing, then you should be ok.
Basically validating user input by replacing special characters usually cause more trouble and doesn't really solve the problem. It all depends what the user will input, sometimes they need the special characters like
& # < > " ’ % # =
think about savvy users could still use xp_ command or even use CONVERT() function to do a ASCII/binary automated attack. As long as you parametrized all input, it should be ok.
i think that the problem is not only about SQL injection attacks, but about Cross Site Scripting and JS execution attacks.
To prevent this you cannot rely on parametrized queries alone, you should do a "sanitization" of the html the user sends! maybe a tool like html tidy could help.