Given the following code which is extremely generic, I was hoping someone could tell me a bit about what is going on behind the scenes...
[HttpPost]
public ActionResult Load(Guid regionID, HttpPostedFileBase file)
{
if (file.ContentLength == 0)
RedirectToAction("blablabla.....");
var fileBytes = new byte[file.ContentLength];
file.InputStream.Read(fileBytes, 0, file.ContentLength);
}
Specifically, is the file completely uploaded to the server before my action method is invoked? Or is it the file.InputStream.Read() method call that causes or rather waits for the entire file to upload. Can I do partial reads on the stream and gain access to the "chunks" of the file as it is uploaded? (If the entire fire is uploaded before my method is invoked then it is all a moot point.)
Can anyone point me to some good info on the inner workings here. Is there any difference between IIS6 or II7 here?
Thanks,
The while file needs to be sent to the server before the action method is invoked. Quote from the documentation:
Files are uploaded in MIME
multipart/form-data format. By
default, all requests, including form
fields and uploaded files, larger than
256 KB are buffered to disk, rather
than held in server memory.
You can specify the maximum allowable
request size by accessing the
MaxRequestLength property or by
setting the maxRequestLength attribute
of the httpRuntime Element (ASP.NET
Settings Schema) element within the
Machine.config or Web.config file. The
default is 4 MB.
The amount of data that is buffered in
server memory for a request, which
includes file uploads, can be
specified by accessing the
RequestLengthDiskThreshold property or
by setting the
requestLengthDiskThreshold attribute
of the httpRuntime Element (ASP.NET
Settings Schema) element within the
Machine.config or Web.config file.
Server memory won't be consumed on the server but the file contents will be buffered to disk. Once the client has sent the whole file the ASP.NET pipeline will invoke your controller action and you could read the request stream in chunks and save it to another file which will be the definitive location of the uploaded file. The action cannot be invoked before the file has finished uploading as there might be some other fields in the multipart/form-data that come after the file and they won't be assigned in this case.
Related
I have an ASP.NET MVC application with a page that allows users to upload files. The files will be several hundred megabytes.
I am using FineUploader on the client side, which will use FileAPI/XHR if the browser supports it, otherwise will fallback to Iframe/form with enctype="multipart whatever".
So on the server side I need to evaluate Request.Files.Count > 1. If true, this is an old school upload and I save the file like Request.Files[0].InputStream.CopyTo(myFileStream) otherwise I do Request.InputStreawm.CopyTo(myFileStream).
Here's some of the actual code I've written that does this stuff: https://github.com/ronnieoverby/file-uploader/blob/master/server/ASP.NET%20MVC%20C%23/FineUpload.cs
This all works fine, but in my testing I've noticed that neither an ASP.NET MVC controller action nor an HttpHandler will begin processing until the entire file is uploaded, which is bad if the file very large because that means it's occupying a lot of the web server's RAM.
I found this: Streaming large file uploads to ASP.NET MVC which sounds promising, but I really don't have an idea of where the code resides in his application.
So, the question is: how to stream uploaded files to disk while the upload is still taking place in ASP.NET?
Update
I just saw a key detail that didn't sink in before. From the HttpPostedFile documentation:
By default, all requests, including form fields and uploaded files,
larger than 256 KB are buffered to disk, rather than held in server
memory.
Ok, that addresses the concern that the web server's RAM utilization could spike during a large upload. But, there's still a problem: After the file is completely transferred to the web server, the server has to spend time moving it to it's final destination. If the file system operation is a copy (guaranteed if the destination is on another physical disk), then the response is delayed unnecessarily.
Honestly, I could probably live with this by increasing response timeout for the upload handler/action. But, it would be nice to stream the bytes directly to their destination.
You can handle uploads in a completely customized way without buffering using
HttpRequest.GetBufferlessInputStream method. Basically you are getting access to the raw incoming data and free to do whatever you want with it.
I've just created small sample which saves raw request content to a file:
Create handler:
public class UploadHandler : IHttpHandler
{
public void ProcessRequest(HttpContext context)
{
using (var stream = context.Request.GetBufferlessInputStream())
using (var fileStream = File.Create("c:\\tempfile.txt"))
{
stream.CopyTo(fileStream);
}
}
public bool IsReusable { get { return true; } }
}
Register in Web.config:
<system.webServer>
<modules runAllManagedModulesForAllRequests="true"/>
<handlers>
<add name="UploadHandler" verb="POST"
path="/upload"
type="UploadHandler"
resourceType="Unspecified"/>
</handlers>
</system.webServer>
Create a page with a form:
<form action="/upload" method="post" enctype='multipart/form-data'>
<input type="file" name="aa" id="aa"/>
<input type="submit"/>
</form>
If the uploading and streaming is using up valuable server resources then you might wanna take a look at hosting your media files on a cloud of some sort. It's possible with ASP.NET to use a Rackspace, Amazon Cloud API have your users upload the files directly to a CDN network and then serve the content that way, I know this isn't answering your question but many people will or already have and thought I'd get my 2 cents in. Many people still not opting to use the cloud amazes me! once you go CDN you never go back. Furthermore with most CDN's you will also be given a streaming URL for your upload container where it supports lots of different movie types, and its lighting fast, not only for your users to upload too but also your never have slow speeds on your website as a result.
We'd like to restrict the maximum upload file size in our web site. We've already set the appropriate limits in our web.config. The problem we're encountering is if a really large file (1 GB, for example) is uploaded, the entire file is uploaded before a server-side error is generated, and the type of the error is different whether the file is huge or not.
Is there a way to detect the size of a pending file upload before the actual upload takes place?
Here's my relevant web.config settings that restrict requests to 16 MB:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.web>
<httpRuntime maxRequestLength="12288"/>
</system.web>
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="12582912"/>
</requestFiltering>
</security>
</system.webServer>
</configuration>
I've tried creating an HTTP module so I could intercept a request early in the request lifecycle, but the uploads seem to take place even before the BeginRequest event of HttpApplication:
public class UploadModule : IHttpModule
{
private const int MaxUploadSize = 12582912;
public void Init(HttpApplication context)
{
context.BeginRequest += handleBeginRequest;
}
public void Dispose()
{
}
private void handleBeginRequest(object sender, EventArgs e)
{
// The upload takes place before this method gets called.
var app = sender as HttpApplication;
if (app.Request.Files.OfType<HttpPostedFile>()
.Any(f => f.ContentLength > MaxUploadSize))
{
app.Response.StatusCode = 413;
app.Response.StatusDescription = "Request Entity Too Large";
app.Response.End();
app.CompleteRequest();
}
}
}
Update:
I know that client-side technologies like Flash can detect file sizes before upload, but we need a server-side workaround because we're wanting to target platforms that have no Flash/Java/ActiveX/Silverlight support. I believe that IIS or ASP.NET has a bug that's allowing large files to be uploaded despite the limits, so I've filed a bug here.
Would an ISAPI extension give me more control over request processing than HTTP modules and handlers, such as allowing me to abort an upload if the Content-Length header is seen to be larger than the allowed limit?
Update 2:
Sigh. Microsoft has closed the bug I filed as a duplicate but has provided no additional information. Hopefully they didn't just drop the ball on this.
Update 3:
Hooray! According to Microsoft:
This bug is being resolved as it has been ported over to the IIS product team. The IIS team has since fixed the bug, which will be included in future release of Windows.
The problem is that the upload happens all at once using the HTTP Post request so you can only detect it after it's done.
If you want more control over this you should try Flash based upload widgets which have this and more. Check out this link http://www.ajaxline.com/10-most-interesting-upload-widgets
Microsoft has responded on their Microsoft Connect site with the following:
This bug is being resolved as it has been ported over to the IIS product team. The IIS team has since fixed the bug, which will be included in future release of Windows.
If you are requesting a fix for the current OS, a QFE request must be opened. Please let me know if this is the route that you want to take. Please note that opening a QFE request does not necessarily mean that it would be approved.
So I guess we have to wait for the next version of IIS for the fix (unless a QFE request is fulfilled, whatever that is).
Is there a way to detect the size of a
pending file upload before the actual
upload takes place?
No. That would require access to the file size on the client. Allowing a web server direct access to files on the client would be a bit dangerous.
Your best bet is to place a line of text stating the maximum allowed file size.
OR you could create some sort of ActiveX control, java applet, etc so that you're not dependent on browser restrictions. Then you have to convince your users to install it. Probably not the best solution.
Well.... Depends how low-level you want to get.
Create a service app that acts as a proxy for IIS. (All incoming port 80 socket requests go to the service.) Have the service pass everything it receives to IIS (website listening on a different port or IP), but monitor the total request size as its received.
When the size from a give connection exceeds you're desired limit, close connection. Return a redirect to an error page if you want to be polite.
Silly, but it'll let you monitor data in transit without waiting for IIS to hand over the request.
Hey, I created a HttpHandler for downloading files from the server. It seems it is not handling anything...I put a breakpoint in the ProcessRequest, it never goes there.
public class DownloadHandler : IHttpHandler
{
public void ProcessRequest(HttpContext context)
{
//download stuff and break point
}
}
It never stops there, as mentioned. I also registered it in the web.config.
<add verb="*" path="????" type="DownloadHandler" />
I am not sure about the path part of that entry. What do I have to enter there? I am downloading txt files, but the URL does not contain the filename, I somehow have to pass it to the handler. How would I do this? Session maybe?
Thanks
Have you read How to register Http Handlers? Are you using IIS 6 or 7?
The path part should contain a (partial) url, so if in your case you are using a static url without the filenames, you should put that there. You can end the url in the name of a non-existent resource and map that to path
e.g. the url is http://myserver.com/pages/downloadfiles
and the path="downloadfiles"
If you do POST, you can put the filename in a hidden field, and extract it in the handler. If you're using GET, I'm not sure, either cross-post the viewstate or put the filename in the session like you said.
Any reason why you can't put the filename in the url?
The path for a handler needs to be the path you are trying to handle - bit of a tautology I know but it's as simple as that. Whatever path on your site (real or much more likely virtual) you want to be handled by this handler.
Now unless the kind of file at the end of that path is normally handled by ASP.NET (e.g. .aspx, .asmx but not a .txt) ASP will never see the request in order for it to go through it's pipeline and end up at your handler. In that case you have to bind the extension type in IIS to ASP.NET.
As far as identifying what file the handler is supposed to respond with you could achieve this any number of ways - I would strongly recommend avoiding session or cookies or anything temporal and implicit. I would instead suggest using the querystring or form values, basically anything which will show up as a request header.
Fianlly, I have to ask why you're using a handler for this at all - .txt will serve just fine normally so what additional feature are you trying to implement here? There might well be a better way.
I'm attempting to call a WCF service via mex from a classic ASP page. I've got the call working using the details on MSDN, but if I pass back an amount of data exceeding 8K I get an exception stating:
The maximum string content length
quota (8192) has been exceeded while
reading XML data. This quota may be
increased by changing the
MaxStringContentLength property on the
XmlDictionaryReaderQuotas object used
when creating the XML reader...
The fix for this is easy enough in .NET client: you can adjust the client config to having a binding with a readerQuotas section including an increased quota. However, since I'm building a service moniker to pass to a GetObject call within ASP, I don't have access to a config to edit. If it were a VB6 app, I could use dllhost.exe.config, but that's not the case. The bindingConfiguration node (and sub nodes) don't appear to be parameters I can set within the moniker string.
Any ideas on how I could influence this parameter within the ASP context? ASP snippet with moniker string referenced below:
Dim strXml, moniker, objProxy
moniker="service:mexAddress='http://localhost/SomeApp/SomeServices/SomeService.svc/mex', "
moniker=moniker + "address='http://localhost/SomeApp/SomeServices/SomeService.svc',"
moniker=moniker + "contract=ISomeService, contractNamespace=http://foo.com, "
moniker=moniker + "binding=WSHttpBinding_ISomeService, bindingNamespace=http://foo.com"
Set objProxy = GetObject(moniker)
strXml = objProxy.DoWork("foo", "bar")
Thanks!
Try setting your maxStringContentLength in your wcf binding configuration on the server side.
It's my understanding that the service:mexAddress moniker actually uses a WCF client behind the COM interface. If that is the case then you can store the WCF config in a file called «foo».exe.config, where «foo» is replaced by the name of the executable.
If you are running the ASP within IIS6 or IIS7, then the EXE that runs the ASP is probably w3wp.exe, which means you need to drop the config into a file called w3wp.exe.config , located in the directory c:\Windows\system32\inetsrv.
We are making an automated patching application and would like to post files on production server through asp.net page (or maybe a web service), since we can only access production server via http. The page should accept files and store them to appropriate location. The path to files will be declared in external XML file.
So, is it possible posting a base64 encoded files within body tag and HOW? maybe even any better approach?
If you plan to use Base64 encoding.
Take a look at
System.Convert.ToBase64String()
System.Convert.FromBase64String()
System.Convert.ToBase64CharArray()
System.Convert.FromBase64CharArray()
See Using XML CDATA nodes to send files via a Web Service
why not create a webservice which accepts an object like:
class postfile
{
public byte[] fileByte;
public string fileName;
}
Then add a web reference in your client app.
.net will serialize the object for you.
You will need to secure this using wse security and might require the service use impersonation to write the file on the server.