We have a web application where the users from whole world would upload there file at a same time. We want an effecient, robust upload system. Max file size would be 50 MB.
There would be atleast 1lac users uploading at same time.
Please suggest which is the better upload system, FTP or HTTP?
Currently we have http based upload where in we do get some errors like connection problem, session time out, time out error, etc...
Even suggest me for any 3rd party ftp upload tools if you come accross.
I will suggest you yo go with HTTP, because it is much favorable in terms of user convenience.
If you are having critical issues with Large file upload then Please have a look at Darren johnstone's Large File upload library for ASP.Net.
Still If you have to go with FTP then I will suggest you to go with using some Client Side reach technology which runs under browser like FLASH, JAVA Applets ( or might be SilverLight )
Depends on what you're doing.
every user I ever met knew how to use a browser, but the standard random user doesn't even have an FTP client installed. So usually HTTP uploads aren't really problematic. I wouldn't wanna upload huge files, but 50megs isn't that bad yet. If you want an FTP upload you probably would go for a java applet, so your users are guaranteed to have even the software needed to upload their files. Any PHP/WebFTP things will just run you into the same problems again.
Sometimes if I don't know if things I want to do work well, its a good time to look at how others are doing it. Gmail for example has a fabulous upload system. imageshack, millions and millions of users are uploading their stuff their, basically thats all the page does, and all of them use "normal" HTTP, with a little bit of JavaScript sugar to display the progress.
edit: here is an example with PHP: (although u seem to be using asp, it might still help)
http://www.devpro.it/upload_progress/
Related
I know I cannot access environnment variables directly in Flash.
My project is a local swf file, run from flash player and not through browser.
The goal is to protect the SWF to be played from an unauthorized PC.
(this is my client requirements).
My idea was to embed it into an EXE (made in Delphi for instance) as activeX.
I am not sure it is the best solution.
I think AIR is even more complex to be done.
Besides, how to forbid the access of the SWF directly ?
Maybe embedding the swf any way ?
Any suggestions, tips are welcome.
regards
I'm going to preface this by saying that I don't think there's a 100% way to stop unauthorised access - if there was, there'd be no such things as pirated copies of windows, or flash. The best you can do is make it hard to hack.
Some suggestions:
You can actually access environment variables, by calling an external process in AIR, using NativeProcess (this link has a quick writeup: http://www.tikalk.com/js/get-windows-environment-variables-air-application) - but it's trivial to hack the .bat or add the env var
You can implement your own serial key system and give out keys to legitimate users. It would ideally need to be verified by a server call
You can code a "phone-home" server call - the app won't work without it. How you identify your users is really up to you; you could try via IP, but it's not perfect
You could disable local execution (check out SecureSWF), and run it online, behind a login wall
You could disable local execution, and run it via an intranet, so people in a company can use it, but not the general public
Depending on your app, on startup, you can download necessary files (content) from the web. This can either necessitate a login, or you can block unauthorised IPs. This is how Ubisoft DRM works on some of their games.
In a similar vein, you can download other SWF files that contain the actual logic of your application. These SWFs would only be stored in memory, never saved to disk
With all of these, the app can eventually be hacked open and modified (e.g. your server-check code could be removed, so the phone-home never happens). At the very least, run your SWF through something like SecureSWF (http://www.kindi.com/) to obfusticate the code before any public release.
It all comes down to how much effort you want to put into tackling the issue. For all the of suggestions that involve the internet, if the network is down, you won't be able to use your app, which understandably will cause frustration. For all of the suggestions that don't involve the internet, you will never know if it was successful or not.
I've done a good bit of research to find an upload component for .NET that I can use to upload large files, has a progress bar, and can resume the upload of large files. I've come across some components like AjaxUploader, SlickUpload, and PowUpload, to name a few. Each of these options cost money and only PowUpload does the resumable upload, but it does it with a java applet. I'm willing to pay for a component that does those things well, but if I could write it myself that would be best.
I have two questions:
Is it possible to resume a file upload on the client without using flash/java/Silverlight?
Does anyone have some code or a link to an article that explains how to write a .NET HTTPHandler that will allow streaming upload and an ajax progress bar?
Thank you,
Austin
[Edit]
I realized I do need to be able to do resumable file uploads for my project, any suggestions for components that can do that?
1) Is it possible to resume a file upload on the client without using flash/java/Silverlight?
No. The actual HTTP protocol itself does not support resume of partial uploads, so even if you did use flash or silverlight, you'd still need to use something else like FTP on the server.
I've "solved" this problem in the past by writing a custom client application in C# which broke the file down into small chunks (2meg), transmitted those separately, and then the server combines them all back together.
2) Does anyone have some code or a link to an article that explains how to write a .NET HTTPHandler that will allow streaming upload and an ajax progress bar?
While this doesn't solve the 'resume' problem, I've used SWFUpload on the client side and it worked brilliantly. It provides a smart file browser (where you can prompt the user for only jpeg files, etc) and upload progress tracking, all without needing to modify your server at all.
It's not possible to resume an upload using standard HTML file input control, because the whole file gets submitted with the HTTP request.
I've used NeatUpload in the past, which gives you a progress bar. It's under an LGPL license, so you don't need to pay for it and it's open source.
Nothing more to add about the resume problem.
I used (and keep on using) telerik radUpload and I am quite satisfied with it
(it can even be used in medium trust mode which was quite important for me). The only problem I had (and was not able to fix) is to upload files bigger than 2GB...
SlickUpload is pretty solid and a lot of big companies use it from what the site says.
This is probably too late for your project, but POW Upload have now implemented auto resume upload in their new version. We're about to implement it on our site.
I have a requirement by my client to be able to upload extremely large files.
I'm talking about 7 GB files. The website they are currently running on is a ASP.NET 4.0 app, so obviously the standard upload scheme for my web app is not going to work.
I'm tossing around multiple options trying to figure out what the best route to go would be.
One option I'm thinking about seeing if I can do would be to have a BitTorrent Uploader. The end users for this app will typically have the same file on hand, so the idea would be that an end user would go to the site, say that they wanted to upload a file. At that point, they would pick the file, and then the server would immediately mark that person as a seed for that file. Then, my web app would go to a preconfigured leech on our side, and instruct the leech to download the file. I would expect at some point during or after this process the torrent would do some magic to find other seeders on the client's network, or wherever, but that's the idea.
Is there any technology out there already that does this? Or am I describing something that I'm going to have to build from the ground up?
It doesn't sound like it's going to be easy to do this with BitTorrent. In order for BT to work, you need torrent files. In order to create a torrent file for a particular file, you need that file (the torrent file basically contains a hash of the file). In general for a torrent, you need a tracker. You could rely on a public one, but that could be a risky dependency. You could operate your own, but that has other challenges (for one, you'd have to make sure it's locked down so it doesn't become a free-for-all for all the latest movies, music & TV).
Assuming you have a tracker in place, you then need to coordinate the downloading of torrents. Your users are going to have to create the torrent files, which is an extra complicated step, then presumably upload them via usual HTTP methods. As well as getting the user to upload the torrent, you'd have to remind the user to start seeding the torrent in their client of choice. You'd then want to automatically begin leeching the torrent (again, security issue here - what if a user uploads a completely unrelated torrent for the latest episode of House?). Apart from the security problem, this is probably the easiest part - most torrent clients can be configured to watch a directory and automatically start downloading torrent files in that directory. Once you've started downloading, you have to make sure that the user continues seeding the torrent until you've completed, otherwise you'll be stuck with a useless partial file.
It could all work, but without a fair bit of customisation work it's going to be a convoluted process at best for your users, and quite possibly beyond them. Obviously I don't know your specific requirements, but I'd be looking at more traditional file transfer protocols, like FTP.....
I understand that file upload speeds are limited by the upload speed of the internet connection among other things. Is it possible to use jquery or some other method to compress the file locally before upload and then upload a file to the server? Any other solutions?
While others have already provided answers, one thing you might be able to do (depending on how your website is setup) is, once the user has chosen the file, begin the upload process immediately. That way, if the user has to fill in additional information about the file (maybe a description of the file, a different name for the server, keywords, etc), their file is uploading in the meantime, and the information can be provided later.
Other than that, you're SOL.
If upload speed is a concern, perhaps consider a client side application the user has to download.
Or a flash based uploader. Using flash you'd get more control over the upload and it is consistent across browsers. This is what YouTube does to allow 2GB video uploads with minimal stress on the user's part. It doesn't make it faster if the client's connection is poor, but it helps with the reliability of the upload.
The browser already takes care of all the little optimizations that would make it faster on the client side, so no, you can't really use JavaScript to speed up a file upload. There isn't much you can do if the clients connection is the bottle neck.
No, you can't read the local filesystem in JavaScript. You can't do it with Flash or Java under the default config, either (with the partial exception of Flash 10). Further, there is no standard way to send compressed requests (the way there is for responses).
The upload time will be determined by a variety of factors. So, for example, network speed, web server response time, upload file size, and so on. Check with your IT department and go over those points. If the issue is one of file size, there are ways to compress and reduce file size in Android. Refer to this sample code for Android compression and document capture.
https://github.com/ExtrieveTechnologies/QuickCapture
Is it possible to upload a virus to a remote computer using a webform? If yes how do we prevent this from happening assuming we are limiting file types to just images?
The best way to avoid problems with user uploaded files of any kind is to have a command line virus scanner on the server, which you use to scan the files after upload. If the scanner result is positive, delete the file, record their IP address and inform the user.
It's a pain to setup first time but it's a life saver.
As I understand, you have a computer with an ASP.NET webserver that has a webform where you can upload files. And you are afraid that someone might upload a virus?
Well, unless you execute the file in some way, there is very little risk. It's just going to sit on your disk as a bunch of bytes.
Now, there is a very small chance that if you somehow process the image (say, resize it), and the processing software has a specific bug that the attacker knows about, then he could hijack the process with a carefully crafted image. But guess how big that risk is.
A few ways to reduce that risk even further are:
Keep the processing software up to date;
Run the processing software as a separate process with very few privileges (sandbox/virtual machine?), and kill it (and its child processes) if it takes too long;
Run the uploaded files through an up-to-date anti-virus software.
Don't use .NET/GDI/GDI+ processing functions (which are popular and people are trying to hack), but use some small 3rd party software (which nobody bothers to hack) that has all the necessary routines (image reading, writing and processing) re-implemented itself.
Although, honestly, I don't think the risk is large enough to worry about it.
First, note that it's unlikely that your server will be affected by a virus as it isn't going to be trying to "view" these files (unless you're doing something specific). So the main concern is to keep other users safe when they view these files.
If you're running the images through some sort of resize process (maybe to make them not-so-big so they download in a reasonable amount of time) then you are inherently creating a new image, which you can be fairly confident is free of viruses. This is a great way to ensure that the image is really an image and that it is free from malicious content.
you can't - even images may contain malware. http://news.cnet.com/JPEG-exploit-could-beat-antivirus-software/2100-7349_3-5388633.html
You could scan them with ClamAV after upload to ensure everything was ok. I think there are wrappers for most programming languages.
Yes, it is possible. But it relies on the remote computer (the server) to execute some part of the upload. This is relatively rare, but if you can not avoid executing the upload you should virus check the upload or run the image in a sandbox.
Obviously, your server could be vulnerable to some form of attack using the HTTP POST method - follow your chosen HTTP server's mailing list for any vulnerability like this.
I might be stating the obvious here, but you could scan all files with a virus checker when they have been uploaded?
just limiting filetype wont help much .. .
coz virus this days can infect any files .. doc to psd anything..
May used think about using AV for Server .. one such product can be found here
http://www.f-secure.com/en_EMEA/products/business/servers/anti-virus-for-servers/index.html
Just a thought. If possible, wouldn't a recompression/rescaling etc to the image corrupt the virus and make it non functional?