I have a requirement by my client to be able to upload extremely large files.
I'm talking about 7 GB files. The website they are currently running on is a ASP.NET 4.0 app, so obviously the standard upload scheme for my web app is not going to work.
I'm tossing around multiple options trying to figure out what the best route to go would be.
One option I'm thinking about seeing if I can do would be to have a BitTorrent Uploader. The end users for this app will typically have the same file on hand, so the idea would be that an end user would go to the site, say that they wanted to upload a file. At that point, they would pick the file, and then the server would immediately mark that person as a seed for that file. Then, my web app would go to a preconfigured leech on our side, and instruct the leech to download the file. I would expect at some point during or after this process the torrent would do some magic to find other seeders on the client's network, or wherever, but that's the idea.
Is there any technology out there already that does this? Or am I describing something that I'm going to have to build from the ground up?
It doesn't sound like it's going to be easy to do this with BitTorrent. In order for BT to work, you need torrent files. In order to create a torrent file for a particular file, you need that file (the torrent file basically contains a hash of the file). In general for a torrent, you need a tracker. You could rely on a public one, but that could be a risky dependency. You could operate your own, but that has other challenges (for one, you'd have to make sure it's locked down so it doesn't become a free-for-all for all the latest movies, music & TV).
Assuming you have a tracker in place, you then need to coordinate the downloading of torrents. Your users are going to have to create the torrent files, which is an extra complicated step, then presumably upload them via usual HTTP methods. As well as getting the user to upload the torrent, you'd have to remind the user to start seeding the torrent in their client of choice. You'd then want to automatically begin leeching the torrent (again, security issue here - what if a user uploads a completely unrelated torrent for the latest episode of House?). Apart from the security problem, this is probably the easiest part - most torrent clients can be configured to watch a directory and automatically start downloading torrent files in that directory. Once you've started downloading, you have to make sure that the user continues seeding the torrent until you've completed, otherwise you'll be stuck with a useless partial file.
It could all work, but without a fair bit of customisation work it's going to be a convoluted process at best for your users, and quite possibly beyond them. Obviously I don't know your specific requirements, but I'd be looking at more traditional file transfer protocols, like FTP.....
Related
I know I cannot access environnment variables directly in Flash.
My project is a local swf file, run from flash player and not through browser.
The goal is to protect the SWF to be played from an unauthorized PC.
(this is my client requirements).
My idea was to embed it into an EXE (made in Delphi for instance) as activeX.
I am not sure it is the best solution.
I think AIR is even more complex to be done.
Besides, how to forbid the access of the SWF directly ?
Maybe embedding the swf any way ?
Any suggestions, tips are welcome.
regards
I'm going to preface this by saying that I don't think there's a 100% way to stop unauthorised access - if there was, there'd be no such things as pirated copies of windows, or flash. The best you can do is make it hard to hack.
Some suggestions:
You can actually access environment variables, by calling an external process in AIR, using NativeProcess (this link has a quick writeup: http://www.tikalk.com/js/get-windows-environment-variables-air-application) - but it's trivial to hack the .bat or add the env var
You can implement your own serial key system and give out keys to legitimate users. It would ideally need to be verified by a server call
You can code a "phone-home" server call - the app won't work without it. How you identify your users is really up to you; you could try via IP, but it's not perfect
You could disable local execution (check out SecureSWF), and run it online, behind a login wall
You could disable local execution, and run it via an intranet, so people in a company can use it, but not the general public
Depending on your app, on startup, you can download necessary files (content) from the web. This can either necessitate a login, or you can block unauthorised IPs. This is how Ubisoft DRM works on some of their games.
In a similar vein, you can download other SWF files that contain the actual logic of your application. These SWFs would only be stored in memory, never saved to disk
With all of these, the app can eventually be hacked open and modified (e.g. your server-check code could be removed, so the phone-home never happens). At the very least, run your SWF through something like SecureSWF (http://www.kindi.com/) to obfusticate the code before any public release.
It all comes down to how much effort you want to put into tackling the issue. For all the of suggestions that involve the internet, if the network is down, you won't be able to use your app, which understandably will cause frustration. For all of the suggestions that don't involve the internet, you will never know if it was successful or not.
I had posted my question in the below link on how to protect my code using Hg
How do you protect code from leaking outside?
The question is:
In a DVCS scenario how can we restrict the code leakages? Is there any way technically to restrict this like when I move our of my work automatically history of codes should not be viewed?
The more programmers need to work on the code, the greater their need for collaborating and having networked access to the code. You'll have to define what the threat model is: what are you trying to protect against, in such a way that you define legitimate and illegitimate access.
In general the source code is just a bunch of text files. Whoever has access to them, can ,,steal'' or ,,pirate'' them, regardless of whether they are stored in CVS, git, Subversion, Mercurial, Windows shared folder, etc.
Distributed version control systems make it easy to grab a copy of all the history, but a hypothetical disgruntled employee can grab complete history from CVS server too. Or from backup tapes, or by manually copying the files over the years.
The only thing you can do is to limit copying all the data in your organization.
iPods
removable USB hard drives
upload to untrusted web sites on the public Internet
SCP/FTP access to servers outside your company
I wouldn't worry all that much, unless your work really needs to be super-secret (think military and such). If you treat your employees right, they will have no reason to hurt your organization.
Is it possible to upload a virus to a remote computer using a webform? If yes how do we prevent this from happening assuming we are limiting file types to just images?
The best way to avoid problems with user uploaded files of any kind is to have a command line virus scanner on the server, which you use to scan the files after upload. If the scanner result is positive, delete the file, record their IP address and inform the user.
It's a pain to setup first time but it's a life saver.
As I understand, you have a computer with an ASP.NET webserver that has a webform where you can upload files. And you are afraid that someone might upload a virus?
Well, unless you execute the file in some way, there is very little risk. It's just going to sit on your disk as a bunch of bytes.
Now, there is a very small chance that if you somehow process the image (say, resize it), and the processing software has a specific bug that the attacker knows about, then he could hijack the process with a carefully crafted image. But guess how big that risk is.
A few ways to reduce that risk even further are:
Keep the processing software up to date;
Run the processing software as a separate process with very few privileges (sandbox/virtual machine?), and kill it (and its child processes) if it takes too long;
Run the uploaded files through an up-to-date anti-virus software.
Don't use .NET/GDI/GDI+ processing functions (which are popular and people are trying to hack), but use some small 3rd party software (which nobody bothers to hack) that has all the necessary routines (image reading, writing and processing) re-implemented itself.
Although, honestly, I don't think the risk is large enough to worry about it.
First, note that it's unlikely that your server will be affected by a virus as it isn't going to be trying to "view" these files (unless you're doing something specific). So the main concern is to keep other users safe when they view these files.
If you're running the images through some sort of resize process (maybe to make them not-so-big so they download in a reasonable amount of time) then you are inherently creating a new image, which you can be fairly confident is free of viruses. This is a great way to ensure that the image is really an image and that it is free from malicious content.
you can't - even images may contain malware. http://news.cnet.com/JPEG-exploit-could-beat-antivirus-software/2100-7349_3-5388633.html
You could scan them with ClamAV after upload to ensure everything was ok. I think there are wrappers for most programming languages.
Yes, it is possible. But it relies on the remote computer (the server) to execute some part of the upload. This is relatively rare, but if you can not avoid executing the upload you should virus check the upload or run the image in a sandbox.
Obviously, your server could be vulnerable to some form of attack using the HTTP POST method - follow your chosen HTTP server's mailing list for any vulnerability like this.
I might be stating the obvious here, but you could scan all files with a virus checker when they have been uploaded?
just limiting filetype wont help much .. .
coz virus this days can infect any files .. doc to psd anything..
May used think about using AV for Server .. one such product can be found here
http://www.f-secure.com/en_EMEA/products/business/servers/anti-virus-for-servers/index.html
Just a thought. If possible, wouldn't a recompression/rescaling etc to the image corrupt the virus and make it non functional?
We have a web application where the users from whole world would upload there file at a same time. We want an effecient, robust upload system. Max file size would be 50 MB.
There would be atleast 1lac users uploading at same time.
Please suggest which is the better upload system, FTP or HTTP?
Currently we have http based upload where in we do get some errors like connection problem, session time out, time out error, etc...
Even suggest me for any 3rd party ftp upload tools if you come accross.
I will suggest you yo go with HTTP, because it is much favorable in terms of user convenience.
If you are having critical issues with Large file upload then Please have a look at Darren johnstone's Large File upload library for ASP.Net.
Still If you have to go with FTP then I will suggest you to go with using some Client Side reach technology which runs under browser like FLASH, JAVA Applets ( or might be SilverLight )
Depends on what you're doing.
every user I ever met knew how to use a browser, but the standard random user doesn't even have an FTP client installed. So usually HTTP uploads aren't really problematic. I wouldn't wanna upload huge files, but 50megs isn't that bad yet. If you want an FTP upload you probably would go for a java applet, so your users are guaranteed to have even the software needed to upload their files. Any PHP/WebFTP things will just run you into the same problems again.
Sometimes if I don't know if things I want to do work well, its a good time to look at how others are doing it. Gmail for example has a fabulous upload system. imageshack, millions and millions of users are uploading their stuff their, basically thats all the page does, and all of them use "normal" HTTP, with a little bit of JavaScript sugar to display the progress.
edit: here is an example with PHP: (although u seem to be using asp, it might still help)
http://www.devpro.it/upload_progress/
I have been utilizing two third party components for PDF document generation (in .NET, but i think this is a platform independent topic). I will leave the company's names out of it for now, but I will say, they are not extremely well known vendors.
I have found that both products make undocumented use of the filesystem (i.e. putting temp files on disk). This has created a problem for me in my ASP.NET web application as I now have to identify the file locations and set permissions on them as appropriate. Since my web application is setup for impersonation using Windows authentication, this essentially means I have to assign write permissions to a few file locations on my web server.
Not that big a deal, once I figured out why the components were failing, but...I see this as a maintenance issue. What happens when we upgrade our servers to some OS that changes one of the temporary file locations? What happens if the vendor decides to change the temporary file location? Our application will "break" without changing a line of our code. Related, but if we have to stand this application up in a "fresh" machine (regardless of environment), we have to know about this issue and set permissions appropriately.
Unfortunately, the components do not provide a way to make this temporary file path "configurable", which would certainly at least make it more explicit about what is going on under the covers.
This isn't really a question that I need answered, but more of a kick off for conversation about whether what these component vendors are doing is appropriate, how this should be documented/communicated to users, etc.
Thoughts? Opinions? Comments?
First, I'd ask whether these PDF generation tools are designed to be run within ASP.NET apps. Do they make claims that this is something they support? If so, then they should provide documentation on how they use the file system and what permissions they need.
If not, then you're probably using an inappropriate tool set. I've been here and done that. I worked on a project where a "well known address lookup tool" was used, but the version we used was designed for desktop apps. As such, it wasn't written to cope with 100's of requests - many simultaneous - and it caused all sorts of hard to repro errors.
Commonplace? yes. Appropriate? usually not.
Temp Files are one of the appropriate uses IMHO, as long as they use the proper %TEMP% folder or even better, use the integrated Path.GetTempPath/Path.GetTempFileName Functions.
In an ideal world, each Third Party component comes with a Code Access Security description, listing in detail what is needed (and for what purpose), but CAS is possibly one of the most-ignored features of .net...
Writing temporary files would not be considered outside the normal functioning of any piece of software. Unless it is writing temp files to a really bizarre place, this seems more likely something they never thought to document rather than went out of their way to cause you trouble. I would simply contact the vendor explain what your are doing and ask if they can provide documentation.
Also Martin makes a good point about whether it is a app that should run with Asp.net or a desktop app.