I may NOT bother with this but if its very simple i may consider it. The site i am working on by design is to hold hundreds of thousands of files. I dont know if we'll have only one download or multiple. Right now the choices are A) Just the file B) An archive that has the file + license and conditions.
I am trying to figure out it can be efficient to offer both and use something like file.open/read prefixing an archive header before it and after it which contains the license and other zip contents. My biggest worries are doing file open/read will not be as efficient as letting the server transmit the file and if its hard to generate and change the contents of the zip dynamically (if a user wishes to change the license or if we want to add other data such as author description, author URL and a permalink on the site)
Is it efficient and how would i create the file dynamically only the original file and data pulled from the database?
PS: I am using debian/apache/asp.net using xsp.net and mono.
SharpZipLib is a very nice stream based library that you can use to create archive files.
You can use zip libraries (System.IO.Packaging.ZipPackage, dotnetzip, SharpZipLib) or event command line programs (say 7zip) for compressing the file
. The library should offer better performance.
However, important thing will be to add an caching layer i.e. zip files should be cached in file system so that they can be served directly if request comes for the same.
Related
I need to store pdf files that will keep increasing with time on a folder for my app. But I'm not sure which method is consider 'good practice' between if I should store it in a folder within the project architecture (i.e. inside Content folder) or in a folder outside the app. I tried searching online but most people just mention 'File System' and don't give examples on where do they store it.
So my question is, when people mention storing files on the File System, do they usually mean storing it in a folder inside the app (like Content folder) or do they mean storing it in a folder outside the app?
For example: let's say my apps folders/files are within the container folder: /Container/AppFolder/. Should I store the pdf files on the apps Content folder (i.e. /Container/AppFolder/Content) or should I create a new folder OUTSIDE the apps folders to hold these pdf files? (i.e. /Container/PDFFiles, so basically two folders within the container folder - one folder for the pdf files and the other folder for the app)? Which one is consider a good practice? I have considered just putting these files on the Content folder but as the amount of files keep increasing I'm not sure if is bad practice.
Thanks in advance.
You can put it where ever you want on the server, the program will just need proper permissions to that location (I use the IO namespace). However you should put thought into how to organize the files and folders. Is each file to be tied to a specific ID or just all in 1 folder? Think of things you'll need to handle like invalid file names (special characters), duplicate file names etc. Is the location strictly 1 way (upload only)? Or are you enabling download functionality? If you have download functionality stress security, probably creating a download.aspx page specifically for handling download requests and authentication.
I'm running RStudio server on an instance of Google Compute Engine. My RScript creates a map file that I would like to include in a public web site.
The file gets created OK.
Separately, I've also created a bucket and can upload images to it, viewing them from a web browser with a URL like this: https://storage.googleapis.com/...
Still, I'm confused as to how to make the image created by the R script viewable by a browser. Does the image have to find its way over to a bucket? Or is it viewable where it is somehow?
There are infinite possible solutions depending on what you want to implement and how much time you want to spend on it (and if you are the only one accessing or not and if you can share the file or they are sensible), therefore I will provide you some hints:
The easiest one is to upload the file to a Google Storage Bucket, then you can control who can access that link (a single user, a domain or everyone), it could be access by accessing with the browser with the following link:
https://storage.googleapis.com/namebucket/folder1/folder2/nome_file
There is no graphical interface, you will need to know the address to download the file (at the end it is enough to know the name). You will need to create a small script to make sure every time a image is available to upload it to the bucket and to make it public available. Or you can decide to make he bucket itself public.
The second possible solution is to do the same but to create an html page REALLY simple, basically a list of links to the files in the bucket, each time you upload a file to the bucket you update the html file. At least you would solve the issue regarding the knowing the names and you can navigate it a bit.
<html><body>
This is a link
</body></html>
If you need to expose the resources to more people, or you would like to have something more "nice" graphically you will have to spend more time and build a decent frontend. You can follow thousands of different approaches.
You have really thousands of possibilities.
P.S.
Documentation regarding uploading a file to bucket.
Documentation regarding managing access to file stored.
Notice that in this way depending on the extension of the file you want to share the browser behaves differently, a .txt, a .jpg are shown an .exe is downloaded.
I am trying to hook into the Wordpress file uploader and would like some suggestions. I would like to be able to grab the path of the source file (not WP path; i.e. K:\docs\file.pdf) so I can download another file of the same name (different extension) & path automatically (i.e. K:\docs\file.txt).
Wordpress provides hooks for after the file is uploaded but all path information at that point is internal to wordpress.
Thanks in advance!!
the path information can be obtained from normal php functions like pathinfo() and realpath() for example .
Bit too late to the party but I think what you are asking for is to find the source of the document in the clients system and based on that you want similar files there to be picked up by the browser and then sent to the user.
I think it will be a fairly complex thing to do, firstly you have to get the source of the document being added to the browser (I am not sure it can be done, but if it can be done then it will be via javascript) and then upload all the files one by one to the server hosting wordpress and you can do this by creating a custom page which will add the files uploaded to it as a wordpress attachment and then using something like jquery file uploader to upload files to that location.
But honestly, I think it would be a very complex thing, unless the client and the server are on the same machine.
I've been messing around a bit with various solutions to what I would see as a fairly common problem, but I've not yet been able to solve it in a satisfactory way.
What I wish to achieve is some kind of functionality where a user can upload new files, or select existing files to reuse them.
What I've been using so far is a combination of the filefield, filefield_sources, imce and ckeditor modules. I guess ckeditor isn't really important for the solution, but I need to be able to embed images from the archive somehow, and this is done with IMCE . Since I do not want everything to be accessible from the filebrowser I created a subdirectory and set full access to it in the IMCE settings, lets call it default/files/site
This worked fine as long as all filehanding was done through IMCE, but when I uploaded files directly from the filefield my files ended up in the default/files root, so I set up folders for my fields, for example default/files/site/movies in a field that allowed the .flv format. This worked fine to, as long as I didn't try to access the files through IMCE. It appears the folders created by filefield are not accessible from IMCE?
I'm also in a position where I need to support large uploads (200MB+), but from my experience in other projects, allowing file uploads through FTP is usually a life-saver, but from what I understand IMCE won't support files not uploaded through Drupal in some way, since they are not present in the database (giving the message: The selected file could not be used because the file does not exist in the database.)
I'm aware that I don't really have a clear question to my problem, but somehow I need to figure this out pretty fast. How would I preferably solve this? I'm aware that I'm not the first to have this problem, but I have not yet been able to find a nice and stable solution. What am I missing?
Also check this thread (http://drupal.org/node/438940) and the reference to John Locke's work at: http://www.freelock.com/blog/john-locke/2010-02/using-file-field-imported-files-drupal-drush-rescue
Well, I'm not personally familiar with IMCE off of the top of my head, but if you need files that have been uploaded via ftp to be added to the files tables, then my impulse would be to write a small module which would then allow the user to click a button and start off a batch process. (This is me assuming that you are using Drupal 6, as the batch api doesn't exist in 5.)
Said batch process would then iterate over all of the files in the appropriate directory, which I would assume you had uploaded the files to, use file_copy() (from Drupal's file API) to copy the files to default/files/site, and then would add said files to the files table, which is actually quite simple with drupal_write_record().
It might not even need to use the batch api - it somewhat matters if you're just uploading 10-30 really big files, or 200-300 MB files.
For using the batch api, I'd look at http://drupal.org/node/180528 - this has a fairly basic example of how the batch api works, which basically consists of telling the api that you want to keep calling function_a, and then inside of function_a setting your progress in the context array until you're done, at which time the batch process finishes. Then you just have whoever uploads the files via ftp to hit a button on the website to move and register the files.
I have a web page where i have an ASP.NET file upload control to upload files from client machine to Server.Now i want to do the uploading n number of times.Ex : I want to upload 100 files from my local pc to server.The 100 file names i can read from an excel file in my program.But is there any way to assign this file to the file upload control ?
No, as a security feature, FilUpload controls do not allow you to set what to download (imagine if you sign on to a website, and it is set to upload a passwords file or something).
Now there is probably another control, or a way to code around this, buut the FileUpload control will not allow it.
I would recommend using the jQuery Multifile Uploader which would take care of a UI (if you need one). And the actual uploads with Free ASP Uploads which takes care of the actual file transfer. Though it sounds like you are tkaing care of the programs programatically, so you can skip the multifule and just work with free asp upload.
You'll have to make your own Flash object or something to accomplish this, the basic HTML/ASP.Net controls won't let you do what you're looking for.
This will require creating some kind of an active or installable control. In order to get around the security hole of doing this, you're ultimately going to have to be able to execute code on the machine to select and upload the file.
And at that point, you're platform specific, so...
I would strongly suggest that instead of trying to have a web site automatically upload files for you, that you make a WinForms utility to accomplish this task and upload the files wherever you need, communicate with the web site over web services, etc.
This is a security restriction, you cant script the file selection of an upload box as it would allow hackers to write scripts to steal files off your computer.
You could use this silverlight upload utility which is my list of "things to use when I get the chance".
It has a nice UI and supports uploading many files at once. I originally tracked it down doing some research for a photography website that we were quoting for but that project fell through.
Anyway the project can be found here:
http://www.michielpost.nl/Silverlight/MultiFileUploader/
It also has full source code included so even if the control's developers abandon it you still have the choice to edit it yourself.