I am looking for a good solution by which we can prevent an exe file to be uploaded on server.
It will be best if we can discard the upload by just reading the file headers as soon as we receive them rather than waiting for entire file to upload.
I have already implemented the extension check, looking for a better solution.
There is a how and a when/where part. The how is fairly simple, as binary files do contain a header and the header is fairly easy to strip out and check. For windows files, you can check the article Executable-File Header Format. Similar formats are used for other binary types, so you can determine types you allow and those you do not.
NOTE: Linked article is for full querying of the file. There are cheap, down and dirty, shortcuts where you only examine a few bytes.
The when/where depends on how you are getting the files. If you are using a highly abstracted methodology (upload library), which is fairly normal, you may have to stream the entire file before you can start querying the bits. Whether it is streamed into memory or you have to save and delete depends on your coding and possibly even the library. If you control the streaming up, you have the ability to stream in the first bytes (header portion) and abort the process in mid stream.
The first point of access to uploaded data would be in a HttpModule.
Technically you can check before all the bytes are sent if you have an .exe on your hands and cancel the upload. It can get quite complicated depending on how far you want to take this.
I suggest you look at the HttpModule of Brettle's NeatUpload. Maybe it gives you a lead on how to deal with this on the level you want.
I think you can do that by a javascript by checking if the file end with .exe before submitting the data and also do the check server side.
Related
I'm making an application which plays music from a remote server, and I would like to be able to sort by author/album/year/etc. AFAIK the only way to do this is by reading the metadata but I don't want to have to download the whole audio file just to read the metadata, is there any way to separate them and download only the metadata?
BTW. I am using webdav_client for flutter, which uses dio as a back-end so instructions for that specifically would be greatly appreciated. TY
Firstly, you can usually make a request for a certain byte range by using ranged requests. This is dependent on server behavior. Most servers support it, but many don't.
Next, you need to figure out the location of the ID3 tags you want. Some versions of ID3 are located at the front of the file. Some are at the back. Therefore, you should probably request the first 128 KB or so of the file and search for ID3 data, while also getting the Length response header. Then if you don't find your tag at the beginning, you can make a request for the last 128 KB or whatever of the file, and search there.
Most MP3 files aren't very big, and bandwidth is usually plentiful. Depending on the size and scope of this project, you might actually find it more efficient to just download the whole files.
I don't think that it is possible to read just the ID3-metadata (at the beginning or the end) of the audiofile without downloading the entire file first.
One idea would be to extract this information on the server side and provide it separately, in addition to the audio file itself. To do this, you would need one of the well-known extraction tools available for your platform. However, if you need to download hundreds or thousands of companion files, I am not sure about the reliability of such a system.
I saw this question:
ASP.NET File Upload: how can I make sure that an uploaded file is really a JPEG?
and similar questions about being sure of the file being uploaded through asp:FileUpload control in ASP.net is really image. But What If users upload virus-infected images? How can I be insured of the image files being uploaded via my ASP.net application does not affect the files in my web app folder and/or images uploaded by other users?
As long as you don't serve it back to anyone as anything other than an image (content-type) and never trying to execute (.exe) the file you'll be fine.
Most anti-virus software run whats known as an "on-access scan". That is, when a file is changed, it automatically scans that file.
So save that file to the file system and let your server's anti-virus software do the work for you.
I'll take what is likely a somewhat controversial position.
There is no way to know with 100% certainty what the intent of a file is, be it good or evil. It is impossible. AV scanners give you a slice of data but they can't give you 100% guarantees either. No one can.
Given this reality, you need to build your app assuming that all files uploaded are bad. Yes, scanning is still fine and will filter out a bunch of stuff. But it will never be 100%. Is it 99.999% or 20%? Who knows. Does it really matter?
I would build any app today assuming that all user supplied content is bad. Very bad. Hostile bad. Because eventually it will be if you make it. And when it is, you'll be ready for them...rather than all the people that have to rearchitect their app because they made bad assumptions early on.
With a bit more data about your exact concerns, I'd be happy to comment on them more specifically...
As a side note, In older version of IIS (6 or prior versions) It could be possible to change FileName to the real malicious file name after save the file with original filename. Which has possibility to be read and execute regularly by the server.
E.G. set the file name like: file.asp;.jpg or file.asp%00.jpg etc...
It also has a possibility to change target directory by manipulation of file name. Which is extremely dangerous
E.G. newfolder.asp::$Index_Allocation or etc...
There is also some new way of attacks. Read more here.
I have a .jpg file which represents the current image from a webcam. User's will be downloading this file at an interval of once a second. Because there could be dozens of users reading it, this could be dozens of times a second (which is normal for any web server).
Problem is, this image is updated by a 3rd party application also once a second which "spiders" my local networks webcam portal image. This is so we can build our webcams into our current administration panel.
The problem I am already finding is ASP.net sometimes gets an error it can not access the file because it is open for write permissions by the bot. Likewise, the bot can not access it because IIS is feeding it to the user.
The bot uses io.streamwriter to save the data to the file, and my script uses Response.WriteFile to send the file to the script. (I need to use an actual ASP.net page with a JPG content-type that feeds the file to make sure only users with a active session can view the JPG).
My question is what is the best practices for this? I know why it's happening but what is the best resolution for this? Would storing as a BLOB in a database maybe be smarter since databases are created for concurrent read/writing already? Is there an easier way of doing this with a file I have not thought of yet?
Thanks in advance,
Anthony Greco
Using a BLOB will work if the readers use SNAPSHOT isolation model (SQL Server 2005 and up). See Download and Upload images from SQL Server via ASP.Net MVC for how to stream an image from a BLOB, and see Understanding Row Versioning-Based Isolation Levels for a lecture on SNAPSHOT.
But using a BLOB may be overkill, you could get away with something much simpler. For instance, if you only have one ASP.Net process, then you could have a global volatile variable for the current file name. The writer writes the JPG into a new file, and then updates the global 'current' file name with an Interlocked.CompareExchange operation (it has to be Compare because a newer writer might actually finish faster, outrun a previous writer, and you want to preserve the latest update). There are still some issues left to solve (find out the file name at startup, clean up old files etc) but they are all fairly ease to solve.
If you have a farm of servers, or multiple ASP.Net processes serving the site, then things could get complicated. I would still do a rotating file name and do a try-and-error approach (try to respond with newest file, fall back to previous older one if conflict is detected).
You could get the bot to write the data to a different filename and then do a delete and rename to the filename being served by ASP.Net. This should reduce the file lock time down to the time for a delete and rename to occur. To clarify:
ASP.Net serving image from "webcam.jpg"
bot writes image data to "temp.jpg"
when last image byte written, bot deletes "webcam.jpg" and renames "temp.jpg" to "webcam.jpg"
ASP.Net should check "webcam.jpg" exists, if not wait 10ms (or suitable small increment) and check again.
I'm coding in ASP.NET and want to store audio files (.mp3, or smaller formats) in a MySQL database; which, I can then retrieve based on certain conditions. Is this possible? Are there any preferred methods to having Audio files on your web pages (besides embedding them in the HTML).
Most solutions that store files in a database do not scale well, but you can certain store audio files, or any other type of file, as a blob (binary large object) in MySQL. You can create an ashx handler that performs the retrieval from the database and writes the content to the ASP.NET output stream as raw binary data. You can then create links that point to the ASHX handler and perform any query logic you want in there based on URL parameters.
If you are using a MySQL database, it seems to do well (at least in my experience) with blobs. It takes a relatively short time to load the MP3 and if you tune your database for audio, you can probably even get better performance (I pretty much use default settings).
One thing to remember is that you define the MIME-type so that users know what they are getting when they click a link to access your MP3.
Again, all of this is my own experience. YMMV.
I prefer to store large files outside of the database, unless there is some overwhelming need to keep everything there.
You could store the location of the file in the database and have the files outside of the webapp directory, so they can't be accessed directly.
Then, in the url for playing the music you can just have a cgi program that will just send that data to the browser, with the correct mime type.
Suppose I have a file, urls.txt, that contains a list of URLs I'm monitoring. My monitoring script edits that file occasionally, say, to indicate whether each URL is reachable. I'd like to also manually edit that file, to add to or change the list of URLs. How can I allow that such that I don't have to think about it when manually editing?
Here are some possible answers. What would you do?
Engage in hackery like having the program check for the lockfiles that vim or emacs create. Since this is just for me, this would actually work.
If the human edits always take precedence, just always have the human clobber the program's changes (eg, ignore the editor's warning that the file has changed on disk). The program can then just redo its changes on its next loop. Still, changing the file while the user edits it is not so nice.
Never let a human touch a file that a program makes ongoing modifications to. Rethink the design and have one file that only the human edits and another file that only the program edits.
Give the human a custom tool to edit the file that does the appropriate file locking. That could be as crude as locking the file and then launching an editor, or a custom interface (perhaps a simple command line interface) for inserting/changing/deleting entries from the file.
Use a database instead of a flat file and then the locking is all taken care of automatically.
(Note that I concocted the URL monitoring example to make this more concrete and because what I actually have in mind is perhaps too weird and distracting -- this question is strictly about how to let humans and programs both modify the same state file.)
I'd use a database since that's basically what you're going to have to build to achieve what you want. Why re-invent the wheel?
If a full-blown DBMS is too much of a load, separate the files into two and synchronize them periodically. Whether the URL is reachable doesn't sound like something the user would be changing, so should not be editable by them.
During the synchronize process (which would have to lock out the monitor and the user although it could be a sub-function of the monitor), remove entries in the monitor file that aren't in the user full. Also, add to the monitor file those that have been added to the user file (and start monitoring them).
But, I'd go the database method with a special front-end for the user, since you can get relatively good light-weight databases nowadays.
Use a sensible version control system!
(Git would work well here).
That said, the nature of the problem implies that a real database would be best - and they will generally have either database-level, table-level, or row-level locking - but then put any scripts you need into version control.
I would go with option 3. In fact, I would have the program read the human-edited input file, and append the results of each query to a log file. In this way, you can also analyse the reachability of sites over time. You can also have the program maintain a file that indicates the current reachability state of each site in the input file, as a snapshot of the current state.
One other option is using two files, one for automated access and one for manual. You'd need a way in the user file to indicate modifications or deletions but you'd have similar problems in some of the other solutions as well.