I have a db with original file names, location to files on disk, meta data like user that owns file... Those files on disk are with scrambled names. When user requests a file, the servlet will check whether he's authorized, then send the file in it's original name.
While researching on the subject i've found several cases that cover that issue, but nothing specific to mine.
Essentially there are 2 solutions:
A custom servlet that handles headers and other stuff the Default Servlet containers don't: http://balusc.omnifaces.org/2009/02/fileservlet-supporting-resume-and.html
Then there is the quick and easy one of just using the Default Servlet and do some path remapping. For ex., in Undertow you configure the Undertow subsystem and add file handlers in the standalone.xml that map http://example.com/content/ to /some/path/on/disk/with/files .
So i am leaning towards solution 1, since solution 2 is a straight path remap and i need to change file names on the fly.
I don't want to reinvent the hot water. And both solutions are non standard. So if i decide to migrate app server to other than Wildfly, it will be problematic. Is there a better way? How would you approach this problem?
While your problem is a fairly common one there isn't necessarily a standards based solution for every possible design challenge.
I don't think the #2 solution will be sufficient - what if two threads try to manipulate the file at the same time? If someone got the link to the file could they share it?
I've implemented something very similar to your #1 solution - the key there is that even if the link to the file got out no one could reuse the link as it requires security. You would just "return" a 401 or 403 for the resource.
Another possibility depends on how you're hosted. Amazon S3 allows you to generate a signed URL that has a limited time to live. In this way your server isn't sending the file directly. It is either sending a redirect or a URL to the front end to use. Keep the lifetime at like 15 seconds (depending on your needs) and then the URL is no longer valid.
I believe that the other cloud providers have a similar capability too.
Related
I have an idea to form NGINX cache in non most common way and I want to ask if this is really possible to achieve.
The common way we all are used is when request hits the backend service and only then it's cached in NGINX.
What I want to achieve is to form that NGINX native cache from separate service. That means I want to manipulate hashed keys that are stored in memory via some NGINX module and also create that directory structure with files that contain cached payloads.
The questions would be:
Is this possible?
How to achieve this, what modules should I include into NGINX, etc.?
NGINX is writing cached data to filesystem using some algorithm described here: http://czerasz.com/2015/03/30/nginx-caching-tutorial/. What is actually stored in the first line of that cached file? Everything from the second line is payload but there are some bytes written into the first line that are non readable and cache does not work in case this line is removed.
Thanks in advance!
I am having some problems at the moment using a LoginServlet running OL 4.9, on Tomcat 7.
I have Tomcat configured to allow crossContext to be true, and that allows me to work with other app contexts on the same server. Specifically a Login Servlet. My only other app is the OpenLaszlo presentation server LPS(lps-4.9.0).
I am using a Tomcat Request Filter that snoops the incoming addresses and looks for a particular cookie of authentication, which then makes its way to the LoginServlet that does a forwarding to the OpenLaszlo page. This was done to KEEP the cookie alive when the Request Filter was awakened at the loading of the OpenLaszlo page.
All of that is working now.
There are no errors or warnings in the lps.log file or the localhost.<date>.log either, however the page loading goes on forever, and never completes.
Could it be something that I am passing along in the forwarded URL? I am using at least 2 parameters to cause lzr to be set to "dhtml" and then lzt to be set to "html."
I can't even get a simple <canvas> page with a simple button to load. Has anyone seen this, and been able to fix the problem?
Since I first wrote my description I wrote another plea for help to some friends and ex coworkers, and this will help update the details of what I have discovered thus far.
Here’s the scenario: I am using Tomcat 7, and have installed the WAR file for OpenLaszlo 4.9.
Alongside of this I created a LoginServlet hierarchy and code and web.xml file just under
“webapps”; the same level that lps-4.9.0 is installed.
The sequence of events is the following:
1. A login page comes up that takes the username and password, and sends that
off to /LoginServlet to process. Note: I have also written and registered a Request Filter
for Tomcat that halts traversal beyone /lps-4.9.0 and checks for proper authentication
as I retrieve the cookies from requests trying to access those levels.
2. In the LoginServlet, I am creating a MACH COOKIE that I’ll send along with the response,
so that the Filter will allow me past the /lps-4.9.0 level. To do this I had to do a FORWARD
operation to preserve the cookie. a REDIRECT would just drop them. Since you can’t
give a relative path higher than the Servlet’s root, I had to turn on Tomcat’s “crossContext”
feature that allows me to do that in the same domain. And I have both contexts registered
in Tomcat’s conf directory in server.xml, I believe. Anyhow it works. I can grab the
/lps-4.9.0 context, get a Request Dispatcher, and then use that dispatcher to FORWARD
the request/response pair to my OpenLaszlo file(the LZX file).
So it seems to get as far as LOADING the OpenLaszlo page, but when I perused the console
messages in Chrome’s Developer Tools debugger, it showed that it was actually trying
to use the context of the original request(i.e. /LoginServlet); and of course that doesn’t
exist. I guess when I passed along the original request/response pair, the request had
the FIRST context used, and then tried to derive the relative path to the file off of that.
QUESTION: Can I just copy the stuff from the original request, but change the context,
and forward THAT? Or architecturally should I try something else?
Thanks,
C
And the answer is..... You CAN'T DO IT... Period.
BTW. The Openlaszlo website server is DOWN, DEAD, KAPUT, NIX, GONE, NO MORE...
This will be the final project that I personally implement with the tool
with no support.
It's very sad to see something that had the right idea about development cycle times,
and keeping the client side GUI construction simple, fast, and easy could be something
that dies because of lack of interest? Say wha? Can't be because FLASH was in jeopardy.
I'm pretty sure that we, as programmers, aren't so paranoid about losing our jobs
that we think we must spend lots of hours CODING an interface to keep it secret.
I'm certainly not paranoid about it. I know there is NET BEANS for swing type
GUIS, and I've heard that GWT has adopted something similar now, and so I'll
keep looking for that perfect invention and deal with what is left over.
Critical Path must have been purchased by someone else too, and so the
site sponsor has no motivation to keep it alive, while it dies a slow death.
I have a couple of ActionMethods that returns content from the database that is not changing very often (eg.: a polygon list of available ZIP-Areas, returned as json; changes twice per year).
I know, there is the [OutputCache(...)] Attribute, but this has some disadvantages (a long time client-side caching is not good; if the server/iis/process gets restartet the server-side cache also stopps)
What i want is, that MVC stores the result in the file system, calculates the hash, and if the hash hasn't changed - it returns a HTTP Status Code 304 --> like it is done with images by default.
Does anybody know a solution for that?
I think it's a bad idea to try to cache data on the file system because:
It is not going to be much faster to read your data from file system than getting it from database, even if you have it already in the json format.
You are going to add a lot of logic to calculate and compare the hash. Also to read data from a file. It means new bugs, more complexity.
If I were you I would keep it as simple as possible. Store you data in the Application container. Yes, you will have to reload it every time the application starts but it should not be a problem at all as application is not supposed to be restarted often. Also consider using some distributed cache like App Fabric if you have a web farm in order not to come up with different data in the Application containers on different servers.
And one more important note. Caching means really fast access and you can't achieve it with file system or database storage this is a memory storage you should consider.
Other than HTTP PUT and POST, what other methods can a web application designer use to allow users to upload content (either files or listbox text) from a page of his web app to a remote server?
On the same topic, I was wondering what technology/APIs does a service like Google Docs or Google Drive use? The reason I ask this is: Our Sys Admin has disabled file uploading (via Squid proxy), yet I was able to create and share a document using Google Docs / Google Drive.
Many thanks in advance,
/HS
EDIT Please see the strikeout above.
This depends on the server in question - as the standard set of HTTP commands can be expanded, and some may not be configured/allowed. One of the common commands is "OPTIONS" that ask "what can I do".
But to answer more helpfully: you generally have two main options:
POST (the one you probably want to user as it's nearly always avaiable
GET. You could use GET (but I'm NOT advocating it - just saying you could you it - you should not use a GET to make changes to the server). There are problems with this approach (including size of files, manually handling the encoding etc) but it's possible if you have to go this route.
PUT it often not enabled on servers for security reasons.
More reading: http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
Edit: if "file uploading" is prevented by proxy, have you tried encoding the POST? i.e. As opposed to sending a multipart POST, try encoding the files yourself into POST string and sending that instead? Or encode the file and split into multiple small posts and piecing them together at the other end?
Google Docs uses a mixture of POST and GET. POST for the updates. Google Drive I don't know.
I am creating a MVC 3 application (although just as applicable to other technologies e.g. ASP.NET Forms) and was just wondering if it is feasible (performance wise) to serve images from code rather than using the direct virtual path (like usual).
The idea is that I improve the common method of serving files to:
Apply security checks
Standardised method of serving files based on route values
Returning modified images (if requested) e.g. different dimentions (ok this would only be used sparingly so don't relate this to the performance question above).
Perform business logic before allowing access to the resource
I know HOW to do it but I don't know IF I should do it.
What are the performance issues (if any)
Does something weird happen e.g. images only load sequentially (maybe that's how HTML does it currently i am not sure - exposing my ignorance here).
Anything else you can think of.
Hope this all makes sense!
Thanks,
Dan.
UPDATE
OK - lets get specific:
What are the performance implications for using this type of method for serving all images in MVC 3 using a memory stream? Note: the image url would be GenericFetchImage/image1 (and just for simplicity - all my images are jpegs).
public FileStreamResult GenericFetchImage(string RouteValueRefToImage)
{
// Create a new memory stream object
MemoryStream ms = new MemoryStream();
// Go get image from file location
ms = GetImageAndPutIntoMemoryStream(RouteValueRefToImage);
// return the output as a file
return new FileStreamResult(ms, "image/jpeg");
}
I know that this method works, because I am using it to dynamically generate an image based on a session value for a captcha image. It's pretty neat - but I would like to use this method for all image retrieval.
I guess I am wondering in the above example if this is ok to do or whether it requires more processing to perform and if so, how much? For example, if the number of visitors were to multiply by 1000 for example, would the server be then processingly burdened in the delivery of images..
THANKS!
A similar question was asked before (Can an ASP.Net MVC controller return an Image?) and it appears that the performance implications are very small to serving images out of actions vs directly. As the accepted answer noted, the difference appears to be on the order of a millisecond (in that test case, about 13%). You could re-run the test locally and see what the difference is on your hardware.
The best answer to your question of if you should be using it is from this answer to (another) similar question (emphasis mine):
DO worry about the following: you will need to re-implement a caching strategy on the server, since IIS manages that for static files requested directly. You will also need to make sure you manage your client-side caching with the correct headers included in the response. Ultimately, just ask yourself if re-inventing a method of serving static files from a server is something that serves your application's needs.
To address the specific cases you provided with the question:
Apply security checks
You can already do this using the IIS 7 integrated pipeline. Relevant bit from documentation:
Allowing services provided by both native and managed modules to apply to all requests, regardless of handler. For example, managed Forms Authentication can be used for all content, including ASP pages, CGIs, and static files.
Standardised method of serving files based on route values
If I'm reading the documentation correctly you can insert a module early enough in the pipeline to re-write incoming URLs to point directly to static resources and let IIS handle the request from there. (For the sake of completeness there also this related question regarding mapping routes to mages: How do I route images using ASP.Net MVC routing?)
Empowering ASP.NET components to provide functionality that was previously unavailable to them due to their placement in the server pipeline. For example, a managed module providing request rewriting functionality can rewrite the request prior to any server processing, including authentication.
There are also some pretty powerful URL rewrite features that come with IIS more or less out of the box.
Returning modified images (if requested) e.g. different dimentions (ok this would only be used sparingly so don't relate this to the performance question above).
It looks like a module that does this is already available for IIS. Not sure if that would fall under serving images from code or not though, I guess it might.
Perform business logic before allowing access to the resource
If you're performing business logic to generate said resources (like a chart) or as you mentioned a captcha image then yeah, you basically have no choice but to do it this way.