I understand that the server send website components to the client after http request, HTML, CSS, Js and other static files that are necessary to build the website in the client browser.
I want to understand what the server actually do to generate the response since it should handle many requests not just one request??
I assume that the server create an instance from the files and send it to the client via http response?? is this right, wrong,inaccurate or incomplete or are there other processes that happen on the server to make it work??
These "website components" (code for the browser) can be created by the server however it wants. There are two typical patterns. Static and dynamic.
Static resources are created ahead of time. These cannot be customised by the server at the time of the request.
Dynamic resources will be generated when the request is received. For example, a HTML asset may be generated to include a particular user's username as found in a cookie sent with the request. This is typically done from templates like jinja2 for Python.
Nowadays serving these resources statically and using a client side JavaScript application with a separate data API to customise content is the most popular way to build interactive websites (web apps)
Related
I want to automate requests to Apache tomcat's manager GUI for the purposes of developing a pentest related application. I captured the packet to upload a .war file to the server and this was the first part of the response:
POST /manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=0DCEAA88E8C558E6F3352C52B4BBCD4B HTTP/1.1
From this I can see that there's some kind of nonce preventing cross site request forgery. The problem is that I want to automate sending these packets to the server, so I'm going to need to generate a valid nonce whenever I do that. Is there a way I can grab the current nonce so I can use it in my script?
I have an HTML5 app written in static html/js/css (it's actually written in Dart, but compiles down to javascript). I'm serving the application files via CDN, with the REST api hosted on a separate domain. The app uses client-side routing, so as the user goes about using the app, the url might change to be something like http://www.myapp.com/categories. The problem is, if the user refreshes the page, it results in a 404.
Are there any CDN's that would allow me to create a rule that, if the user requests a page that is a valid client-side route, it would just return the (in my case) client.html page?
More detailed explanation/example
The static files for my web app are stored on S3 and served via Amazon's CloudFront CDN. There is a single HTML file that bootstraps the application, client.html. This is the default file served when visiting the domain root, so if you go to www.mysite.com the browser is actually served www.mysite.com/client.html.
The web app uses client-side routing. Once the app loads and the user starts navigating, the URL is updated. These urls don't actually exist on the CDN. For example, if the user wanted to browse widgets, she would click a button, client-side routing would display the "widgets" view, and the browser's url would update to www.mysite.com/widgets/browse. On the CDN, /widgets/browse doesn't actually exist, so if the user hits the refresh button on the browser, they get a 404.
My question is whether or not any CDNs support looking at the request URI and rewriting it. So, I could see a request for /widgets/browse and rewrite it to /client.html. That way, the application would be served instead of returning a 404.
I realize there are other solutions to this problem, namely placing a server in front of the CDN, but it's less ideal.
I do this using CloudFront, but I use my own server running Apache to accomplish this. I realize you're using a server with Amazon, but since you didn't specify that you're restricted to that, I figured I'd answer with how to accomplish what you're looking to do anyway.
It's pretty simple. Any time you query something that isn't already in the cache on CloudFront, or exists in the Cache but is expired, CloudFront goes back to your web server asking it to serve up the content. At this point, you have total control over the request. I use the mod_rewrite in Apache to capture the request, then determine what content I'm going to serve depending on the request. In fact, there isn't a single file (spare one php script) on my server, yet cloudfront believes there are thousands. Pretty sure url rewriting is standard on most web servers, I can only confirm on lighttp and apache from my own experience though.
More Info
All you're doing here is just telling your server to rewrite incoming requests in order to satisfy them. This would not be considered a proxy or anything of the sort.
The flow of content between your app and your server, with cloudfront in between is like this:
appRequest->cloudFront
if cloudFront has file, return data to user without asking your server
for the file.
If cloudFront DOESN'T have the file (or it has expired), go back to
the origin server and ask it for a new copy to cache.
So basically, what is happening in your situation is this:
A)app->ask cloudfront for url cloud front doesn't have
B)cloudfront
then asks your source server for the file
C)file doesn't exist there,
so the server tells cloudFront to go fly a kite
D)cloudFront comes back empty handed and makes your app 404
E)app crashes and
burns, users run away and use something else.
So, all you're doing with mod_rewrite is telling your server how it can re-interpret certain formatted requests and act accordingly. You could point all .jpg requests to point to singleImage.jpg, then have your app ask for:
www.mydomain.com/image3.jpg
www.mydomain.com/naughtystuff.jpg
Neither of those images even have to exist on your server. Apache would just honor the request by sending back singleImage.jpg. But as far as cloudfront or your app is concerned, those are two different files residing at two different unique places on the server.
Hope this clears it up.
http://httpd.apache.org/docs/current/mod/mod_rewrite.html
I think you are using the URL structure in a wrong way. the path which is defined by forward slashes is supposed to bring you to a specific resource, in your example client.html. However, for routing beyond that point (within that resource) you should make use of the # - as is done in many javascript frameworks. This should tell your router what the state of the resource (your html page or app) is. if there are other resources referenced, e.g. images, then you should provide different paths for them which would go through the CDN.
I want to upload files with JSF. I want to be able to resume an upload after a pause (voluntarily or not).
They will be about 500Mb in size.
I'm working with PrimeFaces, which has a neat FileUpload-Tag, but it doesn't let me pause/resume uploads.
I did some research on this. The most common answer is "Use an FTP-Client". Others were Java-Applets or Flash.
It should work on the current Firefox/Chrome and IE8.
It's indeed not possible to resume file uploads via HTML <input type="file"> element. There's namely nothing in the multipart/form-data encoding which would ever support that. Even more, there's no standard form encoding specification which supports that. You'd basically need to invent a custom HTTP request format.
In Java terms, your best bet is to homebrew an applet or webstart application for this which uses Swing JFileChooser to pick files and uses URLConnection to send it via HTTP to the server. In the server side, you'd need a custom Servlet which understands the custom request format and processes the partial upload accordingly.
There's a 3rd party applet which is capable of this all: JumpLoader. Its homepage is at the moment unfortunately down (you could however try Google Cache). To the point, it sends some specific HTTP request parameters along with the multipart/form-data upload request telling the server side if it's a partial upload and if so at which index it should start, so that the servlet can glue the pieces together.
Then, to integrate this all with JSF, your best bet is to let the applet pass the session ID around as URL path fragment so that the servlet shares the same HTTP session as the JSF application. This way the upload servlet can access session scoped JSF managed beans and/or the JSF application can poll for some servlet-specific variables in the HTTP session.
Use-case: Upload a simple image file to a server, which clients could later retrieve
Designate a FTP Server for the job.
HTTP Put: It can directly upload files to a server without the need of a server side
component to handle the bytestream.
HTTP Post: Handle the bytestream by the server side component.
I think to safely use PUT on a public website requires even more effort than using POST (and is less commonly done) due to potential security issues. See http://bitworking.org/news/PUT_SaferOrDangerous.
OTOH, I think there are plenty of resources for safely uploading files with POST and checking them in the server side script, and that this is the more common practice.
PUT is only appropriate when you know the URL you are putting to.
You could also do:
4) POST to obtain a URL to which you then PUT the file.
edit: how are you going to get the HTTP server to decide whether it is OK to accept a particular PUT request?
What I usually do (via PHP) is HTTP POST.
And employ PHP's move_uploaded_file() to get it to whatever destination I want.
I have URLs of the form http://domain/image/⟨uuid⟩/42x42/some_name.png. The Web server (nginx) is configured to look for a file /some/path/image/⟨uuid⟩/thumbnail_42x42.png, and if it does not exist, it sends the URL to the backend (Django via mod_wsgi) which then generates the thumbnail. Then the backend emits a 302 redirect to exactly the same URL that was requested by the client, with the idea that upon this second request the server will notice the thumbnail file and send it directly.
The question is, will this work with all the browsers? So far testing has shown no problems, but can I be sure all the user agents will interpret this as intended?
Update: Let me clarify the intent. Currently this works as follows:
The client requests a thumbnail of an image.
The server sees the file does not exist, so it forwards the request to the backend.
The backend creates the thumbnail and returns 302.
The backend releases all the resources, letting the server share the newly generated file to current and subsequent clients.
Having the backend serve the newly created image is worse for two reasons:
Two ways of serving the same data must be created;
The server is much better at serving static content. What if the client has an extremely slow link? The backend is not particularly fast nor memory-efficient, and keeping it in memory while spoon-feeding the client can be wasteful.
So I keep the backend working for the minimum amount of time.
Update²: I’d really appreciate some RFC references or opinions of someone with experience with lots of browsers. All those affirmative answers are pleasant but they look somewhat groundless.
If it doesn't, the client's broken. Most clients will follow redirect loops until a maximum value. So yes, it should be fine until your backend doesn't generate the thumbnail for any reason.
You could instead change URLs to be http://domain/djangoapp/generate_thumbnail and that'll return the thumbnail and the proper content-type and so on
Yes, it's fine to re-direct to the same URI as you were at previously.