I've got POST method which responds with a file. And right after file download I also want to show the message on the UI, that was generated during the file creation in the code (were there any errors, are all items included in the file etc.).
My question is how to implement such behaviour in most appropriate way?
now I've got 2 ideas:
sending another POST method right after download, which will return
JSON with message contents. Which seems to be quite complicated
since I should be storing on the server objects with messages for
each created file
sending a cookie with the same response, that will contain message
contents
I think both of your solutions should work in theory. I am assuming here that cookie headers sent back with an octet stream response will be honoured and instantly accessable via javascript. I've not tried this out myself.
I have two more ideas which might give you some inspiration:
Simple
In the POST handler before you output the file to the user, you could set a session variable containing the message you want to display. Then after the file is downloaded, make a 2nd request to GET that message from the session. Since it would get overwritten each time, it might get a bit unreliable if the user made lots of POSTs at once.
More complicated
For a totally asynchronous solution, in the past I have used services in this way:
POST /process_this -> "{'status':'ok', 'id':123}"
GET /status?id=123 -> "{'status':'processing', 'progress':'45%'}"
GET /status?id=123 -> "{'status':'processing', 'progress':'85%'}"
GET /status?id=123 -> "{'status':'ready', 'meta':{'data':'message'}}"
GET /download?id=123 -> Binary file stream
It gives you the ability to always respond instantly and do some cool progress bar on the frontend. Also you could have a queue system if the processing is intensive. But on the downside, you need to write the processor as a seperate module and have it talking to a database or something that saves state.
Other
Some other thoughts I had included:
Storing the meta data within the file itself. (What is the response file format?)
Using multipart responses (not sure how compatible browsers are on this yet)
Using a combination of your two ideas if cookies are not big enough to hold all of the message.
Related
I want a client to be able to request (via HTTP) whichever document is at the head of a server's output queue, with the understanding that if retrieval is successful, the document will then automatically be deleted from the queue. There is never more than one client per server, but the client could be multithreaded. There is no random access to the queue; only the head item can be retrieved (and deleted). I have not found this scenario discussed either here or elsewhere on the web. Here are the various approaches I can think of:
(1) The client could send a GET request. But GET is not supposed to have side effects, so this doesn't seem like a good idea.
(2) The client could send two requests, a GET to retrieve the document at the head of the queue and a DELETE (with an empty or ignorable URL) to delete the document at the head of the queue. But this requires two calls, which could cause various problems, especially if more than one thread/process in the client is trying to retrieve files.
(3) The client could send a POST request with an empty body; if there is a document at the head of the queue, the server will return a response whose body contains the document, and will also delete the document from the queue. This is somewhat counterintuitive in that it doesn't match the mental model of posting data and receiving a simple return code, but otherwise I like it. I'm not worried about the response getting lost in transit and the document going missing; I expect the connection to be safe enough to prevent this.
It would be nice if there were another HTTP method to handle this situation, but since there isn't, I think (3) is the best approach. Agree? Disagree?
UPDATE: Added (4) after reading Dan675's post below.
(4) The client could send a DELETE request, to which the server could send a response with the document in the body (and delete the document from the queue, of course). Again, this is slightly counterintuitive (you don't usually say "delete the item on top of the stack for me, please" when you want to retrieve it), but would work.
It should be done in two calls the first to GET it then one to DELETE it.
If the delete succeeds then the clients request is valid otherwise just treat it as if the whole request failed and try to get what's on the top of the queue again. This will cause some additional overhead due to failed requests but I would not recommend doing either of the other options.
I guess another way of doing this would be to first maybe do a PUT to mark the top most item as 'reserved' in some way then do a GET and DELETE. In doing it this way it may be possible to traverse this server side queue and look for the top-most item that is not 'reserved'.
My application has to export the result of a stored procedure to .csv format. Basically, the client performs a query, and he can see the results on a paged grid, if it contains what he wants, then he clicks on a "Export to CSV" button and he downloads the whole thing.
The server will have to run a stored procedure that will return the full result without paging, create the file and return it to the user.
The result file could be very large, so I'm wondering what is the best way to create this file in the server on demand and serve it to the client without blow up the server memory or resources.
The easiest way: Call the stored procedure with LINQ, create a stream and iterate over the result collection and creating a line in the file per collection item.
Problem 1: Does the deferred execution applies as well to LINQ to stored procedures? (I mean, will .NET try to create a collection with all the items in the result set in memory? or will it give me the result item by item if I do an iteration instead of a .ToArray?)
Problem 2: Is that stream kept in RAM memory till I perform a .Dispose/.Close?
The not-so-easy way: Call the stored procedure with a IDataReader and per each line, write directly to the HTTP response stream. It looks like a good approach, as long as I read I write to the response, the memory is not blown up.
Is it really worth it?
I hope I have explained myself correctly.
Thanks in advance.
Writing to a stream is the way to go, as it will rougly consume not more than the current "record" and associated memory. That stream can be a FileStream (if you create a file) or the ASP.NET stream (if you write directly to the web), or any other useful stream.
The advantage of creating a file (using the FileStream) is to be able to cache the data to serve the same request over and over. Depending on your need, this can be a real benefit. You must come up with an intelligent algorithm to determine the file path & name from the input. This will be the cache key. Once you have a file, you can use the TransmitFile api which leverages Windows kernel cache and in general very efficient. You can also play with HTTP client caches (headers like last-modified-since, etc.), so next time the client request the same information, you may return a not modified (HTTP 304 status code) response. The disadvantages of using cache files is you will need to manage these files, disk space, expiration, etc.
Now, Linq or IDataReader should not change much about perf or memory consumption provided you don't use Linq method that materialize the whole data (exhaust the stream) or a big part of it. That means you will need to avoid ToArray(), ToList() methods and other methods like this, and concentrate only on "streamed" methods (enumerations, skips, while, etc.).
I know I'm late to the game on here, but theoretically how many records are we talking here? I saw 5000 being thrown around, and if its around there that shouldn't be a problem for your server.
Answering the easiest way:
It does unless you specify otherwise (you disable lazy loading).
Not sure I get what you're asking here. Are you referring to a streamreader you'd be using for creating the file, or the datacontext you are using to call the SP? I believe the datacontext will clean up for you after you're done (always good practice to close anyway). Streamreader or the like will need a dispose method run to remove from memory.
That being said, when dealing with file exports I've had success in the past building the Table (CSV) programmatically (via iteration), then sending the structured data as an HTTP response with the type specified in the header, the not so easy way as you so eloquently stated :). Heres a question that asks how to do that with CSV:
Response Content type as CSV
"The server will have to run a stored procedure that will return the full result without paging..."
Perhaps not, but I believe you'll need Silverlight...
You can set up a web service or controller that allows you to retrieve data "by the page" (much like just calling a 'paging' service using GridView or other repeater). You can make async calls from silverlight to get each "page" of data until completed, then use the SaveFileDialog to save to the harddisk.
Hope this helps.
Example 1 |
Example 2
What you're talking about isn't really deferred execution, but limiting the results of a query. When you say objectCollection.Take(10), the SQL that is generated when you iterate the enumerable only takes the top 10 results of that query.
That being said, a stored procedure will return whatever results you are passing back, whether its 5 or 5000 rows of data. Performing a .Take() on the results won't limit what the database returns.
Because of this, my recommendation (if possible for your scenario), is to add paging parameters to your stored procedure (page number, page size). This way, you will only be returning the results you plan to consume. Then when you want the full list for your CSV, you can either pass a large page size, or have NULL values mean "Select all".
Below is the statement written at http://hateinterview.com/java-script/methods-get-vs-post-in-html-forms/1854.html
By specification, GET is used basically for retrieving data where as POST is used for data storing, data updating, ordering a product or even e-mailing
Whenever i used get or post method, i used them to get the parameters with getparameter() method of httprequest. I did not get in above statement, how post method is used for datastoring or dataupdation which we cant achieve with get method. Looking for a very brief example.
EDIT: Thanks all for your answers but i am specifically looking for meaning of data storing, data updating in post method apart from file loading stuff.
GET can in theory also store and update data, but it's simply not safe. It's too easy to accidently store or update data by just bookmarking it, following a link or being indexed by a searchbot. POST requests are not bookmarkable/linkable and not indexed by searchbots. Also, the GET query string has a limited length, the safe limit is 255 characters. The POST request body can however be as large as 2GB. Also, uploading files is not possible by GET.
One difference is that the GET data (in the URL, as another answer says) show up at a *nix server as the contents of the environment variable QUERY_STRING, whereas POST data appear on stdin. Regardless of how they are packaged and sent, GET and POST data are identical in format, in my experience.
#Mohit edited his question to add: "Thanks all for your answers but i am specifically looking for meaning of data storing, data updating in post method apart from file loading stuff. "
Read rfc2616, Hypertext Transfer Protocol -- HTTP/1.1, specifically Sections 9.3 GET and 9.5 POST:
"The GET method means retrieve ... information."
"The POST method is used to request that the origin server accept" information.
To be strictly compliant with rfc2616, use the GET method to read data from the server. Use the POST method to write data to the server.
The "meaning of data storing, data updating" is exactly that. How could this possibly be any clearer or more explicit?
POST sends the data in the body while GET puts the data in the URL...
For example to upload a file you use POST... since GET puts the data into URL the data is visible to the user and the length is limited.
see for example http://www.cs.tut.fi/~jkorpela/forms/methods.html
there are things you can not do with GET !first of them is with post you can upload files!
See this: Article or this one
This question already has answers here:
When should I use GET or POST method? What's the difference between them?
(15 answers)
Closed 9 years ago.
I've only recently been getting involved with PHP/AJAX/jQuery and it seems to me that an important part of these technologies is that of POST and GET.
First, what is the difference between POST and GET? Through experimenting, I know that GET appends the returning variables and their values to the URL string
website.example/directory/index.php?name=YourName&bday=YourBday
but POST doesn't.
So, is this the only difference or are there specific rules or conventions for using one or the other?
Second, I've also seen POST and GET outside of PHP: also in AJAX and jQuery. How do POST and GET differ between these 3? Are they the same idea, same functionality, just utilized differently?
GET and POST are two different types of HTTP requests.
According to Wikipedia:
GET requests a representation of the specified resource. Note that GET should not be used for operations that cause side-effects, such as using it for taking actions in web applications. One reason for this is that GET may be used arbitrarily by robots or crawlers, which should not need to consider the side effects that a request should cause.
and
POST submits data to be processed (e.g., from an HTML form) to the identified resource. The data is included in the body of the request. This may result in the creation of a new resource or the updates of existing resources or both.
So essentially GET is used to retrieve remote data, and POST is used to insert/update remote data.
HTTP/1.1 specification (RFC 2616) section 9 Method Definitions contains more information on GET and POST as well as the other HTTP methods, if you are interested.
In addition to explaining the intended uses of each method, the spec also provides at least one practical reason for why GET should only be used to retrieve data:
Authors of services which use the HTTP protocol SHOULD NOT use GET based forms for the submission of sensitive data, because this will cause this data to be encoded in the Request-URI. Many existing servers, proxies, and user agents will log the request URI in some place where it might be visible to third parties. Servers can use POST-based form submission instead
Finally, an important consideration when using GET for AJAX requests is that some browsers - IE in particular - will cache the results of a GET request. So if you, for example, poll using the same GET request you will always get back the same results, even if the data you are querying is being updated server-side. One way to alleviate this problem is to make the URL unique for each request by appending a timestamp.
A POST, unlike a GET, typically has relevant information in the body of the request. (A GET should not have a body, so aside from cookies, the only place to pass info is in the URL.) Besides keeping the URL relatively cleaner, POST also lets you send much more information (as URLs are limited in length, for all practical purposes), and lets you send just about any type of data (file upload forms, for example, can't use GET -- they have to use POST plus a special content type/encoding).
Aside from that, a POST connotes that the request will change something, and shouldn't be redone willy-nilly. That's why you sometimes see your browser asking you if you want to resubmit form data when you hit the "back" button.
GET, on the other hand, should be idempotent -- meaning you could do it a million times and the server will do the same thing (and show basically the same result) each and every time.
Whilst not a description of the differences, below are a couple of things to think about when choosing the correct method.
GET requests can get cached by the browser which can be a problem (or benefit) when using ajax.
GET requests expose parameters to users (POST does as well but they are less visible).
POST can pass much more information to the server and can be of almost any length.
POST and GET are two HTTP request methods. GET is usually intended to retrieve some data, and is expected to be idempotent (repeating the query does not have any side-effects) and can only send limited amounts of parameter data to the server. GET requests are often cached by default by some browsers if you are not careful.
POST is intended for changing the server state. It carries more data, and repeating the query is allowed (and often expected) to have side-effects such as creating two messages instead of one.
If you are working RESTfully, GET should be used for requests where you are only getting data, and POST should be used for requests where you are making something happen.
Some examples:
GET the page showing a particular SO question
POST a comment
Send a POST request by clicking the "Add to cart" button.
With POST you can also do multipart mime encoding which means you can attach files as well. Also if you are using post variables across navigation of pages, the user will get a warning asking if they want to resubmit the post parameter. Typically they look the same in an HTTP request, but you should just stick to POST if you need to "POST" something TO a server and "GET" if you need to GET something FROM a server as that's the way they were intended.
The only "big" difference between POST & GET (when using them with AJAX) is since GET is URL provided, they are limited in ther length (since URL arent infinite in length).
I have REST services which should receive really long queries via GET. Say for example I want to query a service with many geographical coordinates to find out something about all this coordinates.
1) My first thought was to use long URIs and increase the max URI length of the servlet container.
It would look like this:
GET http://some.test/myresource?query={really big JSON object}
But it seems that URIs longer than 2 KB are not reliable due to old proxy servers (is that right?).
2) My workaround is to create a temporary resource via POST first and use the URI of this resource as parameter in the actual GET request. That would look like this:
POST http://some.test/temp
Request Body: {really big JSON object}
201 Created Location: http://some.test/temp/12309871
GET http://some.test/myresource?query=http://some.test/temp/12309871
3) Use body of GET request. I've read the answers to the question whether it is a good idea to use the body of a GET request for the query, and the consensus is: no. Even Roy Fielding says that this is a bad idea.
4) Another approach could be to interpret POST as "create query result resource" and delete this resource after the request. But I consider that to be not RESTful and to be a bad idea.
Is there a better way to handle big queries with GET requests?
Use PUT.
Why? For the following reasons:
Just because the verb PUT 'may update' the resource, doesn't mean it will or must alter underlying state of the resource.
No new resource identifier (url) should be created by the API side of a PUT. Yes, technically a PUT with a client specified identifier is possible, but in this case you're hitting an existing resource.
PUT is like GET in the fact that it should be idempotent, meaning the results of the request will always be the same regardless of how often you call it and it has no side effects.
PUT means you're putting resource data to an existing resource. In terms of a article or post in the document / blog post worlds, it would be like uploading a new revision of some document to an existing resource URL. If you upload the same revision to the same URL, nothing should change in the resource you get back.
In your case, the geo data is some new resource data you're uploading and the result you get back should be the same every time you make the same request.
A more purist method to use the GET verb for the request might be:
Create an endpoint for a query resource type
POST the JSON set of query details to a query resource endpoint and get an identifier for the query resource (say it returns a query id of 123)
Submit to the get request a query identifier http://some.test/myresource?query_id=123
Delete the query resource 123
I see the pure method much more overhead than using PUT with query resource data in the body.
I thought that the whole point in REST was to work on "documents" (or something alike). The URI part of a request is there to identify uniquely the resource to work on. The body part in contrast is there for the "contents" part of the document.
Hence, use the "body" part of the request.
Also note that the semantics of a "GET" request isn't supposed to be used for "PUTTING" or "POSTING" documents (comment in relation to your "query" example above which seems to "create" an object).
In any case, as you have pointed out, the URI part is limited (for good reason I am sure).
If you are concerned with caching, then the use of ETag/Last-Modified fields (in conjunction with "conditional GET" helps for this purpose.
Here is a slight variation on your second option. Create yourself a processor resource called QueryMaker. POST your parameters to it and let it redirect you to a temporary query resource that will return your results.
POST /QueryMaker
Body: Big Json representation of parameters
303: See Other
Location: http://example.org/TemporaryQueries/123213
If you are using a GET request to send large objects, you are not using REST correctly.
GET should be used for retrieving
resources (via some sort of unique
identifier)
POST should be used for
creating resources (with the contents
in the body)
PUT should be used for
updating a resource (with the
contents in the body)
DELETE should be used for deleting a resource
If you follow these guidelines you will never have to have overly long URIs.
Some best practice REST guidelines are here: http://www.xml.com/pub/a/2004/08/11/rest.html
The biggest limitation of URL lengths on the open Web is actually IE, which constraints them to 2083 characters.
Some proxies (e.g., all but the most recent versions of Squid) will limit them to about 4k, although this is moving towards 8k slowly.
Your #2 workaround is a good approach, depending on your use case.
Sending bodies on GETs may be allowed by some implementations, and disallowed by others, so it's a bad idea for interoperability as well as theoretical reasons. Most importantly, how will a cache know what to use as a key?
Can't you just send the big JSON data with the GET request body, instead of creating the temp resource?
Although it's not 100% kosher, i've found it works nicely with firefox and IE and IMO, the querystring is inelegant and usually exposes implementation details that don't belong in the URI. Just make sure to add a cache buster querystring parameter if you need up-to-date data because the server will ignore the data when determining whether it can return a cached response.
See here for a discussion of pros and cons of stuffing data in the GET request body.