I'm using Jetty 9's implementation of HttpServletRequest#getPart(name), and it appears to eagerly processes the entire request (or at least the Part in question) before continuing, even though the resulting Part exposes a getInputStream() method.
Is there a way for getPart to return immediately, and leave request streaming to the resulting Part's InputStream?
For reference, here's the relevant snippet from my Servlet implementation:
override def doPost(req: HttpServletRequest, res: HttpServletResponse) {
println("ABOUT TO GET PART") // this happens immediately
val file = req.getPart("file")
println("GOT PART") // it takes a long time to get here if the upload is large
It's wicked tedious, but this can be done using MultipartStream from commons-fileupload:
try {
MultipartStream multipartStream = new MultipartStream(input, boundary);
boolean nextPart = multipartStream.skipPreamble();
OutputStream output;
while(nextPart) {
String header = multipartStream.readHeaders();
// process headers
// create some output stream
multipartStream.readBodyData(output);
nextPart = multipartStream.readBoundary();
}
} catch(MultipartStream.MalformedStreamException e) {
// the stream failed to follow required syntax
} catch(IOException e) {
// a read or write error occurred
}
This requires the use of the InputStream from HttpServletRequest#getInputStream(), and the boundary delimiter encoded in the HTTP request's content type:
Content-Type: multipart/form-data; boundary=------------------------bd019839518ca918
Related
I'm creating a custom framework (something like portal) for numerous JSF 1.x and 2.x applications. For that purpose I created a servlet filter that "enrich" application HTML with framework menu, breadcrumb, logout, etc. In that filter I read app's HTML, modify it and write it to an output stream. So far everything worked great but now I'm having problem with creating a custom error page.
I tried to read a response status code and based on that code, I'm creating output HTML:
public void doFilter(ServletRequest req, ServletResponse resp, FilterChain chain) throws ServletException, IOException {
HttpServletRequest req = (HttpServletRequest) req;
HttpServletResponse res = (HttpServletResponse) resp;
StringServletResponseWrapper responseWrapper = new StringServletResponseWrapper(res);
// Invoke resource, accumulating output in the wrapper.
chain.doFilter(req, responseWrapper);
String contentType = res.getContentType();
byte[] data;
if (contentType.contains("text/html")) {
String html = null;
int statusCode = res.getStatus();
LOG.debug("status: {}, committed: {}", statusCode, res.isCommitted());
if (statusCode != 200) {
html = "<!DOCTYPE html>\r\n" +
"<html xmlns=\"http://www.w3.org/1999/xhtml\">\r\n" +
"<head>\r\n" +
"<script type=\"text/javascript\" src=\"/path/to/jquery/jquery-1.11.1.min.js\"></script>\r\n" +
"<title>Error</title>\r\n" +
"</head>\r\n" +
"<body>\r\n" +
"<h1>Error</h1>\r\n" +
"</body>\r\n" +
"</html>";
Collection<String> headerNames = res.getHeaderNames();
Map<String, String> headerMap = new HashMap<String, String>();
for (String header : headerNames) {
headerMap.put(header, res.getHeader(header));
}
res.reset();
for (Map.Entry<String,String> entry : headerMap.entrySet()) {
res.setHeader(entry.getKey(), entry.getValue());
}
res.setStatus(statusCode);
response.setContentType("text/html");
} else {
html = responseWrapper.getCaptureAsString();
}
if (ObjectUtils.isNotEmpty(html)) {
// do some modification
String modifiedResponse = doModification(html);
data = modifiedResponse.getBytes("UTF-8");
response.setContentLength(data.length);
response.getOutputStream().write(data); // this line causes error
}
} else {
data = responseWrapper.getCaptureAsBytes();
response.setContentLength(data.length);
response.getOutputStream().write(data);
}
}
This code works without any problem if status code equals 200 (else clause), but when it's not equal to 200 (I triggered 404 error), the following error occures:
com.ibm.ws.webcontainer.webapp.WebApp logServletError SRVE0293E: [Servlet Error]-[Faces Servlet]: java.lang.IllegalStateException: SRVE0209E: Writer already obtained
I don't really understand why does this error appear. The only difference between two cases is HTML content which is valid in both cases. Any help?
Using Websphere Application Server 8.5.5.18.
EDIT: I've tried to call reset() and then set headers and status code again, but that reset() call causes an IllegalStateException - as stated in javadoc, apparently response has already been committed. As far as I understand, flush() method of ServletOutputStream could cause response to be committed, but I'm not calling it anywhere. I've also added some log to see if response really is committed. In both cases (status 200 and 404) response.isCommitted() returns true. Does that mean that response is committed before doFilter is called?
Option 1 - downgrade JAX-RS to 1.1
Once JAX-RS version is changed back to 1.1 the errors in SystemOut.log will not be shown.
Do the following steps:
Change the JAX-RS version to 1.1 using WAS 9 Admin console. See the detailed instructions at
https://www.ibm.com/support/knowledgecenter/SSEQTP_9.0.0/com.ibm.websphere.base.doc/ae/twbs_jaxrs_coexist_adminconsole.html
Option 2 - move chain.doFilter to the end of your doFilter method
chain.doFilter(req, resp);
}
Option 3 - Remove other usages of PrintWriter or OuputStream
Review application to determine if both PrintWriter and OuputStream were obtained. Modify the failing servlet/JSP to only obtain one or the other.
I am using as Web API what uses AuthorisationManager owin middleware to handle token based security.
My problem is that various errors within the response body have various different formats.
Within my api, I usually send errors back with the structure
{"code": "error code", "message": "error message"}
However some of the errors coming from the security may use
{"error": "error code", "error_description": "error message"}
or sometimes just
{"error": "error mesage"}
I would like to unify these to all have the same structure I use elsewhere, ie the
{"code": "error code", "message": "error message"}
I have seen quite a few posts on replacing a response body.
I first tried this method, ie using the DelegatingHandler. This worked in most cases, but it did not catch my authorization failed error messages comding out of my OAuthAuthorizationServerProvider
I next tried using a middleware approach as shown here.
Here is my full interpretation..
public override async Task Invoke(IOwinContext context)
{
try
{
// hold a reference to what will be the outbound/processed response stream object
var stream = context.Response.Body;
// create a stream that will be sent to the response stream before processing
using (var buffer = new MemoryStream())
{
// set the response stream to the buffer to hold the unaltered response
context.Response.Body = buffer;
// allow other middleware to respond
await this.Next.Invoke(context);
// Error codes start at 400. If we have no errors, no more to d0.
if (context.Response.StatusCode < 400) // <---- *** COMMENT1 ***
return;
// we have the unaltered response, go to start
buffer.Seek(0, SeekOrigin.Begin);
// read the stream
var reader = new StreamReader(buffer);
string responseBody = reader.ReadToEnd();
// If no response body, nothing to do
if (string.IsNullOrEmpty(responseBody))
return;
// If we have the correct error fields names, no more to do
JObject responseBodyJson = JObject.Parse(responseBody);
if (responseBodyJson.ContainsKey("code") && responseBodyJson.ContainsKey("message"))
return;
// Now we will look for the known error formats that we want to replace...
byte[] byteArray = null;
// The first one from the security module, errors come back as {error, error_description}.
// The contents are what we set (so are correct), we just want the fields names to be the standard {code, message}
var securityErrorDescription = responseBodyJson.GetValue("error_description");
var securityErrorCode = responseBodyJson.GetValue("error");
if (securityErrorDescription != null && securityErrorCode != null)
byteArray = CreateErrorObject(securityErrorCode.ToString(), securityErrorDescription.ToString());
// The next horrible format, is when a refresh token is just sends back an object with 'error'.
var refreshTokenError = responseBodyJson.GetValue("error");
if (refreshTokenError != null)
{
// We will give this our own error code
var error = m_resourceProvider.GetRefreshTokenAuthorisationError(refreshTokenError.ToString());
byteArray = CreateErrorObject(error.Item2, error.Item3);
}
else
{
byteArray = Encoding.ASCII.GetBytes(responseBody);
}
// Now replace the response (body) with our now contents
// <---- *** COMMENT2 ***
context.Response.ContentType = "application / json";
context.Response.ContentLength = byteArray.Length;
buffer.SetLength(0);
buffer.Write(byteArray, 0, byteArray.Length);
buffer.Seek(0, SeekOrigin.Begin);
buffer.CopyTo(stream);
}
}
catch (Exception ex)
{
m_logger.WriteError($"ResponseFormattingMiddleware {ex}");
context.Response.StatusCode = 500;
throw;
}
}
private byte[] CreateErrorObject(string code, string message)
{
JObject newMessage = new JObject();
newMessage["code"] = code;
newMessage["message"] = message;
return Encoding.ASCII.GetBytes(newMessage.ToString());
}
So this basically seemed to work, and catch ALL responses, which is good.
However, what I was hoping to do, is, when there is no error, (or the error is already in the correct format), just pass the response on without doing anything with it.
I am mainly thinking of some of my GETs, where the data may be large, I was hoping to avoid having to do the extra copying back. In the above code, where I have marked *** COMMENT1 ***, I have an early return to try to avoid this, ie the line...
// Error codes start at 400. If we have no errors, no more to d0.
if (context.Response.StatusCode < 400)
return;
The problem, is when I do this, I get no body at all returned, ie no data for all the GET calls, etc.
Is there a way to avoid this extra copying (ie at the line *** COMMENT2 ***) when we don't want to do any modifications?
Thanks in advance for any suggestions.
Adding an answer since it has a code snippet but this is really just a comment.
Our services use the delegatingHandler approach you mentioned you tried first. Do you have try/catch around the call to base.SendAsync. In this snippet the requestState is just a wrapper around the incoming request with some timers, loggers, etc. In many cases we replace the response as you are trying. I stepped through the exception and used the VS debugger immediate window to modify the error response. It works for me.(TM)
try
{
return base
.SendAsync(request, cancellationToken)
.ContinueWith(
(task, requestState) => ((InstrumentedRequest)requestState).End(task),
instrumentedRequest,
CancellationToken.None,
TaskContinuationOptions.ExecuteSynchronously,
TaskScheduler.Default)
.Unwrap();
}
catch (Exception ex)
{
instrumentedRequest.PrematureFault(ex);
throw;
}
Below code is written with Spring MVC. I simulate the dynamic response generation by reading a file first and send it to client.
For a GET method, the response will contain the Transfer-Encoding: chunked header rather than the Content-Length header.
For a HEAD method, how should I implement the response? Should I manually insert the Transfer-Encoding: chunked header and remove the Content-Length header?
#RestController
public class ChunkedTransferAPI {
#Autowired
ServletContext servletContext;
#RequestMapping(value = "xxx.iso", method = { RequestMethod.GET })
public void doChunkedGET(HttpServletResponse response) {
String filename = "/xxx.iso";
try {
ServletOutputStream output = response.getOutputStream();
InputStream input = servletContext.getResourceAsStream(filename);
BufferedInputStream bufferedInput = new BufferedInputStream(input);
int datum = bufferedInput.read();
while (datum != -1) {
output.write(datum); //data transfer happens here.
datum = bufferedInput.read();
}
output.flush();
output.close();
} catch (IOException e) {
// TODO Auto-generated catch block
System.out.println(e.getMessage());
}
}
#RequestMapping(value = "xxx.iso", method = { RequestMethod.HEAD })
public void doChunkedHEAD(HttpServletResponse response) {
// response.setHeader("Server", "Apache-Coyote/1.1");
// response.setHeader("Transfer-Encoding", "chunked");
}
}
My client's behavior is:
Initiate a HEAD request first to get the anticipated response size. This size is used to allocate some buffer.
Then initiate a GET request to actually get the response content and put it in the buffer.
I kind of have the feeling that I am catering to the client's behavior rather than following some RFC standard. I am worried that even if I can make the client happy with my response, it will fail with other servers' responses.
Anyone could shed some light on this? How should I implement the HEAD response?
Or maybe the client should NEVER rely on the HEAD response to decide the size of a GET response because the RFC says:
The server SHOULD send the same header fields in response to a HEAD
request as it would have sent if the request had been a GET, except
that the payload header fields (Section 3.3) MAY be omitted.
And Content-Length happens to be one of the payload header fields.
My RSS servlet uses try-with-resource for the OutputStream out of the HttpServletResponse and the writer for it. In some cases SomeException is thrown whilst generating the RSS document, in which case I need to return an HTTP status 500 to the client:
try (ServletOutputStream out = response.getOutputStream();
OutputStreamWriter writer = new OutputStreamWriter(out, "utf-8")) {
response.setContentType("text/xml");
// Generate RSS here
} catch (SomeException e) {
response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, e.getMessage());
return;
}
However, by the time response.sendError() is called, the $out$ has already been closed and I get said IllegalStateException saying that the response has already been committed (closing the stream seems to commit the response automatically ).
If I move the initialization of out and writer outside of the try-block and close them in a finally-block (the pre-Java7 way), the error code gets sent correctly.
I was wondering whether there's a way to keep using try-with-resource and still be able to return error codes in case of an exception.
Thanks!
You don't need to close resources which you didn't create yourself. The container created the underlying OutputStream all by itself and is therefore also all by itself responsible for properly closing it. You should visualise it that the container has already put a try-with-resources around the servlet's doXxx() method. See also Should I close the servlet outputstream?
Differently put, the whole try-with-resources on OutputStream inside doXxx() is unnecessary.
Just do:
try {
// Generate RSS here
OutputStreamWriter writer = new OutputStreamWriter(response.getOutputStream(), "UTF-8")) {
response.setContentType("text/xml");
// Write RSS here.
} catch (SomeException e) {
response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, e.getMessage());
}
Unrelated to the concrete problem, if you rethrow any checked exception as ServletException, then the container will also all by itself take care of the proper response code and message.
try {
// ...
} catch (SomeException e) {
throw new ServletException(e);
}
I'm attempting to write a Java Servlet to receive binary data requests and reply to them, using HttpServletRequest.getOutputStream() and HttpServletResponse.getInputStream(). This is for a project which involves having a request sent by a Silverlight client to which this servlet responds to through an HTTP POST connection. For the time being, to test the Servlet I'm implementing a client in Java which I'm more familiar with than Silverlight.
The problem is that in my test project I send the data from a Servlet client as a byte array and expect to receive a byte array with the same length -- only it doesn't, and instead I'm getting a single byte. Therefore I'm posting here the relevant code snippets in the hopes that you might point me where I'm doing wrong and hopefully provide relevant bibliography to help me further.
So here goes.
The client servlet handles POST requests from a very simple HTML page with a form which I use as front-end. I'm not too worried about using JSP etc, instead I'm focused on making the inter-Servlet communication work.
// client HttpServlet invokes this method from doPost(request,response)
private void process(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
String firstName = (String) request.getParameter("firstname");
String lastName = (String) request.getParameter("lastname");
String xmlRequest = "<MyRequest><Person><Name Firstname=\""+firstName+"\" Lastname=\""+lastName+"\" /></Person></MyRequest>";
OutputStream writer = null;
InputStream reader = null;
try {
URL url = new URL("http://localhost:8080/project/Server");
URLConnection conn = url.openConnection();
conn.setDoInput(true);
conn.setDoOutput(true);
writer = conn.getOutputStream();
byte[] baXml = xmlRequest.getBytes("UTF-8");
writer.write(baXml, 0,baXml.length);
writer.flush();
// perhaps I should be waiting here? how?
reader = conn.getInputStream();
int available = reader.available();
byte[] data = new byte[available];
reader.read(data,0,available);
String xmlResponse = new String(data,"UTF-8");
PrintWriter print = response.getWriter();
print.write("<html><body>Response:<br/><pre>");
print.write(xmlResponse);
print.write("</pre></body></html>");
print.close();
} finally {
if(writer!=null)
writer.close();
if(reader!=null)
reader.close();
}
}
The server servlet handles HTTP POST requests. This is done by receiving requests the requests from a client Servlet for testing purposes above, but in the future I intend to use it for clients in other languages (specifically, Silverlight).
// server HttpServlet invokes this method from doPost(request,response)
private void process(HttpServletRequest request, HttpServetResponse response)
throws ServletException, IOException {
ServletInputStream sis = null;
try {
sis = request.getInputStream();
// maybe I should be using a BufferedInputStream
// instead of the InputStream directly?
int available = sis.available();
byte[] input = new byte[available];
int readBytes = sis.read(input,0,available);
if(readBytes!=available) {
throw new ServletException("Oops! readBytes!=availableBytes");
}
// I ONLY GET 1 BYTE OF DATA !!!
// It's the first byte of the client message, a '<'.
String msg = "Read "+readBytes+" bytes of "
+available+" available from request InputStream.";
System.err.println("Server.process(HttpServletRequest,HttpServletResponse): "+msg);
String xmlReply = "<Reply><Message>"+msg+"</Message></Reply>";
byte[] data = xmlReply.getBytes("UTF-8");
ServletOutputStream sos = response.getOutputStream();
sos.write(data, 0,data.length);
sos.flush();
sos.close();
} finally {
if(sis!=null)
sis.close();
}
}
I have been sticking to byte arrays instead of using BufferInputStreams so far because I've not decided yet if I'll be using e.g. Base64-encoded strings to transmit data or if I'll be sending binary data as-is.
Thank you in advance.
To copy input stream to output stream use the standard way:
InputStream is=request.getInputStream();
OutputStream os=response.getOutputStream();
byte[] buf = new byte[1000];
for (int nChunk = is.read(buf); nChunk!=-1; nChunk = is.read(buf))
{
os.write(buf, 0, nChunk);
}
The one thing I can think of is that you are reading only request.getInputStream().available() bytes, then deciding that you have had everything. According to the documentation, available() will return the number of bytes that can be read without blocking, but I don't see any mention of whether this is actually guaranteed to be the entire content of the input stream, so am inclined to assume that no such guarantees are made.
I'm not sure how to best find out when there is no more data (maybe Content-Length in the request can help?) without risking blocking indefinitely at EOF, but I would try looping until having read all the data from the input stream. To test that theory, you could always scan the input for a known pattern that occurs further into the stream, maybe a > matching the initial < that you are getting.