I am trying to compress my response when the request header contains gzip in Accept-Encoding. However, adding following to app.properties only works when controller method is returning an object.
server.compression.enabled=true
server.compression.min-response-size=1
server.compression.mime-types=application/json,application/xml,text/html,text/xml,text/plain,application/javascript,text/css
I am writing directly to response stream. So, above compression properties don't work.
My method looks like this:
#GetMapping(path = "/something", produces = MediaType.TEXT_PLAIN_VALUE)
public void getSomething(#RequestParam(name = "paramA") String paramA,
HttpServletRequest request,
HttpServletResponse response) throws IOException {
PrintWriter writer = null;
if (request.getHeader("Accept-Encoding").contains("gzip")) {
writer = new PrintWriter(new OutputStreamWriter(new GZIPOutputStream(response.getOutputStream())));
response.addHeader(HttpHeaders.CONTENT_ENCODING, "gzip");
} else {
writer = response.getWriter();
}
final PrintWriter fwriter = writer;
someObjectInstance.getSomething(paramA).forEach(x -> {
fwriter.write(x.toString());
});
fwriter.flush();
}
When I curl the above method, I get an empty file.
I did try to use the GzipFilter by referring to the following link.
http://www.javablog.fr/javaweb-gzip-compression-protocol-http-filter-gzipresponsewrapper-gzipresponsewrapper.html, which works by the way.
However, the filter requires alot of boilerplate code. Is there a way I can make changes in controller method and solve the problem as in the link mentioned above.
Just realised I was not closing the writer.
After adding fwriter.close(); in the end, the problem was solved.
Related
I'm creating a custom framework (something like portal) for numerous JSF 1.x and 2.x applications. For that purpose I created a servlet filter that "enrich" application HTML with framework menu, breadcrumb, logout, etc. In that filter I read app's HTML, modify it and write it to an output stream. So far everything worked great but now I'm having problem with creating a custom error page.
I tried to read a response status code and based on that code, I'm creating output HTML:
public void doFilter(ServletRequest req, ServletResponse resp, FilterChain chain) throws ServletException, IOException {
HttpServletRequest req = (HttpServletRequest) req;
HttpServletResponse res = (HttpServletResponse) resp;
StringServletResponseWrapper responseWrapper = new StringServletResponseWrapper(res);
// Invoke resource, accumulating output in the wrapper.
chain.doFilter(req, responseWrapper);
String contentType = res.getContentType();
byte[] data;
if (contentType.contains("text/html")) {
String html = null;
int statusCode = res.getStatus();
LOG.debug("status: {}, committed: {}", statusCode, res.isCommitted());
if (statusCode != 200) {
html = "<!DOCTYPE html>\r\n" +
"<html xmlns=\"http://www.w3.org/1999/xhtml\">\r\n" +
"<head>\r\n" +
"<script type=\"text/javascript\" src=\"/path/to/jquery/jquery-1.11.1.min.js\"></script>\r\n" +
"<title>Error</title>\r\n" +
"</head>\r\n" +
"<body>\r\n" +
"<h1>Error</h1>\r\n" +
"</body>\r\n" +
"</html>";
Collection<String> headerNames = res.getHeaderNames();
Map<String, String> headerMap = new HashMap<String, String>();
for (String header : headerNames) {
headerMap.put(header, res.getHeader(header));
}
res.reset();
for (Map.Entry<String,String> entry : headerMap.entrySet()) {
res.setHeader(entry.getKey(), entry.getValue());
}
res.setStatus(statusCode);
response.setContentType("text/html");
} else {
html = responseWrapper.getCaptureAsString();
}
if (ObjectUtils.isNotEmpty(html)) {
// do some modification
String modifiedResponse = doModification(html);
data = modifiedResponse.getBytes("UTF-8");
response.setContentLength(data.length);
response.getOutputStream().write(data); // this line causes error
}
} else {
data = responseWrapper.getCaptureAsBytes();
response.setContentLength(data.length);
response.getOutputStream().write(data);
}
}
This code works without any problem if status code equals 200 (else clause), but when it's not equal to 200 (I triggered 404 error), the following error occures:
com.ibm.ws.webcontainer.webapp.WebApp logServletError SRVE0293E: [Servlet Error]-[Faces Servlet]: java.lang.IllegalStateException: SRVE0209E: Writer already obtained
I don't really understand why does this error appear. The only difference between two cases is HTML content which is valid in both cases. Any help?
Using Websphere Application Server 8.5.5.18.
EDIT: I've tried to call reset() and then set headers and status code again, but that reset() call causes an IllegalStateException - as stated in javadoc, apparently response has already been committed. As far as I understand, flush() method of ServletOutputStream could cause response to be committed, but I'm not calling it anywhere. I've also added some log to see if response really is committed. In both cases (status 200 and 404) response.isCommitted() returns true. Does that mean that response is committed before doFilter is called?
Option 1 - downgrade JAX-RS to 1.1
Once JAX-RS version is changed back to 1.1 the errors in SystemOut.log will not be shown.
Do the following steps:
Change the JAX-RS version to 1.1 using WAS 9 Admin console. See the detailed instructions at
https://www.ibm.com/support/knowledgecenter/SSEQTP_9.0.0/com.ibm.websphere.base.doc/ae/twbs_jaxrs_coexist_adminconsole.html
Option 2 - move chain.doFilter to the end of your doFilter method
chain.doFilter(req, resp);
}
Option 3 - Remove other usages of PrintWriter or OuputStream
Review application to determine if both PrintWriter and OuputStream were obtained. Modify the failing servlet/JSP to only obtain one or the other.
I am using spring RestTemplate to download a file. The file size is small.
I want to get base64 encoded String. but I see the base64 encoded string is truncated from what it is supposed to be.
Here is my code
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(
new ByteArrayHttpMessageConverter());
StreamResourceReader reader = new StreamResourceReader();
restTemplate.execute(uri, HttpMethod.POST, null,
new StreamResponseExtractor(reader));
return reader.getEncodedString();
StreamResourceReader.java
public class StreamResourceReader {
private String encodeString;
public void read(InputStream content) {
try {
encodeString = Base64.encodeBase64String(IOUtils.toByteArray(content));
} catch (IOException e) {
throw new IllegalStateException(e);
}
}
public ByteArrayOutputStream getOutputStream(){
return outputStream;
}
public String getEncodedString() {
return encodeString;
}
}
StreamResponseExtractor.java
public class StreamResponseExtractor implements ResponseExtractor<InputStream> {
private StreamResourceReader reader;
public StreamResponseExtractor(StreamResourceReader resourceReader) {
this.reader=resourceReader;
}
#Override
public InputStream extractData(ClientHttpResponse response) throws IOException {
reader.read(response.getBody());
return null;
}
}
EDIT
just found out that inputStream is truncated. I dont know why and what the fix is. any help here would be appreciated.
Thanks
To confirm if your input stream is indeed truncated you can try few things. What IOUtils.toByteArray(content) does is buffers internally the content of input stream and returns the buffer. You can compare the length of buffer array with the byte array the file actually represents. You can do latter with below code
String filePath = "/test.txt";
byte[] fileByteArray= Files.readAllBytes(Paths.get(filePath));
Also ClientHttpResponse ( client view of http response) too has the inputstream available which you can check for content.
InputStream getBody() throws IOException;
As a test for this scenario , I created spring boot Rest client using Rest Template (using the code you shared) and a service for file download again using Spring Boot. On comparing the base encoded String from download vs direct file access, both return same content (compared using String equals method).
UPDATE: Another thing worth trying is just use java.net.HttpURLConnection
in a simple program (for help see here) and try to download the content and check whether this works properly because behind all the Spring abstractions, in this case the underlying object used is HttpURLConnection only
SimpleClientHttpResponse extends AbstractClientHttpResponse {
public InputStream getBody() throws IOException {
InputStream errorStream = this.connection.getErrorStream();
this.responseStream = (errorStream != null ? errorStream : this.connection.getInputStream());
return this.responseStream;
}
...........
...........
}
If this also gives you the same issue, then it's time to look at the server side. May be the server is not sending the complete data.
This is driving me crazy! I'm trying to serve a JPG image. I am sure this method was working fine the other day, so I don't know what's changed. I've tried many different things to get it to work but I can't seem to get past the exception.
Basically I'm trying to serve an image from the database.
I thought maybe the actual bytes are corrupt so I wrote them to a file, and checked the file content. Just in Finder on Mac, the file in the temp directory looks fine in the preview application so I'm pretty sure it's not the content itself causing the problem.
This is the controller method:
#RequestMapping(value="/binaries/**", method = RequestMethod.GET, produces={MediaType.APPLICATION_JSON_VALUE, MediaType.IMAGE_GIF_VALUE,
MediaType.IMAGE_JPEG_VALUE, MediaType.IMAGE_PNG_VALUE, "application/javascript"})
public #ResponseBody
ResponseEntity<byte[]> serveResource(WebRequest webRequest, HttpServletResponse response, String uri) throws IOException {
String path = (String)request.getAttribute( HandlerMapping.PATH_WITHIN_HANDLER_MAPPING_ATTRIBUTE );
BinaryFile bf = binaryService.findByUri(path);
String tmpdir = System.getProperty("java.io.tmpdir");
File dest = new File(tmpdir + File.separator + bf.getFileName());
FileUtils.writeByteArrayToFile(dest, bf.getResource());
logger.debug("file written: " + dest.getAbsolutePath());
// response.addHeader("Cache-Control", "public, max-age=3600");
if (webRequest.checkNotModified(bf.getLastModifiedDate().toDate().getTime()))
{
return null;
};
return ResponseEntity.ok().contentType(MediaType.IMAGE_JPEG).body(bf.getResource());
}
This is the exception:
Request: http://localhost:8080/binaries/products/shortcode_1/test_image2.jpg raised org.springframework.web.HttpMediaTypeNotAcceptableException: Could not find acceptable representation org.springframework.web.HttpMediaTypeNotAcceptableException: Could not find acceptable representation
Anyone have any ideas? It's Spring 4.1.4.RELEASE
Oh never mind, I figured out what changed. I'd overridden the MessageConverters because I was working on some Jackson stuff, so the fix was that I needed to manually add back the ByteArrayHttpMessageConverter.
#Bean
public ByteArrayHttpMessageConverter byteArrayHttpMessageConverter(){
ByteArrayHttpMessageConverter bam = new ByteArrayHttpMessageConverter();
List<org.springframework.http.MediaType> mediaTypes = new LinkedList<org.springframework.http.MediaType>();
mediaTypes.add(org.springframework.http.MediaType.APPLICATION_JSON);
mediaTypes.add(org.springframework.http.MediaType.IMAGE_JPEG);
mediaTypes.add(org.springframework.http.MediaType.IMAGE_PNG);
mediaTypes.add(org.springframework.http.MediaType.IMAGE_GIF);
mediaTypes.add(org.springframework.http.MediaType.TEXT_PLAIN);
bam.setSupportedMediaTypes(mediaTypes);
return bam;
}
#Override
public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
MappingJackson2HttpMessageConverter mapper = new MappingJackson2HttpMessageConverter();
ObjectMapper om = new ObjectMapper();
om.registerModule(new JodaModule());
om.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false);
mapper.setObjectMapper(om);
converters.add(mapper);
converters.add(byteArrayHttpMessageConverter());
super.configureMessageConverters(converters);
}
I am trying to test this method.
#Override
public JSON connectResource() throws IOException {
//get the location and credentials for the certificates
System.setProperty("javax.net.ssl.trustStore", "C:/Program Files/Java/jdk1.7.0_40/jre/lib/security/cacerts");
System.setProperty("javax.net.ssl.trustStorePassword", "changeit");
HttpRequest httpRequest = new HttpGet(url);
System.out.println("hello");
httpRequest.addHeader("Accept", "application/json");
HttpResponse response = httpClient.execute((HttpUriRequest) httpRequest);
System.out.println("hello1");
HttpEntity httpEntity = response.getEntity();
String data = this.getData(httpEntity);
return JSONSerializer.toJSON(data.toString());
}
My set up method is:
#Before
public void setUp() throws Exception{
mockHttpClient = mock(DefaultHttpClient.class);
mockHttpRequest = mock(HttpUriRequest.class);
mockHttpResponse = mock(BasicHttpResponse.class);
mockHttpEntity = mock(HttpEntity.class);
mockInputStream = mock(InputStream.class);
mockInputStreamReader = mock(InputStreamReader.class);
mockBufferedReader = mock(BufferedReader.class);
mockHttpGet = mock(HttpGet.class);
mockHttpRequestBase = mock(HttpRequestBase.class);
//when(mockHttpClient.execute(Mockito.isA(HttpUriRequest.class))).thenReturn(mockHttpResponse);
//when(mockHttpClient.execute(mockHttpRequest)).thenReturn(mockHttpResponse);
//when(mockHttpClient.execute(mockHttpRequestBase)).thenReturn(mockHttpResponse);
//when(mockHttpClient.execute(mockHttpGet)).thenReturn(mockHttpResponse);
when(mockHttpResponse.getEntity()).thenReturn(mockHttpEntity);
when(mockHttpEntity.getContent()).thenReturn(mockInputStream);
PowerMockito.whenNew(InputStreamReader.class)
.withArguments(mockInputStream).thenReturn(mockInputStreamReader);
PowerMockito.whenNew(BufferedReader.class)
.withArguments(mockInputStreamReader).thenReturn(mockBufferedReader);
PowerMockito.when(mockBufferedReader.readLine())
.thenReturn(JSON_STRING)
.thenReturn(null);
PowerMockito.whenNew(HttpGet.class).withArguments(VALID_URL)
.thenReturn(mockHttpGet);
}
And my test case is :
#Test
public void testConnectResource() throws IOException {
when(mockHttpClient.execute(mockHttpGet)).thenReturn(mockHttpResponse);
HttpConnectGithub connHandle = new HttpConnectGithub(VALID_URL);
JSON jsonObject = connHandle.connectResource();
System.out.println(jsonObject);
//assertThat(jsonObject, instanceOf(JSON.class));
}
However, the execution stops at
HttpResponse response = httpClient.execute((HttpUriRequest) httpRequest);
you can see all that I tried in the comments of my set up method.
Does anyone find an issue with anything? I debugged through my test case and all mock objects are properly initialized.
I have tried exchanging HttpUriRequest and HttpRequest, HttpResponse and BasicHttpResponse etc but without much luck.
Please guide on how to tackle this issue.
Part of the problem you're running into is matching the arguments:
#Test
public void testConnectResource() throws IOException {
when(mockHttpClient.execute(mockHttpGet)).thenReturn(mockHttpResponse);
HttpConnectGithub connHandle = new HttpConnectGithub(VALID_URL);
JSON jsonObject = connHandle.connectResource();
System.out.println(jsonObject);
//assertThat(jsonObject, instanceOf(JSON.class));
}
With the line you've specified above
when(mockHttpClient.execute(mockHttpGet)).thenReturn(mockHttpResponse);
The mocking will only trigger when the instance of mockHttpGet you've defined is passed.
Your method under test on the other hand is creating a new HttpGet instance which is not going to be the same as the mockHttpGet instance. You will need to alter the 'when' statement so that you have something like
when(mockHttpClient.execute(Matchers.any(HttpGet.class))).thenReturn(mockHttpResponse);
I'm doing this exclusively from memory so the Matchers.any() may be incorrect, but you should be able to make headway based on what I've given you above.
The problem is with mockHttpClient. It is not able to mock it automatically for some reason. The fix is to pass httpclient as a parameter through some method (constructor in my case)
I am trying to implement restful protocol on jetty server. I have runnable server and i can access it from my rest client. My server side project is a maven project. I have a problem about the character encoding.When i check response, before send it from controller, there is no encoding problem. But after i return response to client, i see broken data. Response header is UTF-8. Also i have a listener for this problem and i am setting to request and response to UTF-8. I guess problem happens when i try to write my response data to response.
#GET
#Path("/")
#Produces({"application/xml;charset=UTF-8","application/json;charset=UTF-8"})
public String getPersons(#Context HttpServletRequest request, #Context HttpServletResponse response) {
List<Person> persons = personService.getPersons(testUserId, collectionOption, null);
if (persons == null) {
persons = new ArrayList<Person>();
}
String result = JsonUtil.listToJson(persons);
//result doesnt has any encoding problem at this line
response.setContentType("application/json");
response.setContentLength(result.length());
response.setCharacterEncoding("utf-8");
//i guess problem happen after this line
return result;
}
Is there any jetty configuration or resteasy configuration for it? Or is there any way to solve this problem? Thanks for your helps.
Which resteasy version are you using? There is a known issue (RESTEASY-467) with Strings in 2.0.1 an prior.
These are your options:
1) force the encoding returning byte[]
public byte[] getPersons
and then
return result.getBytes("UTF8");
2) return List (or create a PersonListing if you need it)
public List<Person> getPersons
and let resteasy handle the json transformation.
3) return a StreamingOutput
NOTE: with this option the "Content-Length" header will be unknown.
return new StreamingOutput()
{
public void write(OutputStream outputStream) throws IOException, WebApplicationException
{
PrintStream writer = new PrintStream(outputStream, true, "UTF-8");
writer.println(result);
}
};
4) upgrade to 2.2-beta-1 or newer version.