I am trying to implement restful protocol on jetty server. I have runnable server and i can access it from my rest client. My server side project is a maven project. I have a problem about the character encoding.When i check response, before send it from controller, there is no encoding problem. But after i return response to client, i see broken data. Response header is UTF-8. Also i have a listener for this problem and i am setting to request and response to UTF-8. I guess problem happens when i try to write my response data to response.
#GET
#Path("/")
#Produces({"application/xml;charset=UTF-8","application/json;charset=UTF-8"})
public String getPersons(#Context HttpServletRequest request, #Context HttpServletResponse response) {
List<Person> persons = personService.getPersons(testUserId, collectionOption, null);
if (persons == null) {
persons = new ArrayList<Person>();
}
String result = JsonUtil.listToJson(persons);
//result doesnt has any encoding problem at this line
response.setContentType("application/json");
response.setContentLength(result.length());
response.setCharacterEncoding("utf-8");
//i guess problem happen after this line
return result;
}
Is there any jetty configuration or resteasy configuration for it? Or is there any way to solve this problem? Thanks for your helps.
Which resteasy version are you using? There is a known issue (RESTEASY-467) with Strings in 2.0.1 an prior.
These are your options:
1) force the encoding returning byte[]
public byte[] getPersons
and then
return result.getBytes("UTF8");
2) return List (or create a PersonListing if you need it)
public List<Person> getPersons
and let resteasy handle the json transformation.
3) return a StreamingOutput
NOTE: with this option the "Content-Length" header will be unknown.
return new StreamingOutput()
{
public void write(OutputStream outputStream) throws IOException, WebApplicationException
{
PrintStream writer = new PrintStream(outputStream, true, "UTF-8");
writer.println(result);
}
};
4) upgrade to 2.2-beta-1 or newer version.
Related
I have written a HTTP client, where I am reading the data response from a REST web service. My confusion arises after reading multiple blogs on EntityUtils.consume() and EntiryUtils.toString(). I wanted to know the following:
If EntityUtils.toString(..) ONLY is sufficient as it also closes the stream after reading char bytes. Or I should also do EntityUtils.consume(..) as a good practice.
If both toString() and consume() operation can be used. If yes, then what should be there order.
If I EntityUtils.toString() closes the stream; then why the next call in EntityUtils.consume(..) operations which is entity.isStreaming() still returns true?
Could anyone guide me here to use these operations in a standard way. I am using HTTP version 4+.
I have to use these configurations in multithreaded(web-app) environment.
Thanks
I looked at the recommended example from the apache httpclient commons website.
In the example, they used EntityUtils.toString(..) without needing to use EntityUtils.consume(..) before or after.
They mention that calling httpclient.close() ensures all resources are closed.
source: https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientWithResponseHandler.java
CloseableHttpClient httpclient = HttpClients.createDefault();
try {
HttpGet httpget = new HttpGet("http://httpbin.org/");
System.out.println("Executing request " + httpget.getRequestLine());
// Create a custom response handler
ResponseHandler<String> responseHandler = new ResponseHandler<String>() {
#Override
public String handleResponse(
final HttpResponse response) throws ClientProtocolException, IOException {
int status = response.getStatusLine().getStatusCode();
if (status >= 200 && status < 300) {
HttpEntity entity = response.getEntity();
return entity != null ? EntityUtils.toString(entity) : null;
} else {
throw new ClientProtocolException("Unexpected response status: " + status);
}
}
};
String responseBody = httpclient.execute(httpget, responseHandler);
System.out.println("----------------------------------------");
System.out.println(responseBody);
} finally {
httpclient.close();
}
This is what is quoted for the above example:
This example demonstrates how to process HTTP responses using a response handler. This is the recommended way of executing HTTP requests and processing HTTP responses. This approach enables the caller to concentrate on the process of digesting HTTP responses and to delegate the task of system resource deallocation to HttpClient. The use of an HTTP response handler guarantees that the underlying HTTP connection will be released back to the connection manager automatically in all cases.
I am using spring RestTemplate to download a file. The file size is small.
I want to get base64 encoded String. but I see the base64 encoded string is truncated from what it is supposed to be.
Here is my code
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(
new ByteArrayHttpMessageConverter());
StreamResourceReader reader = new StreamResourceReader();
restTemplate.execute(uri, HttpMethod.POST, null,
new StreamResponseExtractor(reader));
return reader.getEncodedString();
StreamResourceReader.java
public class StreamResourceReader {
private String encodeString;
public void read(InputStream content) {
try {
encodeString = Base64.encodeBase64String(IOUtils.toByteArray(content));
} catch (IOException e) {
throw new IllegalStateException(e);
}
}
public ByteArrayOutputStream getOutputStream(){
return outputStream;
}
public String getEncodedString() {
return encodeString;
}
}
StreamResponseExtractor.java
public class StreamResponseExtractor implements ResponseExtractor<InputStream> {
private StreamResourceReader reader;
public StreamResponseExtractor(StreamResourceReader resourceReader) {
this.reader=resourceReader;
}
#Override
public InputStream extractData(ClientHttpResponse response) throws IOException {
reader.read(response.getBody());
return null;
}
}
EDIT
just found out that inputStream is truncated. I dont know why and what the fix is. any help here would be appreciated.
Thanks
To confirm if your input stream is indeed truncated you can try few things. What IOUtils.toByteArray(content) does is buffers internally the content of input stream and returns the buffer. You can compare the length of buffer array with the byte array the file actually represents. You can do latter with below code
String filePath = "/test.txt";
byte[] fileByteArray= Files.readAllBytes(Paths.get(filePath));
Also ClientHttpResponse ( client view of http response) too has the inputstream available which you can check for content.
InputStream getBody() throws IOException;
As a test for this scenario , I created spring boot Rest client using Rest Template (using the code you shared) and a service for file download again using Spring Boot. On comparing the base encoded String from download vs direct file access, both return same content (compared using String equals method).
UPDATE: Another thing worth trying is just use java.net.HttpURLConnection
in a simple program (for help see here) and try to download the content and check whether this works properly because behind all the Spring abstractions, in this case the underlying object used is HttpURLConnection only
SimpleClientHttpResponse extends AbstractClientHttpResponse {
public InputStream getBody() throws IOException {
InputStream errorStream = this.connection.getErrorStream();
this.responseStream = (errorStream != null ? errorStream : this.connection.getInputStream());
return this.responseStream;
}
...........
...........
}
If this also gives you the same issue, then it's time to look at the server side. May be the server is not sending the complete data.
I'm new to Retrofit. I make a POST request to a website. Website returns response as HTML. So I will parse it. However Retrofit try to parse it as JSON. How can do it?
#FormUrlEncoded
#POST("/login.php?action=login")
void postCredentials(#Field("username") String username,
#Field("password") String password);
Should I use a callback?
Retrofit uses a converter to process responses from endpoints and requests as well. By default, Retrofit uses GsonConverter, which encoded JSON responses to Java objects using the gson library. You can override that to supply your own converter when constructing your Retrofit instance.
The interface you need to implement is available here (github.com). Here's a short tutorial as well, although for using Jackson library, many bits are still relevant: futurestud.io/blog
Also note that the converter works both ways, converting requests and responses. Since you want HTML parsing in one direction only, you may want to use GsonConverter in your custom converter, to convert outgoing Java objects to JSON, in the toBody method.
May be not the best solution but this how i managed to get the source of an html page with retrofit:
MainActivity.java
ApiInterface apiService = ApiClient.getClient(context).create(ApiInterface.class);
//Because synchrone in the main thread, i don't respect myself :p
StrictMode.ThreadPolicy policy = new StrictMode.ThreadPolicy.Builder().permitAll().build();
StrictMode.setThreadPolicy(policy);
//Execution of the call
Call<ResponseBody> call = apiService.url();
response = call.execute();
//Decode the response text/html (gzip encoded)
ByteArrayInputStream bais = new ByteArrayInputStream(((ResponseBody)response.body()).bytes());
GZIPInputStream gzis = new GZIPInputStream(bais);
InputStreamReader reader = new InputStreamReader(gzis);
BufferedReader in = new BufferedReader(reader);
String readed;
while ((readed = in.readLine()) != null) {
System.out.println(readed); //Log the result
}
ApiInterface.java
#GET("/")
Call<ResponseBody> url();
ApiClient.java
public static final String BASE_URL = "https://www.google.com";
private static Retrofit retrofit = null;
public static Retrofit getClient(Context context) {
if (retrofit==null) {
OkHttpClient okHttpClient = new OkHttpClient().newBuilder()
.build();
retrofit = new Retrofit.Builder()
.baseUrl(BASE_URL)
.addConverterFactory(ScalarsConverterFactory.create())
.client(okHttpClient)
.build();
}
return retrofit;
}
This is driving me crazy! I'm trying to serve a JPG image. I am sure this method was working fine the other day, so I don't know what's changed. I've tried many different things to get it to work but I can't seem to get past the exception.
Basically I'm trying to serve an image from the database.
I thought maybe the actual bytes are corrupt so I wrote them to a file, and checked the file content. Just in Finder on Mac, the file in the temp directory looks fine in the preview application so I'm pretty sure it's not the content itself causing the problem.
This is the controller method:
#RequestMapping(value="/binaries/**", method = RequestMethod.GET, produces={MediaType.APPLICATION_JSON_VALUE, MediaType.IMAGE_GIF_VALUE,
MediaType.IMAGE_JPEG_VALUE, MediaType.IMAGE_PNG_VALUE, "application/javascript"})
public #ResponseBody
ResponseEntity<byte[]> serveResource(WebRequest webRequest, HttpServletResponse response, String uri) throws IOException {
String path = (String)request.getAttribute( HandlerMapping.PATH_WITHIN_HANDLER_MAPPING_ATTRIBUTE );
BinaryFile bf = binaryService.findByUri(path);
String tmpdir = System.getProperty("java.io.tmpdir");
File dest = new File(tmpdir + File.separator + bf.getFileName());
FileUtils.writeByteArrayToFile(dest, bf.getResource());
logger.debug("file written: " + dest.getAbsolutePath());
// response.addHeader("Cache-Control", "public, max-age=3600");
if (webRequest.checkNotModified(bf.getLastModifiedDate().toDate().getTime()))
{
return null;
};
return ResponseEntity.ok().contentType(MediaType.IMAGE_JPEG).body(bf.getResource());
}
This is the exception:
Request: http://localhost:8080/binaries/products/shortcode_1/test_image2.jpg raised org.springframework.web.HttpMediaTypeNotAcceptableException: Could not find acceptable representation org.springframework.web.HttpMediaTypeNotAcceptableException: Could not find acceptable representation
Anyone have any ideas? It's Spring 4.1.4.RELEASE
Oh never mind, I figured out what changed. I'd overridden the MessageConverters because I was working on some Jackson stuff, so the fix was that I needed to manually add back the ByteArrayHttpMessageConverter.
#Bean
public ByteArrayHttpMessageConverter byteArrayHttpMessageConverter(){
ByteArrayHttpMessageConverter bam = new ByteArrayHttpMessageConverter();
List<org.springframework.http.MediaType> mediaTypes = new LinkedList<org.springframework.http.MediaType>();
mediaTypes.add(org.springframework.http.MediaType.APPLICATION_JSON);
mediaTypes.add(org.springframework.http.MediaType.IMAGE_JPEG);
mediaTypes.add(org.springframework.http.MediaType.IMAGE_PNG);
mediaTypes.add(org.springframework.http.MediaType.IMAGE_GIF);
mediaTypes.add(org.springframework.http.MediaType.TEXT_PLAIN);
bam.setSupportedMediaTypes(mediaTypes);
return bam;
}
#Override
public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
MappingJackson2HttpMessageConverter mapper = new MappingJackson2HttpMessageConverter();
ObjectMapper om = new ObjectMapper();
om.registerModule(new JodaModule());
om.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false);
mapper.setObjectMapper(om);
converters.add(mapper);
converters.add(byteArrayHttpMessageConverter());
super.configureMessageConverters(converters);
}
I am using Web api controller as follows:
[HttpPost]
public HttpResponseMessage PostMethod(string filename)
{
Stream downloadStream = BL.method(fileName);
HttpResponseMessage response = new HttpResponseMessage();
response.content= new StreamContent(downloadStream);
return response;
}
When I try to call the above method using fiddler I am getting an exception saying
'downloadStream.ReadTimeout' threw an exception of type
'System.InvalidOperationException'.
Can the stream be set in response and sent? Is there any modification for the above code?
There seems to be a problem with your stream. Without knowing how stream is generated it is difficult to say. If you replace BL.method(fileName); with just loading the file yourself using FileStream this should work (I just tested it myself).
On the side note, there are a few problems with your approach:
You use POST. Since you are not changing anything GET is better.
You are not setting ContentType header so client can have problem using resource
You are not disposing the stream so this stream could stay in limbo and generally not good.
Try using the PushStreamContent, maybe by not buffering the file in memory, you might avoid your timeout.
[HttpPost]
public HttpResponseMessage PostMethod(string filename)
{
Stream downloadStream = BL.method(fileName);
HttpResponseMessage response = new HttpResponseMessage();
response.Content = new PushStreamContent((responseStream, httpContent, tc) => {
downloadStream.CopyTo(responseStream);
responseStream.Close();
}, "application/octet-stream");
return response;
}