Apache Http EntityUtils.consume() vs EntityUtils.toString()? - http

I have written a HTTP client, where I am reading the data response from a REST web service. My confusion arises after reading multiple blogs on EntityUtils.consume() and EntiryUtils.toString(). I wanted to know the following:
If EntityUtils.toString(..) ONLY is sufficient as it also closes the stream after reading char bytes. Or I should also do EntityUtils.consume(..) as a good practice.
If both toString() and consume() operation can be used. If yes, then what should be there order.
If I EntityUtils.toString() closes the stream; then why the next call in EntityUtils.consume(..) operations which is entity.isStreaming() still returns true?
Could anyone guide me here to use these operations in a standard way. I am using HTTP version 4+.
I have to use these configurations in multithreaded(web-app) environment.
Thanks

I looked at the recommended example from the apache httpclient commons website.
In the example, they used EntityUtils.toString(..) without needing to use EntityUtils.consume(..) before or after.
They mention that calling httpclient.close() ensures all resources are closed.
source: https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientWithResponseHandler.java
CloseableHttpClient httpclient = HttpClients.createDefault();
try {
HttpGet httpget = new HttpGet("http://httpbin.org/");
System.out.println("Executing request " + httpget.getRequestLine());
// Create a custom response handler
ResponseHandler<String> responseHandler = new ResponseHandler<String>() {
#Override
public String handleResponse(
final HttpResponse response) throws ClientProtocolException, IOException {
int status = response.getStatusLine().getStatusCode();
if (status >= 200 && status < 300) {
HttpEntity entity = response.getEntity();
return entity != null ? EntityUtils.toString(entity) : null;
} else {
throw new ClientProtocolException("Unexpected response status: " + status);
}
}
};
String responseBody = httpclient.execute(httpget, responseHandler);
System.out.println("----------------------------------------");
System.out.println(responseBody);
} finally {
httpclient.close();
}
This is what is quoted for the above example:
This example demonstrates how to process HTTP responses using a response handler. This is the recommended way of executing HTTP requests and processing HTTP responses. This approach enables the caller to concentrate on the process of digesting HTTP responses and to delegate the task of system resource deallocation to HttpClient. The use of an HTTP response handler guarantees that the underlying HTTP connection will be released back to the connection manager automatically in all cases.

Related

Set headers for grpc-web call

I'm currently facing an issue with grpc-web, and a loadbalancer.
Trying to call our grpc webservices from our gateway API, results in the following error:
Status(StatusCode="Unknown", Detail="Bad gRPC response. HTTP status code: 411")
It appears that the either of the following headers are required, content-length or Transfer-Encoding.
I have a method for setting metadata in my client.
private async Task<Metadata> SetMetadata()
{
//More stuff here
headers.Add("Transfer-Encoding", "chunked");
return headers;
}
Here is how i create my client:
private async Task<Services.Protobuf.ServiceClient> CreateClient()
{
var httpMessageHandler = new HttpClientHandler();
_grpcChannel ??= GrpcChannel.ForAddress(
await _serviceAddressProvider.GetServiceAddress<ServiceClient>() ??
throw new InvalidOperationException(),
new GrpcChannelOptions()
{
HttpHandler = new GrpcWebHandler(httpMessageHandler)
});
return new(_grpcChannel);
}
And here is how i use the two
var serviceClient = await CreateClient();
var request = new Request
{
//Request stuff
};
var getListReply = await serviceClient.GetListReplyAsync(request, await SetMetadata());
Now. The issue is that I cannot set either Transfer-Encoding or Content-Lenght headers. They simply get stripped somewhere.
If fiddler is running they get added (by fiddler i assume), and the request actually works. But if fiddler is not running, the headers are not there, and i get the above error. (I honestly don't understand the part with fiddler, i'm only reporting what i'm seeing).
Does anyone have any idea why this happens? and if it's even possible to add the headers i'm trying to add with grpc-web?
I don't know much about grpc-web but grpc-gateway does strip HTTP headers if they don't have a grpcmetadata prefix when it forwards the HTTP request to the grpc server
You can take a look at this issue thread https://github.com/grpc-ecosystem/grpc-gateway/issues/1244

Triggering a fallback using #HystrixProperty timeout for HTTP status codes and other exceptions

I have a function in my #Service class that is marked with #HystrixCommand.
This method acts as a client which sends a request to another service URL and gets back a response.
What I want to do is to trigger a fallback function when the response status code is anything other than 200. It will also trigger a fallback for any other exceptions (RuntimeExceptions etc.).
I want to do this by making use of the #HystrixProperty or #HystrixCommandProperty.
I want the client to ping the URL and listen for a 200 response status and if it does not get back a 200 status within a certain time-frame I want it to fallback.
If it gets back a 200 status normally within a certain time it should not trigger the fallback.
#HystrixCommand(fallbackMethod="fallbackPerformOperation")
public Future<Object> performOperation(String requestString) throws InterruptedException
return new AsyncResult<Object>() {
#Override
public Object invoke() {
Client client = null;
WebResource webResource = null;
ClientResponse response =null;
String results = null;
try{
client = Client.create();
webResource = client.resource(URL);
client.setConnectTimeout(10000);
client.setReadTimeout(10000);
response = webResource.type("application/xml")
.post(ClientResponse.class, requestString);
} finally {
client.destroy();
webResource = null;
}
return results;
}
};
}
I specifically want to make use of the #HystrixProperty or #HystrixCommandProperty so performing a check inside the method for response status code not being 200 and then throwing an Exception is not acceptable.
Instead of using Annotations will creating my own Command by extending the HystrixCommand Interface work?
Any ideas or resources for where I can start with this are more than welcome.
I don’t understand why you don’t want to check the response http status code and throw an exception if it is not 200? Doing that will give you the behaviour you desire. i.e. it will trigger a fall back for exceptions or non 200 responses.
You can set the timeout in the client, however I would opt for using the hystrix timeout values. That way you can use Archaius to dynamically change the value at runtime if desired.
You can use the Hystrix command annotation or extend the HystrixCommand class. Both options will provide you with your desired behaviour
Here is an example using the annotation.
#HystrixCommand(fallbackMethod = "getRequestFallback")
public String performGetRequest(String uri) {
Client client = Client.create();
WebResource webResource = client.resource(uri);
ClientResponse response = webResource.get(ClientResponse.class);
if (response.getStatus() != 200) {
throw new RuntimeException("Invalid response status");
}
return response.getEntity(String.class);
}
public String getRequestFallback(String uri) {
return "Fallback Value";
}

Response Mapping in servlet

How does response mapping work in a Servlet?
How does each response know to flush out the output on that particular HTML or handlebars?
There's a ember-java github projects with Jersey REST Service
It uses com.sun.jersey.spi.container.servlet.ServletContainer which extends HttpServlet. it return the output using ServletResponse which write the body and Calling flush()
getOutputStream ..
Calling flush() on the ServletOutputStream commits the response. Either this method or getWriter() may be called to write the body
If you check specific service in servlet it returns json response (#Produces)
#Produces(MediaType.APPLICATION_JSON)
public Customers get(#QueryParam("ids[]") List<Long> ids) {
if (ids != null && !ids.isEmpty()) {
return new Customers(manager.findByIds(ids));
}
return new Customers(manager.findAll());
}
Ember is connection to HTTP Server using adapters
The store uses an object called an adapter to know how to communicate over the network. By default, the store will use DS.RESTAdapter, an adapter that communicates with an HTTP server by transmitting JSON via XHR
ember-data.js have the adapter configuration
DS.RESTAdapter = DS.Adapter.extend({ ...
It return Customers which is defined in router.js
App.Router.map(function() {
this.resource("customers");
this.resource("customer", {
path: '/customer/:customer_id'
});
and have customer templates in customer.hbs

Apache Camel - from jms to http

I have a spring-boot project using Apache Camel.
I want to read a message from an activemq queue containing a file and send it to a web server.
I am trying to find the proper way to do this.
I believe I can make something like:
from("activemq:queue").bean(MyBean.class, "process")
And manually build a http request but I can't help thinking there is probably a better way to do it. Like:
from("activemq:queue").bean(MyBean.class, "process")
.setHeader(Exchange.HTTP_METHOD,constant("POST"))
.to("http://localhost:8080/test");
But I don't know how to manipulate the "exchange" to have a valid http Message.
MyBean receives an Exchange object containing a JmsMessage. I see that there is also a HTTPMessage but I don't think I should build that manually. (It requires HTTPRequest and Response objects I am not sure how to get.)
Can someone shed some light on this problem?
Update
I am going for the bean solution.
from("activemq:queue").bean(MyBean.class, "sendMultipart");
public void sendMultipart(Exchange exchange) {
ByteArrayInputStream in = new ByteArrayInputStream((byte[]) exchange.getIn().getBody());
InputStreamBody contentBody = new InputStreamBody(in, ContentType.create("application/octet-stream"), "filename");
HttpEntity entity = MultipartEntityBuilder
.create()
.addPart("file", contentBody)
.build();
HttpPost httpPost = new HttpPost("http://localhost:8080/upload/");
httpPost.setEntity(entity);
CloseableHttpClient httpClient = HttpClients.createDefault();
try {
CloseableHttpResponse httpResponse = httpClient.execute(httpPost);
System.out.println(httpResponse);
} catch (IOException e) {
e.printStackTrace();
}
}
Updated post
I found this http://hilton.org.uk/blog/camel-multipart-form-data. It allows you to leverage the camel http component.
"jms:queue/SomeQ" ==> {
process(toMultipart)
setHeader(Exchange.CONTENT_TYPE, "multipart/form-data")
process((e: Exchange) => e.getIn.setHeader(Exchange.HTTP_URI,"http://localhost:8111/foo"))
to ("http:DUMMY")
}
def toMultipart(exchange: Exchange): Unit = {
val data = exchange.in[java.io.File]
val entity = MultipartEntityBuilder.create()
entity.addBinaryBody("file", data)
entity.addTextBody("name", "sample-data")
// Set multipart entity as the outgoing message’s body…
exchange.in = entity.build
}
Side note: this would really be a nice use-case to try-out reactive streams.
Original post
I am still having some problems understanding your actual problem. Perhaps some code might help:
I am now assuming you are receiving bytes in some character encoding and want to sent it onward to a dynamically established http-endpoint.
Is the following something you are looking for (code is in camel's scala-dsl)
"jms:queue/SomeQ" ==> {
convertBodyTo(classOf[String],"UTF-32" )
process((e: Exchange) => e.in = e.in[String].toUpperCase + "!")
process((e: Exchange) => e.getIn.setHeader(Exchange.HTTP_URI,"http://localhost:8111/foo"))
to ("http:DUMMY")
}
It will be send as an HTTP POST as the body is not null.
I receive it all well on another endpoint i created to ensure the code above is correct:
"jetty:http://localhost:8111/foo" ==> {
log("received on http 8111 endpoint ${body}")
}

spring-mvc with resteasy character encoding problem on jetty server

I am trying to implement restful protocol on jetty server. I have runnable server and i can access it from my rest client. My server side project is a maven project. I have a problem about the character encoding.When i check response, before send it from controller, there is no encoding problem. But after i return response to client, i see broken data. Response header is UTF-8. Also i have a listener for this problem and i am setting to request and response to UTF-8. I guess problem happens when i try to write my response data to response.
#GET
#Path("/")
#Produces({"application/xml;charset=UTF-8","application/json;charset=UTF-8"})
public String getPersons(#Context HttpServletRequest request, #Context HttpServletResponse response) {
List<Person> persons = personService.getPersons(testUserId, collectionOption, null);
if (persons == null) {
persons = new ArrayList<Person>();
}
String result = JsonUtil.listToJson(persons);
//result doesnt has any encoding problem at this line
response.setContentType("application/json");
response.setContentLength(result.length());
response.setCharacterEncoding("utf-8");
//i guess problem happen after this line
return result;
}
Is there any jetty configuration or resteasy configuration for it? Or is there any way to solve this problem? Thanks for your helps.
Which resteasy version are you using? There is a known issue (RESTEASY-467) with Strings in 2.0.1 an prior.
These are your options:
1) force the encoding returning byte[]
public byte[] getPersons
and then
return result.getBytes("UTF8");
2) return List (or create a PersonListing if you need it)
public List<Person> getPersons
and let resteasy handle the json transformation.
3) return a StreamingOutput
NOTE: with this option the "Content-Length" header will be unknown.
return new StreamingOutput()
{
public void write(OutputStream outputStream) throws IOException, WebApplicationException
{
PrintStream writer = new PrintStream(outputStream, true, "UTF-8");
writer.println(result);
}
};
4) upgrade to 2.2-beta-1 or newer version.

Resources