InputStream closed unexpectedly while using Jersey MultiPart file upload & Server-Sent Events (SSE) - asynchronous

I'm using Jersey Multipart for uploading file to the server via Rest API. In the resource method, I accessed the file content via InputStream. I want to return the uploaded file size to the client with EventOutput using SSE so the client easily get the uploaded file size directly from upload resource method.
I'm using Jersey as JAX-RS implementation in java with Grizzly Http server. Here is my code:
#POST
#Path("upload")
#Produces(SseFeature.SERVER_SENT_EVENTS)
#Consumes("multipart/form-data;charset=utf-8")
public EventOutput upload(#FormDataParam("file") InputStream file,
#FormDataParam("file") FormDataContentDisposition fileDisposition) {
final EventOutput eventOutput = new EventOutput();
try {
new Thread(new Runnable() {
#Override
public void run() {
try {
int read = -1;
byte[] buffer = new byte[1024];
OutboundEvent.Builder eventBuilder
= new OutboundEvent.Builder();
OutboundEvent event = null;
long totalRead = 0, lastReadMB = 0;
while ((read = file.read(buffer)) != -1) {
totalRead += read;
if (lastReadMB != (totalRead / (1024 * 1024))) {
lastReadMB = totalRead / (1024 * 1024);
event = eventBuilder.name("uploaded").data(Long.class, totalRead).build();
eventOutput.write(event);
}
}
event = eventBuilder.name("uploaded").data(Long.class, totalRead).build();
eventOutput.write(event);
} catch (Exception e) {
throw new RuntimeException(
"Error when writing the event.", e);
} finally {
try {
eventOutput.close();
} catch (Exception ioClose) {
throw new RuntimeException(
"Error when closing the event output.", ioClose);
}
}
}
}).start();
return eventOutput;
} catch (Exception e) {
logger.error(e.toString(), e);
}
throw new WebApplicationException(Response.status(Response.Status.INTERNAL_SERVER_ERROR).
entity("something happened").build());
}
​
The problem is when my resource method return EventOutput as a response and request processing thread back to the I/O container, the InputStream closed and the processing thread can't access to the uploaded file. Here is the exception:
Exception in thread "Thread-1" java.lang.RuntimeException: Error when writing the event.
at com.WebService.ContentService$1.run(ContentService.java:192)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.jvnet.mimepull.MIMEParsingException: java.io.IOException: Stream Closed
at org.jvnet.mimepull.WeakDataFile.read(WeakDataFile.java:115)
at org.jvnet.mimepull.DataFile.read(DataFile.java:77)
at org.jvnet.mimepull.FileData.read(FileData.java:69)
at org.jvnet.mimepull.DataHead$ReadMultiStream.fetch(DataHead.java:265)
at org.jvnet.mimepull.DataHead$ReadMultiStream.read(DataHead.java:219)
at java.io.InputStream.read(InputStream.java:101)
at com.WebService.ContentService$1.run(ContentService.java:181)
... 1 more
Caused by: java.io.IOException: Stream Closed
at java.io.RandomAccessFile.seek0(Native Method)
at java.io.RandomAccessFile.seek(RandomAccessFile.java:557)
at org.jvnet.mimepull.WeakDataFile.read(WeakDataFile.java:112)
... 7 more
​
1- What's the problem in the code? Why InputStream is closed in the middle of the file transfer?
2- Is there any alternative way to return the uploaded file size to the client in server side? (REQUIREMENT: the upload resource method must handle upload file asynchronously in different thread)

Related

How to debug exceptions in TCP connection when App is restarted?

I have an application that uses Spring Integration to send messages to a vendor application over TCP and receive and process responses. The vendor sends messages without a length header or an message-ending token and the message contains carriage returns so I have implemented a custom deserializer. The messages are sent as XML strings so I have to process the input stream, looking for a specific closing tag to know when the message is complete. The application works as expected until the vendor application is restarted or a port switch occurs on my application, at which time the CPU usage on my application spikes and the application becomes unresponsive. The application throws a SocketException: o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Send Failed; nested exception is java.net.SocketException: Connection or outbound has closed when the socket closes. I have set the SocketTimeout to be 1 minute.
Here is the connection factory implementation:
#Bean
public AbstractClientConnectionFactory tcpConnectionFactory() {
TcpNetClientConnectionFactory factory = new TcpNetClientConnectionFactory(this.serverIp,
Integer.parseInt(this.port));
return getAbstractClientConnectionFactory(factory, keyStoreName, trustStoreName,
keyStorePassword, trustStorePassword, hostVerify);
}
private AbstractClientConnectionFactory getAbstractClientConnectionFactory(
TcpNetClientConnectionFactory factory, String keyStoreName, String trustStoreName,
String keyStorePassword, String trustStorePassword, boolean hostVerify) {
TcpSSLContextSupport sslContextSupport = new DefaultTcpSSLContextSupport(keyStoreName,
trustStoreName, keyStorePassword, trustStorePassword);
DefaultTcpNetSSLSocketFactorySupport tcpSocketFactorySupport =
new DefaultTcpNetSSLSocketFactorySupport(sslContextSupport);
factory.setTcpSocketFactorySupport(tcpSocketFactorySupport);
factory.setTcpSocketSupport(new DefaultTcpSocketSupport(hostVerify));
factory.setDeserializer(new MessageSerializerDeserializer());
factory.setSerializer(new MessageSerializerDeserializer());
factory.setSoKeepAlive(true);
factory.setSoTimeout(60000);
return factory;
}
Here is the deserialize method:
private String readUntil(InputStream inputStream) throws IOException {
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
String s = "";
byte[] closingTag = CLOSING_MESSAGE_TAG.getBytes(ASCII);
try {
Integer bite;
while (true) {
bite = inputStream.read();
byteArrayOutputStream.write(bite);
byte[] bytes = byteArrayOutputStream.toByteArray();
int start = bytes.length - closingTag.length;
if (start > closingTag.length) {
byte[] subarray = Arrays.copyOfRange(bytes, start, bytes.length);
if (Arrays.equals(subarray, closingTag)) {
s = new String(bytes, ASCII);
break;
}
}
}
} catch (SocketTimeoutException e) {
logger.error("Expected SocketTimeoutException thrown");
} catch (Exception e) {
logger.error("Exception thrown when deserializing message {}", s);
throw e;
}
return s;
}
Any help in identifying the cause of the CPU spike or a suggested fix would be greatly appreciated.
EDIT #1
Adding serialize method.
#Override
public void serialize(String string, OutputStream outputStream) throws IOException {
if (StringUtils.isNotEmpty(string) && StringUtils.startsWith(string, OPENING_MESSAGE_TAG) &&
StringUtils.endsWith(string, CLOSING_MESSAGE_TAG)) {
outputStream.write(string.getBytes(UTF8));
outputStream.flush();
}
}
the inbound-channel-adapter uses the ConnectionFactory
<int-ip:tcp-inbound-channel-adapter id="tcpInboundChannelAdapter"
channel="inboundReceivingChannel"
connection-factory="tcpConnectionFactory"
error-channel="errorChannel"
/>
EDIT #2
Outbound Channel Adapter
<int-ip:tcp-outbound-channel-adapter
id="tcpOutboundChannelAdapter"
channel="sendToTcpChannel"
connection-factory="tcpConnectionFactory"/>
Edit #3
We have added in the throw for the Exception and are still seeing the CPU spike, although it is not as dramatic. Could we still be receiving bytes from socket in the inputStream.read() method? The metrics seem to indicate that the read method is consuming server resources.
#Artem Bilan Thank you for your continued feedback on this. My server metrics seem to indicate that they deserialize method is what is consuming the CPU. I was thinking that the SendFailed error occurs because of the vendor restarting their application.
Thus far, I have been unable to replicate this issue other than in production. The only exception I can find in production logs is the SocketException mentioned above.
Thank you.

HttpClient.SendAsync exeption : The underlying connection was closed: The connection was closed unexpectedly

I've got an asp.net core 2.0 web app (Running using Kestrel) with following controller :
public IActionResult GetUpdateList(string apiCode, int softwareId, [FromBody] List<SoftwareFile> updateFiles)
{
try
{
var stream = SoftwareUpdateFilesHandler.GetUpdateZipFileStream(updateFiles, softwareId);
return File(stream.BaseStream, "application/octet-stream", "UpdateFile");
}
catch (System.Exception ex)
{
return NotFound(ex.ToString());
}
}
and this code on my client :
public async static Task<byte[]> GetUpdateAsync(string apiCode, int softwareId, List<SoftwareFile> updatefiles)
{
try
{
StringContent content = null;
if (updatefiles != null && updatefiles.Count > 0)
{
content = new StringContent(Newtonsoft.Json.JsonConvert.SerializeObject(updatefiles));
content.Headers.ContentType = System.Net.Http.Headers.MediaTypeHeaderValue.Parse("application/json");
}
string address = $"{baseAddress}GetUpdate?softwareId={softwareId}";
HttpRequestMessage request = new HttpRequestMessage(new HttpMethod("POST"), address);
request.Content = content;
var response = await httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
var responseContent = await response.Content.ReadAsByteArrayAsync();
return responseContent;
}
catch(Exception ex)
{
return null;
}
}
But when code reaches to httpClient.SendAsync code hangs and after a few seconds client app crashes and I get this exception : The underlying connection was closed: The connection was closed unexpectedly
All of this works fine when I test my code in local but when I publish code and try to call GetUpdateList I get exception.
Odd thing about this is I can't handle exception in my catch block. Somehow catch block doesn't catch this exception and I can see exception in application crash.

SocketTimeoutException Spring 4

I am using spring MVC 4 with tomcat 8. while uploading file I am receiving following error. I have configured session to expire after an hour.Can anyone guide me in right direction
Could not parse multipart servlet request; nested exception is java.io.IOException: org.apache.tomcat.util.http.fileupload.FileUploadBase$IOFileUploadException: Processing of multipart/form-data request failed. null] with root cause
java.net.SocketTimeoutException
at org.apache.tomcat.util.net.NioBlockingSelector.read(NioBlockingSelector.java:201)
at org.apache.tomcat.util.net.NioSelectorPool.read(NioSelectorPool.java:235)
at org.apache.tomcat.util.net.NioSelectorPool.read(NioSelectorPool.java:216)
at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.fillReadBuffer(NioEndpoint.java:1233)
at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.read(NioEndpoint.java:1182)
at org.apache.coyote.http11.Http11InputBuffer.fill(Http11InputBuffer.java:708)
at org.apache.coyote.http11.Http11InputBuffer.access$300(Http11InputBuffer.java:40)
at org.apache.coyote.http11.Http11InputBuffer$SocketInputBuffer.doRead(Http11InputBuffer.java:1057)
at org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:139) ...
In controller :
#PostMapping(value = "/uploadSurvey")
#ResponseBody
public String uploadSurvey(#Valid FileBucket fileBucket, BindingResult result, HttpSession session) {
if (result.hasErrors())
return "redirect:/surveys/uploadSurvey";
else {
byte[] bytes;
try {
bytes = fileBucket.getFile().getBytes();
String decoded = new String(bytes, "UTF-8");
//Here I do object mapping using jackson
}
catch (Exception e){
e.printStackTrace();
}
}
return "xxx";
}
on client side i am using angularjs to send the file:
fd.append('file', file);
fd.append($scope.csrfParamName, $scope.csrfToken);
$http.post(uploadUrl, fd, {
transformRequest : angular.identity,
headers : {
'Content-Type' : undefined
}
})

Postman throwing 400 Bad request for multipart/form-data image upload with jersey 2.0

REQUEST :
URL: http://localhost:8080/RESTfulExample/rest/file/upload
METHOD : POST
HEADER: Content-Type : multipart/form-data
RESPONSE :
HTTP Status 400 - Bad Request
The same code is working with html forms but in postman it's throwing 400 BAD REQUEST, I looked up on google for solution and found that boundary is missing, How to resolve it ? As I have to recieve files from multiple clients like mobile application and web clients via Jquery and rest client.
#Path("/file")
public class UploadFileService {
#POST
#Path("/upload")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFile(#FormDataParam("file") InputStream uploadedInputStream,
#FormDataParam("file") FormDataContentDisposition fileDetail) {
try {
String uploadedFileLocation = "/home/nash/" + fileDetail.getFileName();
// save it
writeToFile(uploadedInputStream, uploadedFileLocation);
String output = "File uploaded to : " + uploadedFileLocation;
System.out.println("File uploaded..........");
return Response.status(200).entity(output).build();
} catch (Exception e) {
e.printStackTrace();
System.out.println("Exception " + e);
return null;
}
}
// save uploaded file to new location
private void writeToFile(InputStream uploadedInputStream, String uploadedFileLocation) {
try {
OutputStream out = new FileOutputStream(new File(uploadedFileLocation));
int read = 0;
byte[] bytes = new byte[1024];
out = new FileOutputStream(new File(uploadedFileLocation));
while ((read = uploadedInputStream.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
out.flush();
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Please follow these steps:
Add jersey-multipart dependency.
In Your Application Class (or in web.xml) enable MultipartFeature.class.
DO NOT Add Content-Type header in your postman request.
For me the above steps worked. Do let me know if that helped you or not.

How to avoid java.io.IOException: Attempted read on closed stream

I'm trying to find a way to avoid the IOException related to the fact that I read on a closed stream.
I'm calling a webservice method that returns a Stream:
InputStream stream = callRestWebService();
try {
parkingState = objectMapper.readValue(stream, ParkingState.class);
} catch (IOException e) {
throw new ParkingMeasurementProviderException("Could not retrieve data.", e);
}
Then, I have my Web Service method where I close the get connection:
public InputStream callRestWebService() {
int parkingId = 2803;
String endpointURL = REST_ENDPOINT + URI_INFO_PATH + parkingId + "/parkingState";
InputStream inputStream = null;
// Create an instance of HttpClient.
HttpClient httpclient = new HttpClient();
// Create a method instance.
GetMethod getMethod = new GetMethod(endpointURL);
getMethod.addRequestHeader("accept", "application/json");
try {
// Execute the method.
int statusCode = httpclient.executeMethod(getMethod);
inputStream = getMethod.getResponseBodyAsStream();
} catch (IOException e) {
e.printStackTrace();
} finally {
// Release the connection.
getMethod.releaseConnection();
}
return inputStream;
}
Is there a way to avoid having this exception without removing the : getMethod.releaseConnection();
The stack trace:
Disconnected from the target VM, address: '127.0.0.1:62152', transport: 'socket'
at be.ixor.itg.server.service.parking.hermesWS.HermesWSParkingControllerMeasurementProvider.getHermesMechelenData(HermesWSParkingControllerMeasurementProvider.java:126)
at be.ixor.itg.server.service.parking.hermesWS.Main.main(Main.java:14)
Caused by: java.io.IOException: Attempted read on closed stream.
at org.apache.commons.httpclient.AutoCloseInputStream.isReadAllowed(AutoCloseInputStream.java:183)
at org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java:86)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager$RewindableInputStream.read(XMLEntityManager.java:2977)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:702)
at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:232)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:124)
at be.ixor.itg.server.service.parking.hermesWS.HermesWSParkingControllerMeasurementProvider.getHermesMechelenData(HermesWSParkingControllerMeasurementProvider.java:116)
... 1 more
Because you are calling releaseConnection() in your finally block, the input stream will no longer be available.
If you do not expect the content to be large, you could read the data from the input stream into a buffer and return the buffer instead of the input stream. Otherwise, you will need to change your code to allow the called to process the data from the input stream before releasing the connection.
BufferedReader br = new BufferedReader(new
InputStreamReader((response.getEntity().getContent())));
String response = br.readLine();
System.out.println("response" + response );
This code is working for me.

Resources