Apache commons fileupload timeout only with Firefox - http

I use the Apache commons fileupload 1.4 library in my java project.
I have a html part with a classic form with a file input and some hidden fields.
I have a problem with uploading files of around >500ko only with Firefox >= 52
It works well with files of 10mo in Chrome or Internet Explorer.
But with Firefox, I have a timeout after waiting several minutes after submitting the form.
After some debugging, I see that the code responsible of the timeout is :
List<FileItem> items = (new ServletFileUpload(new DiskFileItemFactory())).parseRequest(request);
The part with cause wait is "parseRequest".
I try to debug the content of request with debugger in IntelliJ, but there is no way to copy entire content value of this request object in raw format.
It's working in these cases :
- Firefox : version <= 52 or file size < 500ko (around, it's not really precise)
- Internet Explorer
- Chrome
There is no file size limit, it seems that depends on the request size, because the parsing request part is taking too much time...
I get the HTTP request with a Firefox extension in two cases.
One generating uploading a file of 3mo which doesn't works (the request file is huge, 3x the size of the uploaded file) :
https://code.empreintesduweb.com/13561.html
One generated uploading a file of 200ko which works (the request file is small) :
https://code.empreintesduweb.com/13560.html
In fact, the main difference is that in Chrome or IE, I don't have the raw content of the uploaded file in the request headers :
The part with :
obj
stream
....
endstream
endobj
Only appear with Firefox...

You can try setting the maximum file size, maybe the file size exceeds the maximum threshold .According to the documentation :
Uploaded items should be retained in memory as long as they are reasonably small.
Larger items should be written to a temporary file on disk.
Very large upload requests should not be permitted.
The built-in defaults for the maximum size of an item to be retained in memory, the maximum permitted size of an upload
request, and the location of temporary files are acceptable.
Try the following :
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
try {
// Set factory constraints
DiskFileItemFactory factory = new DiskFileItemFactory();
factory.setSizeThreshold(yourMaxMemorySize);
ServletContext servletContext = this.getServletConfig().getServletContext();
File repository = (File) servletContext.getAttribute("javax.servlet.context.tempdir");
factory.setRepository(repository);
List<FileItem> items = new ServletFileUpload(factory).parseRequest(request);
for (FileItem item : items) {
if (item.isFormField()) {
// Process regular form field (input type="text|radio|checkbox|etc", select, etc).
String fieldName = item.getFieldName();
String fieldValue = item.getString();
// ... (do your job here)
} else {
// Process form file field (input type="file").
String fieldName = item.getFieldName();
String fileName = FilenameUtils.getName(item.getName());
InputStream fileContent = item.getInputStream();
// ... (do your job here)
}
}
} catch (FileUploadException e) {
throw new ServletException("Cannot parse multipart request.", e);
}
// ...
}
Here, we are providing a temp location for the file since the file is large.

A few things that are worth to try here:
Explicit the encoding: https://stackoverflow.com/a/10488411/4279120
Decompose your call and add iteration and try catch, ex. : https://www.programcreek.com/java-api-examples/?api=org.apache.commons.fileupload.FileItemIterator
Take a look at the MultipartConfig, it seems to provide such attributes as maxFileSize and maxRequestSize (see: https://www.codejava.net/java-ee/servlet/java-file-upload-example-with-servlet-30-api#maxFileSize%28%29)
Manually define the header of your Request if you can. It seems that adding "X-File-Name" and "X-File-Size", can also help, but this is a little old: AJAX File Upload with XMLHttpRequest
We may also help you better if you provide some more informations, like the versions of apache / java / servlet, and a few more code (especially the definition of request)
Some ressources that could be helpful:
XMLHttpRequest
Sending_files_using_a_FormData_object
How to set a header for a HTTP GET request, and trigger file download?

try this to set session timeout using setMaxInactiveInterval method
request.getSession().setMaxInactiveInterval(1200);
parameter Specifies the time, in seconds, between client requests before
the servlet container will invalidate this session. An interval value
of zero or less indicates that thesession should never timeout.

Thanks for all your answer.
Finally, I successfully resolve this issue, but in fact... not really.
I notice that there was some specific things in my form.
I had two inputs, one standard file input, and another which receive the file content encoded in base64 by some weird js before any upload.
So I was having one time the raw content of the file, and also the file in base64. Why ?! I don’t know.
But I delete all this, I create a new simple and clean form with a standard input file.
I use the stream API from ServletFileUpload, and it works, and takes only few seconds for big files.
So I don’t understand everything (why the problem was only on some browser for example), but I find a solution ;)
Thank you !

Related

JSF - Trying to download a PDF file but it rerenders the page with strange characters [duplicate]

everyone!
I have a trouble. I tried to save excel file in jsf web application.
I generated file by my utils and trying to get "save" window, but I failed.
Here is my code:
<div>
<h:commandButton value="Apply" actionListener="#{hornPhonesBean.generateReport}"/>
</div>
and:
public void generateReport(ActionEvent event) {
System.out.println("GENERATE REPORT FROM = " + this.dateFrom + "; TO = " + this.dateTo);
try {
XSSFWorkbook workbook = (XSSFWorkbook) HornReportGenerator.getWorkbook(null, null);
String fileName = "1.xlsx";
FacesContext fc = FacesContext.getCurrentInstance();
ExternalContext ec = fc.getExternalContext();
// Some JSF component library or some Filter might have set some headers in the buffer beforehand. We want to get rid of them, else it may collide.
ec.responseReset();
// Check http://www.w3schools.com/media/media_mimeref.asp for all types. Use if necessary ExternalContext#getMimeType() for auto-detection based on filename.
ec.setResponseContentType("application/vnd.ms-excel");
// Set it with the file size. This header is optional. It will work if it's omitted, but the download progress will be unknown.
//ec.setResponseContentLength(contentLength);
// The Save As popup magic is done here. You can give it any file name you want, this only won't work in MSIE, it will use current request URL as file name instead.
ec.setResponseHeader("Content-Disposition", "attachment; filename=\"" + fileName + "\"");
OutputStream output = ec.getResponseOutputStream();
workbook.write(output);
output.flush();
output.close();
fc.responseComplete(); // Important! Otherwise JSF will attempt to render the response which obviously will fail since it's already written with a file and closed.
System.out.println("END");
} catch (Exception e) {
e.printStackTrace();
}
}
I read suggestions here and from another forums - everyone says I shouldnt use , but I didn't use it at all.
Then I thought that the problem could be in the
<ice:form>,
where I kept the
<h:commandButton>,
and I changed to
<h:form>,
but it didn't help.
Maybe the problem in the request - it has header Faces-Request partial/ajax. But I am not sure.
Please give me some ideas - I already spent 4 hours for this crazy jsf download issue)
Maybe the problem in the request - it has header Faces-Request partial/ajax. But I am not sure.
This suggests that the request is an ajax request. You can't download files by ajax. Ajax requests are processed by JavaScript which has for obvious security reasons no facilities to programmatically pop a Save As dialogue nor to access/manipulate client's disk file system.
Your code snippet does however not show that you're using ajax. Perhaps you oversimplified it too much or you're using ICEfaces which silently auto-enables ajax on all standard JSF command components.
In any case, you need to make sure that it's not sending an ajax request.
See also:
How to provide a file download from a JSF backing bean?
ICEfaces libary in classpath prevents Save As dialog from popping up on file download

Why Tomcat returns different headers for HEAD and GET requests to my RESTful API?

My initial purpose was to verify the HTTP chunked transfer. But accidentally found this inconsistency.
The API is designed to return a file to client. I use HEAD and GET methods against it. Different headers are returned.
For GET, I get these headers: (This is what I expected.)
For HEAD, I get these headers:
According to this thread, HEAD and GET SHOULD return identical headers but not necessarily.
My question is:
If Transfer-Encoding: chunked is used because the file is dynamically fed to the client and Tomcat server cannot know its size beforehand, how could Tomcat know the Content-Length when HEAD method is used? Does Tomcat just dry-run the handler and count all the file bytes? Why doesn't it simply return the same Transfer-Encoding: chunked header?
Below is my RESTful API implemented with Spring Web MVC:
#RestController
public class ChunkedTransferAPI {
#Autowired
ServletContext servletContext;
#RequestMapping(value = "bootfile.efi", method = { RequestMethod.GET, RequestMethod.HEAD })
public void doHttpBoot(HttpServletResponse response) {
String filename = "/bootfile.efi";
try {
ServletOutputStream output = response.getOutputStream();
InputStream input = servletContext.getResourceAsStream(filename);
BufferedInputStream bufferedInput = new BufferedInputStream(input);
int datum = bufferedInput.read();
while (datum != -1) {
output.write(datum);
datum = bufferedInput.read();
}
output.flush();
output.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
ADD 1
In my code, I didn't explicitly add any headers, then it must be Tomcat that add the Content-Length and Transfer-Encoding headers as it sees fit.
So, what are the rules for Tomcat to decide which headers to send?
ADD 2
Maybe it's related to how Tomcat works. I hope someone can shed some light here. Otherwise, I will debug into the source of Tomcat 8 and share the result. But that may take a while.
Related:
HTTP HEAD and GET different result
Content-Length header with HEAD requests?
Does Tomcat just dry-run the handler and count all the file bytes?
Yes, the default implementation of javax.servlet.http.HttpServlet.doHead() does that.
You can look at helper classes NoBodyResponse, NoBodyOutputStream in HttpServlet.java
The DefaultServlet class (the Tomcat servlet that is used to serve static files) is more wise. It is capable of sending the correct Content-Length value, as well as serving GET requests for a subset of the file (the Range header). You can forward your request to that servlet, with
ServletContext.getNamedDispatcher("default").forward(request, response);
Although it seems strange, it might make sense to send the size only in response to a HEAD request and chunked in response to a GET request, depending on the type of data that has to be returned by the server.
While your API seems to provide a static file, you also talk about dynamically created files or data, so I will be talking in general here (also for webservers in general).
First let's have a look at the different usages for GET and HEAD:
With GET the client is requesting the whole file or data (or a range of the data), and wants it as fast as possible. So there is no specific reason for the server to send the size of the data first, especially when it could start sending faster/sooner in chunked mode. So the fastest possible way is preferred here (the client will have the size after the download anyway).
With HEAD on the other hand, the client usually wants some specific information. This could just be a check on existance or 'last-changed', but it could also be used if the client wants a certain part of the data (with a range request, including a check to see if range requests are supported for that request), or just needs to know the size of the data up front for some reason.
Lest's look at some possible scenarios:
Static file:
HEAD: there's no reason to not include the size in the response-header because that information is available.
GET: most of the time the size will be inluded in the header and the data sent in one go, unless there are specific performance reasons to send it in chunks. On the other hand it seems you are expecting chunked transfer for you file, so this could make sense here.
Live logfile:
Ok, somewhat strange, but possible: downloading a file where the size could change while downloading.
HEAD: again, the client probably wants the size, and the server can easily provide the size of the file at that specific time in the header.
GET: since loglines could be added while downloading, the size is unknown up front. Only option is to send chunked.
Table with fixed-sized records:
Let's imagine a server needs to send back a table with fixed-length records coming from multiple sources/databases:
HEAD: size is probably wanted by the client. The server could quickly do a query for count in each database, and send the calculated size back to the client.
GET: instead of doing a query for count in each database first, the server better starts sending the resulting records from each database in chunks.
Dynamically generated zip-files:
Maybe not common, but an interesting example.
Imagine you want to provide dynamically generated zip-files to the user based on some parameters.
Let's first have a look at the structure of a zip-file:
There are two parts: first there's a block for each file: a small header followed by the compressed data for that file. Then there's a list of all the files inside the zip-file (including sizes/positions).
So the prepared blocks for each file could be pre-generated on disk (and the names/sizes stored in some data structure.
HEAD: the client probably wants to know the size here. The server can easily calculate the size of all the needed blocks + the size of the second part with the list of the files inside.
If the client wants to extract a single file, it could directly ask for the last part of the file (with a range-request) to have the list, and then with a second request ask for that single file. Although the size is not necessarily needed to get the last n bytes, it could be handy if for example if you wanted to store the different parts in a sparse file with the same size of the full zip-file.
GET: no need to do the calculations first (including generating the second part to know its size). It would be better and faster to just start sending each block in chunks.
Fully dynamically generated file:
In this case it wouldn't be very efficient to return the size to a HEAD request of course, since the whole file would need to be generated just to know its size.

Streaming a file in Liferay Portlet

I have written downloading a file in a simple manner:
#ResourceMapping(value = "content")
public void download(ResourceRequest request, ResourceResponse response) {
//...
SerializableInputStream serializableInputStream = someService.getSerializableInputStream(id_of_some_file);
response.addProperty(HttpHeaders.CACHE_CONTROL, "max-age=3600, must-revalidate");
response.setContentType(contentType);
response.addProperty(HttpHeaders.CONTENT_TYPE, contentType);
response.addProperty(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename*=UTF-8''"
+ URLEncoder.encode(fileName, "UTF-8"));
OutputStream outputStream = response.getPortletOutputStream();
byte[] parcel = new byte[4096];
while (serializableInputStream.read(parcel) > 0)
outputStream.write(parcel);
outputStream.flush();
serializableInputStream.close();
outputStream.close();
//...
}
The SerializableInputStream is described here - JavaDocs. It allows an InputStream to be serialized and, for instance, passed over remoting.
I read from input and write it to the output, not all bytes at once. But unfortunately the portlet isn't "streaming" the contents - the file (e.g. an image) is sent to the browser only after reading the entire input stream - this is how it looks like. I see the file being read from the database (from live logs), but I don't see any "growing" image on the screen.
What am I doing wrong? Is it possible to really stream a file in Liferay 6.0.6 and Spring Portlet MVC?
Where are you doing this? I fear that you're doing this instead of rendering your portlet's HTML (e.g. render phase). Typically the portlet content is embedded in an HTML page, thus you need the resource phase, which (roughly) behaves like a servlet.
Also, the code you give does not match the actual question you ask: You use a comment //read from input stream (file), write file to os and ask what to do differently in order to not have the full content in memory.
As the comment does not have anything in memory and you could loop through reading from the input file while writing to the output stream: What's the underlying question? Do you have problems with implementing download-streaming in a portal environment or difficulties (i.e. using too much memory) reading from a file while writing to a stream?
Edit: Thanks for clarifying. Have you tried to flush the stream earlier? You can do that whenever you want - e.g. every loop (though that might be a bit too much). Also, keep in mind that the browser as well as the file itself must handle it in a way that you expect: If an image is not encoded "incrementally" a browser might not show it that way.
Have you tried this with huge files as well? It might be that the automatic flushing is just not triggered because your files are too small for it to be triggered...
Also, I think that filename*=UTF-8'' looks strange. Might be valid encoding, but I've never seen this

Sharepoint If-Modified-Since

I wrote a handler to be used within SharePoint that will generate a JSON string from a given parameter in the query-string. This JSON string will then be used by a JS function to display the relevant data in HTML to the client, however, the call is somewhat costly and would like to cache the output once.
The handler currently caters for everything regarding OUTPUT cache and tested within an application bares fruits, however, I'm baffled by the fact that, specifically in SharePoint (2007) the "If-Modified-Since" header attribute never appears, basically it always comes back as null.
I found blog-on-blog that discuss this in length in regards to images, and include files but I can't find anything specific regarding this with pages (ASPX, AXD, ASHX) and the handler self.
My only assumption here is the fact that I'm using an AXD file, which is not directly supported by OUTPUT cache by default?
The code looks something like:
bool isModifiedSinceLast = (context.Request.Headers.Get("If-Modified-Since") != null)
: true
? false;
if (!isModifiedSinceLast)
{
context.Response.Headers.AppendHeader("If-Modified-Since", Guid.NewGuid());
}
else
{
// complete the call from cache
}
Thanks,
Eric

How to get WebResource.axd querystring in correct case?

I am getting the css files for minifying and compressing from QueryString["path"] everything works correctly for my own css files like main.css. But when I try to acess the webresource files I receive a 500 error. The parameter which comes after the webresource.axd is case sensitive and I receive it from QueryString["path"] lowercase.
This is what I get from QueryString["path"] :
http://localhost/test/webresource.axd?d=-phgrn6r6zgehvbi697-bxvkl_gidnplxpdeukz5kncgr9hvnfvttpgykwyw05cda-nymtz9od_bbww3ynzxha2&t=633789305460522066
The above link generate error : CryptographicException: Padding is invalid and cannot be removed.
This is what the correct link look like :
http://localhost/test/WebResource.axd?d=-pHGRn6r6ZGehvBI697-BxVKl_GIdNPlxPdEUKZ5KNcGR9hvnfVtTpgyKwYw05cDa-NymTz9OD_bBwW3ynZXhA2&t=633789305460522066
The only difference is in the case. CryptographicException seem to be common but even setting machineKey didn't fixed the problem. Any hint on how could I get the the webresource.axd in the original case?
EDIT
Code was requested :
public void ProcessRequest(HttpContext context) {
Control c = new Control();
string root = context.Request.Url.GetLeftPart(UriPartial.Authority);
string path = context.Request.QueryString["path"];
string content = string.Empty;
if (!string.IsNullOrEmpty(path)) {
if (context.Cache[path] == null) {
List<string> dependencies = new List<string>();
string[] styles = path.Split(new string[] { "," }, StringSplitOptions.RemoveEmptyEntries);
foreach (string style in styles) {
content += RetrieveStyle(root + c.ResolveUrl(style)) + Environment.NewLine;
dependencies.Add(context.Server.MapPath(style));
}
content = StripWhitespace(content);
context.Cache.Insert(path, content, new CacheDependency(dependencies.ToArray()), Cache.NoAbsoluteExpiration, new TimeSpan(DAYS_IN_CACHE, 0, 0, 0));
}
}
}
It crashes in RetreiveStyle when I call :
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
The culprit looks like the code that is generating the “path” querystring csv or some hardware or filter between that source and your handler.
If the source of handler request is a browser what does the handler url look like through view source or firebug? Is it lowercase already?
Working forward from that, do you have any modules etc registered in you IIS pipeline?
I do not have an answer but we have experienced a similar problem and I have a few things to add, which could help identifying the issue.
So, here it goes:
We have an iHTTPHandler (lets call it Login.ashx) that accepts a GET request, which contains a token in base64 format.
The token is then decrypted using Rijndael algorithm.
This process is working most of the time, however, in the last month we had several requests that failed due to System.Security.Cryptography.CryptographicException: Padding is invalid and cannot be removed. error. This error is raised in our case when a token (base64 string) is in lowercase and cannot be decrypted.
After going through logs and activity records I can see that a particular user would attempt to come to our side via Login.ashx and the request would fail due to the error in question. The whole querystring of the request (there is more than just token) including names and values is in lowercase. Then the same user would attempt a login a few minutes later and is able to get in because the querystring was not transformed to lower case.
So, I have a feeling that the issue could be browser related. I am not sure if proxy could affect this.
Additional info:
There is no browser information captured in the server variables.
ALL_HTTP and ALL_RAW variables have almost no data:
ALL_HTTP HTTP_CACHE_CONTROL: no-cache HTTP_HOST:our server name
ALL_RAW Cache-Control: no-cache Host: our server name
There is also no HTTP_REFFERER.
I have tried to replicate this issue with different browsers (Safari3, Chrome1, Opera9.2, IE6,7,8, Firefox3) with no luck.
We have a web farm with 10 servers configured identically (at least I hope they are)
I will add more info if I get any progress.

Resources