Do you know, if the HTTP protocol supports to request the maximal size of a Post request?
I want to implement an upload of a large file via HTTP Http. The file upload needs to be split into chunks, because the server/application has a limit of 25 MB of Post-Request and the files are larger than 25 MB. To avoid setting a hardcoded limit of 25 MB I rather would get the limit size by the server.
Related
I want to serve files(images) via gRPC Gateway from gRPC server. Since protocol buffers messages have sctructure, I don't see how I could ensure the gateway to send content of the bytes field of the response message instead of the entire json-encoded message. Is there a native solution for this or does one simply have to write a dedicated http muxer to handle these requests?
Rather than sending arbitrarily-sized files (probably as bytes), it would probably be better to include a URL to the file in the message and then host/serve the file over HTTP (e.g. from S3, Google Cloud Storage etc. from which you could generate signed URLs to limit access).
I think the max message size is 2GB (source?) and the recommendation (Large Data Sets) is to consider alternative techniques once messages sizes exceed few MBs.
I'm been building a NodeJS application with a team. A team member limited the server request size to 8Kb - if it's bigger than that, the request will be rejected on the server. The idea is that we don't want to process requests that are too big to avoid a potential DoS.
This brings up the issue that what if we wanted to make generally big requests (batch a couple of small requests together, since according to this, it's better than sending a bunch of small requests). And example would be for a TODO list, if I edit 100 TODOs at the same time (I send the UUIDs of each of the todo items back to the server along with the updates); this request could exceed 8Kb in size. I couldn't find if there are standards for max HTTP request sizes.
What would be the solution for wanting to send back larger HTTP requests from the client to the server? Should I:
Increase the HTTP request size on the server? What's the standard? I could 100x it and that would solve much of the problem
Limit the request size on the client. For example, limit it so the user could only edit 100 TODOs at max! Anymore, and the request won't send.
A combination of 1 and 2?
Thank you!
As I understand, the body size of HTTP POST request is unlimited. Thus a client may send gigabytes of data in one HTTP request. Now I wonder how an HTTP server should handle such requests. How do Tomcat and Jetty handle them ?
Not true. For example Apache has a default size of 20 MB, configurable in httpd.conf. Then TCP connection will be closed so client cannot send anymore.
Tomcat is configured to limit the post size to 2 MB by default. See http://tomcat.apache.org/tomcat-7.0-doc/config/http.html:
maxPostSize
The maximum size in bytes of the POST which will be handled by the container FORM URL parameter parsing. The limit can be disabled by setting this attribute to a value less than or equal to 0. If not specified, this attribute is set to 2097152 (2 megabytes).
What is the specification for maximum data size one can send with HTTP POST method?
Quite amazing how all answers talk about IIS, as if that were the only web server that mattered. Even back in 2010 when the question was asked, Apache had between 60% and 70% of the market share. Anyway,
The HTTP protocol does not specify a limit.
The POST method allows sending far more data than the GET method, which is limited by the URL length - about 2KB.
The maximum POST request body size is configured on the HTTP server and typically ranges from
1MB to 2GB
The HTTP client (browser or other user agent) can have its own limitations. Therefore, the maximum POST body request size is min(serverMaximumSize, clientMaximumSize).
Here are the POST body sizes for some of the more popular HTTP servers:
Nginx (largest web server market share as of April 2019) - default 1MB, no practical maximum (2**63)
Apache - maximum 2GB, no default documented
IIS - default 28.6MB for the request length, 2048 bytes for the query string; maximum undocumented
InfluxDB - default ~25MB, maximum undocumented
EDIT (2019) This answer is now pretty redundant but there is another answer with more relevant information.
It rather depends on the web server and web browser:
Internet explorer All versions 2GB-1
Mozilla Firefox All versions 2GB-1
IIS 1-5 2GB-1
IIS 6 4GB-1
Although IIS only support 200KB by default, the metabase needs amending to increase this.
http://www.motobit.com/help/scptutl/pa98.htm
The POST method itself does not have any limit on the size of data.
There is no limit according to the HTTP protocol itself, but implementations will have a practical upper limit. I have sent data exceeding 4 GB using POST to Apache, but some servers did have a limit of 4 GB at the time.
POST allows for an arbitrary length of data to be sent to a server, but there are limitations based on timeouts/bandwidth etc.
I think basically, it's safer to assume that it's not okay to send lots of data.
Different IIS web servers can process different amounts of data in the 'header', according to this (now deleted) article; http://classicasp.aspfaq.com/forms/what-is-the-limit-on-form/post-parameters.html;
Note that there is no limit on the
number of FORM elements you can pass
via POST, but only on the aggregate
size of all name/value pairs. While
GET is limited to as low as 1024
characters, POST data is limited to 2
MB on IIS 4.0, and 128 KB on IIS 5.0.
Each name/value is limited to 1024
characters, as imposed by the SGML
spec. Of course this does not apply to
files uploaded using
enctype='multipart/form-data' ... I
have had no problems uploading files
in the 90 - 100 MB range using IIS
5.0, aside from having to increase the server.scriptTimeout value as well as
my patience!
In an application I was developing I ran into what appeared to be a POST limit of about 2KB. It turned out to be that I was accidentally encoding the parameters into the URL instead of passing them in the body. So if you're running into a problem there, there is definitely a very small limit on the size of POST data you can send encoded into the URL.
HTTP may not have an upper limit, but webservers may have one. In ASP.NET there is a default accept-limit of 4 MB, but you (the developer/webmaster) can change that to be higher or lower.
I have a web application that adds contextual information to XmlHttpRequest objects using the setRequestHeader API. I am using a custom header name (e.g. X-Foo) and a JSON structured value. It isn't part of the URL QueryString or POST body because it is meta information about the request.
Is there a practical size limit to the header value? If my JSON gets truncated, it becomes unparseable. I am most concerned with limits in Apache 2, Tomcat 6 and IIS 7. I did a Google search for http header length limit, but many of the results seem dated. There are some relevant comments in How big can a user agent string get? but not as specific as I would like.
Edit:
I just ran across this similar question - Maximum on http header values?
Although each web server software has some limitations, there is a difference whether there’s a limit for the HTTP request line plus header fields or for each header field.
Here’s a summary:
Apache 1.3, 2.0, 2.2, 2.3: 8190 Bytes (for each header field)
IIS:
4.0: 2097152 Bytes (for the request line plus header fields)
5.0: 131072 Bytes, 16384 Bytes with Windows 2000 Service Pack 4 (for the request line plus header fields)
6.0: 16384 Bytes (for each header fields)
Tomcat:
5.5.x/6.0.x: 49152 Bytes (for the request line plus header fields)
7.0.x: 8190 Bytes (for the request line plus header fields)
So to conclude: To be accepted by all web servers above, a request’s request line plus header fields should not exceed 8190 Bytes. This is also the limit for each header fields (effectively even less).
Yes, but the limits are configurable and dependent on platform. For example, Tomcat has a default limit of 8K. I believe that IIS 6, not sure about IIS 7, has a limit of 16K. I ran into this when using integrated windows authentication for several web sites. Turns out my security token was too large when encoded into the header. Fortunately, these are configurable. Registry settings for IIS can be found at http://support.microsoft.com/kb/820129. I believe the key settings to change are MaxFieldLength (per header size) and MaxRequestBytes (total size of request).
For Apache, I found this Server Limits for Apache Security article that lists these directives:
# allow up to 100 headers in a request
LimitRequestFields 100
# each header may be up to 8190 bytes long
LimitRequestFieldsize 8190
For Nginx, the large_client_header_buffers directive from HttpCoreModule controls this:
The longest header line of request also must be not more than the size
of one buffer, otherwise the client get the error "Bad request" (400).
By default the size of one buffer is equal to the size of page,
depending on platform this either 4K or 8K
While you can configure the server, it's unlikely that you really can configure the whole way through firewalls, load balancers and proxies. Keeping the header size small keeps problems away.
The Flash Media Server 4.5 has a very short default header length limit which can cause the server to simply not respond, particularly in circumstances where there is a moderate cookie load.
See: Flash Media Server 4.5 Configuration and Administration: Configuring the server
Configuring Apache HTTP Server: Specify the maximum HTTP header line length
In the Flash Media Server Adaptor.xml file, the MaxHeaderLineLength
element determines the size of the HTTP header the server can handle.
The default value for MaxHeaderLineLength is 1024 bytes. Some browsers
send a header larger than 1024 bytes. In this scenario, Apache sends
back an empty response. To fix this issue, configure
MaxHeaderLineLength to 8192.
Note: By default, the Apache HTTP header size limit is 8 KB (8190 bytes plus a carriage return).
Putting this here in case the header size limit on Flash Media Server bites someone else.