I am trying to resume an upload from google drive, I have the id of the file when I close the connection I make this session after the Internet is connected again:
var request = (HttpWebRequest)WebRequest.Create(fileUri);
request.Method = "PUT";
request.ContentLength = 0;
request.Headers.Add("Content-Range", "bytes */" + FileByteArray.Length);
try
{
var response = request.GetResponse();
}
catch (WebException e)
{
fileRange = e.Response.Headers.Get("Range");
}
}
I am not getting the Range header, why is that?
Content-Range indicates which range of the original entity is contained in the body of either a request (like your PUT request) or a 206 partial response. Range is set by the client not the server in order to request a sub-range. I would assume that the server you are talking to will not respond with the uploaded chunk, so a Content-Range (and Range in no case) will not be present as a response header.
In your code snippet the actual upload range is missing for Content-Range (see the updated HTTP RFC). It has to have the form of:
Content-Range: bytes 42-1233/1234
which means: upload byte 42-1233 of an entity whose total size is 1234 byte.
Or when the complete length is unknown:
Content-Range: bytes 42-1233/*
So remove the check for the Range header and specify the complete upload range and you should be fine.
Related
RequestLogHandler requestLogHandler = new RequestLogHandler();
Slf4jRequestLog requestLog = new CustomSlf4jRequestLog();
requestLogHandler.setRequestLog(requestLog);
Slf4jRequestLog is only logging request method, url and date, and response status code and bytes written.
I definitely want to log body for my PUT/POST requests.
I derived CustomSlf4jRequestLog from Slf4jRequestLog and I tried:
public void log(Request request, Response response) {
StringBuilder sb = new StringBuilder("RequestBody: ");
try {
LOG.info("BODY SIZE: " + request.getContentLength());
BufferedReader in = new BufferedReader(new InputStreamReader(request.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
sb.append(line);
}
LOG.info(sb.toString());
Unfortunately no body is printed out as it is already processed by handler?
Is it possible to get body of request here?
(I really care about body because I have JsonProvider and I want to see a whole body of request when Json fails to parse data) Or when my app fails I want to see what caused that without adding logging for each input request.
Servlet spec getting in your way here.
The actual servlet that is processing the request has already read the request body, rending further reads from request.getInputStream() invalid (you are at EOF)
If you want to capture the request body, you'll need to create a custom Servlet Filter, and then wrap the HttpServletRequest, overriding the getInputStream() AND getReader() methods with your own implementations that make a copy of the data that was read.
Then its up to you to determine what you want to do with that copy of the request body content.
or
You can just use a network capture utility like Wireshark to see what the request body was. Even if you use HTTPS, you can configure Wireshark with your server certificate to inspect encrypted conversation.
The logger is calling getInputStream() on the same request again. You are not allowed to read the same data twice. Yo should create a ServletRequestWrapper to make a copy of the body of the request.
i have images url , i need to check url is responding or not .
For Example :Below i i have written three image url, first two url is not valid only third url is valid .but second and fourth url is responding as valid image
and but there is no image.
http://media.expedia.com/hotels/1000000/90000/84900/84853/84853_744_b.jpg
http://www.iceportal.com/brochures/media/show.aspx?brochureid=ICE19044&did=3073&mtype=3073&type=pic&lang=en&publicid=4175749&resizing=X
http://images.trvl-media.com/hotels/1000000/30000/20400/20313/20313_166_b.jpg
http://www.iceportal.com/brochures/ice/ErrorPages/404.htm?aspxerrorpath=/brochures/media/show_A.aspx
here is my code:
public static bool CheckUrlExists(string url)
{
try
{
Uri u = new Uri(url);
WebRequest w = WebRequest.Create(u);
w.Method = WebRequestMethods.Http.Head;
using (StreamReader s = new StreamReader(w.GetResponse().GetResponseStream()))
{
return (s.ReadToEnd().Length >= 0);
}
}
catch
{
return false;
}
}
with this code i am validating only those url which is showing 404 error,but not those url which showing 'Sorry, requested brochure is temporarily un-published 'or any other type of message.
You will need a more complex logic to validate if the URL points to an image. If a resource is missing from the server or it is otherwise unavailable, you may get a HTTP error like the infamous 404, which will trigger a WebException. However, that is only part of the story.
Your second URL returns HTTP 200, confirming that the resource is there when in fact the resource is missing. What you really get there is a HTML document explaining the resource is not available. This is bad practice, but not without example.
At very least, you should examine the MIME type (Content-Type header, see WebResponse.ContentType) of the resource you test. A content type of image/* suggests an image-type resource. Failing to detect a known MIME type (e.g. if you receive application/octet-stream) you can actually HTTP GET get resource and run image type detection on the downloaded content.
I would suggest using HttpWebRequest and HttpWebResponse to do this, they are sub classes of WebRequest and WebResponse and as such are more granular for what you're trying to achive. The following code works with the example URIs provided
public static bool CheckUrlExists(string url)
{
try
{
Uri u = new Uri(url);
HttpWebRequest w = (HttpWebRequest)WebRequest.Create(u);
w.AllowAutoRedirect = false;
w.Method = WebRequestMethods.Http.Head;
HttpWebResponse response = (HttpWebResponse)w.GetResponse();
return response.StatusCode == HttpStatusCode.OK; //Check http status code
}
catch(WebException ex)
{
return false;
}
}
What's important here is that I'm checking the HttpStatus code. You're catch will already catch the 404s but the problem URIs ultimately lead to a 200 (OK). By setting AllowAutoRedirect to false the HttpWebRequest instance returns a 302 (redirect) status code, instead of following the redirect through to the "Sorry, requested brochure is temporarily un-published." page which is returning 200 (OK). This should serve your purpose.
Also: Catching a WebException will allow you to examine the status code (400+,500+, etc).
Be aware however, that you may be redirected to a new location for the image you're requesting. Taking that you might want to use PeterK's mime type check.
An image gets corrupted while being retrieved (through HTTP) and then sent (through HTTP) to a database. Image's raw data is handled in String form.
The service sends a GET for an image file, receives response with the raw image data (response's body) and the Content-Type. Then, a PUT request is sent with the aforementioned request's body and Content-Type header. (The PUT request is constructed by providing the body in String) This PUT request is sent to a RESTful database (CouchDB), creating an attachment (for those unfamiliar with CouchDB an attachment acts like a static file).
Now I have the original image, which my service GETs and PUTs to a database, and this 'copy' of the original image, that I can now GET from the database. If I then `curl --head -v "[copy's url]" it has the Content-Type of the original image, but Content-Length has changed, went from 200kb to about 400kb. If I GET the 'copy' image with a browser, it is not rendered, whereas, the original renders fine. It is corrupted.
What might be the cause? My guess is that while handling the raw data as a string, my framework guesses the encoding wrong and corrupts it. I have not been able to confirm or deny this. How could I handle this raw data/request body in a safe manner, or how could I properly handle the encoding (if that proves to be the problem)?
Details: Play2 Framework's HTTP client, Scala. Below a test to reproduce:
"able to copy an image" in {
def waitFor[T](future:Future[T]):T = { // to bypass futures
Await.result(future, Duration(10000, "millis"))
}
val originalImageUrl = "http://laughingsquid.com/wp-content/uploads/grumpy-cat.jpg"
val couchdbUrl = "http://admin:admin#localhost:5984/testdb"
val getOriginal:ws.Response = waitFor(WS.url(originalImageUrl).get)
getOriginal.status mustEqual 200
val rawImage:String = getOriginal.body
val originalContentType = getOriginal.header("Content-Type").get
// need an empty doc to have something to attach the attachment to
val emptyDocUrl = couchdbUrl + "/empty_doc"
val putEmptyDoc:ws.Response = waitFor(WS.url(emptyDocUrl).put("{}"))
putEmptyDoc.status mustEqual 201
//uploading an attachment will require the doc's revision
val emptyDocRev = (putEmptyDoc.json \ "rev").as[String]
// create actual attachment/static file
val attachmentUrl = emptyDocUrl + "/0"
val putAttachment:ws.Response = waitFor(WS.url(attachmentUrl)
.withHeaders(("If-Match", emptyDocRev), ("Content-Type", originalContentType))
.put(rawImage))
putAttachment.status mustEqual 201
// retrieve attachment
val getAttachment:ws.Response = waitFor(WS.url(attachmentUrl).get)
getAttachment.status mustEqual 200
val attachmentContentType = getAttachment.header("Content-Type").get
originalContentType mustEqual attachmentContentType
val originalAndCopyMatch = getOriginal.body == getAttachment.body
originalAndCopyMatch aka "original matches copy" must beTrue // << false
}
Fails at the last 'must':
[error] x able to copy an image
[error] original matches copy is false (ApplicationSpec.scala:112)
The conversion to String is definitely going to cause problems. You need to work with the bytes as Daniel mentioned.
Looking at the source it looks like ws.Response is just a wrapper. If you get to the underlying class then there are some methods that may help you. On the Java side, someone made a commit on GitHub to expose more ways of getting the response data other than a String.
I'm not familiar with scala but something like this may work:
getOriginal.getAHCResponse.getResponseBodyAsBytes
// instead of getOriginal.body
WS.scala
https://github.com/playframework/playframework/blob/master/framework/src/play/src/main/scala/play/api/libs/ws/WS.scala
WS.java
Here you can see that Response has some new methods, getBodyAsStream() and asByteArray.
https://github.com/playframework/playframework/blob/master/framework/src/play-java/src/main/java/play/libs/WS.java
I'm in the process of creating a REST API using HttpListener in .NET (C#). This all works out great, except for one slight issue.
I'm trying to return responses with Status Codes other than OK (200), for instance ResourceNotFound (404).
When I set the StatusCode of the HttpListenerResponse to something other than 200, and create a response body (using HttpListenerResponse.OutputStream), it seems to be resetting the status code to 200. I'm not able to send a response with StatusCode 404 and a message body. However, this should be possible according to the HTTP specs. I'm checking the requests and responses with Fiddler, but I'm not able to get what I'm looking for.
I've had the same problem and found the source of the problem :
If you write the body in the OutputStream before set the StatusCode (or any other property), the response will be sent before the modification is applied !
So, you have to proceed in this order :
public void Send(HttpListenerContext context, byte[] body)
{
// First, set a random status code and other stuffs
context.Response.StatusCode = (int)HttpStatusCode.Unauthorized;
context.Response.ContentType = "text/plain";
// Write to the stream IN LAST (will send request)
context.Response.OutputStream.Write(body, 0, body.Length);
}
I am trying to send a file to HTTP server via POST request (c++ and winapi), steps:
// Read file into "buff" and file size into "buffSize"
....
....
....
HINTERNET internetRoot;
HINTERNET httpSession;
HINTERNET httpRequest;
internetRoot = InternetOpen(agent_info, INTERNET_OPEN_TYPE_DIRECT, NULL, NULL, NULL);
//Connecting to the http server
httpSession = InternetConnect(internetRoot, IP,PORT_NUMBER, NULL, NULL, INTERNET_SERVICE_HTTP, NULL, NULL);
//Creating a new HTTP POST request to the default resource on the server
httpRequest = HttpOpenRequest(httpSession, TEXT("POST"), TEXT("/Post.aspx"), NULL, NULL, NULL, INTERNET_FLAG_RELOAD | INTERNET_FLAG_NO_CACHE_WRITE, NULL);
//Send POST request
HttpSendRequest(httpRequest, NULL, NULL, buff, buffSize);
//Closing handles...
In server I am recieving the file using this code (asp.net)
Stream httpStream;
try
{
httpStream = request.RequestContext.HttpContext.Request.InputStream;
}
catch (HttpException)
{
return;
}
byte[] tmp = new byte[httpStream.Length];
int bytesRead = httpStream.Read(tmp, 0, 1024 * 1024);
int totalBytesRead = bytesRead;
while (bytesRead > 0)
{
bytesRead = httpStream.Read(tmp, totalBytesRead, 1024 * 1024);
totalBytesRead += bytesRead;
}
httpStream.Close();
httpStream.Dispose();
//Save "tmp" to file...
I can send large files on local server (visual studio asp server), but I cannot send files over 1 MB to internet server. (HttpOpenRequest is failing)
is there a better way to upload files?
Caveat: My Wininet is very rusty these days.
I wonder whether you ought to be setting the "Content-Length" header yourself. Your code seems to assume that either a) you are making a HTTP/1.0 request or b) that HttpSendRequest will add the header for your (which I don't think it does).
Either way without the server being told how big the incoming request is the default configuration of IIS will reject it if it can't determine the request size itself quickly.
My guess is if you use the lpszHeaders and dwHeadersLength parameters of the HttpSendRequest function to include the appropriate "Content-Length" header the problem will be resolved.
What error do you receive? I mean what does GetLastError() returns? If you send file 800KB then it works OK? I dont really see how because HttpOpenRequest does not know about size of the data.
Maybe it timeouts? But this would mean that HttpSendRequest actually fails. It might buffer all data but since size is huge, then it takes more time than timeout allows.
use following code to query current timeouts (in ms):
InternetQueryOption(h, INTERNET_OPTION_RECEIVE_TIMEOUT, &dwReceiveTimeOut, sizeof(dwReceiveTimeOut));
InternetQueryOption(h, INTERNET_OPTION_SEND_TIMEOUT, &dwSendTimeOut, sizeof(dwSendTimeOut));
and following to set new ones:
InternetSetOption(h, INTERNET_OPTION_RECEIVE_TIMEOUT, &dwNewReceiveTimeOut, sizeof(dwNewReceiveTimeOut));
InternetSetOption(h, INTERNET_OPTION_SEND_TIMEOUT, &dwNewSendTimeOut, sizeof(dwNewSendTimeOut));