I used jxbrowser to tracking data exchange through NetworkDelegate interface, I can get all response data except redirection occurs.
if redirection occurs, the onDataReceived event for original URL will not be happened, so the question is, how can I get response content when redirection occurs, or how to disable automatic redirection.
thanks for any help!
There is no data for the original URL. The response in this case usually contains the status line and Location header. For example:
HTTP/1.1 301 Moved Permanently
Location: http://www.example.org/index.asp
You can obtain this information in JxBrowser using the following approach:
Browser browser = new Browser();
browser.getContext().getNetworkService().setNetworkDelegate(new DefaultNetworkDelegate() {
#Override
public void onBeforeRedirect(BeforeRedirectParams params) {
System.out.println("URL: " + params.getURL());
System.out.println("New URL: " + params.getNewURL());
System.out.println("Response code: " + params.getResponseCode());
}
});
Related
I am trying to play Widevine encrypted content on an Android TV application using Exoplayer. I have my video URL which is served from a CDN and acquired with a ticket. I have my widevine license URL, a ticket and a auth token for the license server.
I am creating a drmSessionManager, putting the necessary headers needed by the license server as follows:
UUID drmSchemeUuid = C.WIDEVINE_UUID;
mediaDrm = FrameworkMediaDrm.newInstance(drmSchemeUuid);
static final String USER_AGENT = "user-agent";
HttpMediaDrmCallback drmCallback = new HttpMediaDrmCallback("my-license-server", new DefaultHttpDataSourceFactory(USER_AGENT));
keyRequestProperties.put("ticket-header", ticket);
keyRequestProperties.put("token-header", token);
drmCallback.setKeyRequestProperty("ticket-header", ticket);
drmCallback.setKeyRequestProperty("token-header", token);
new DefaultDrmSessionManager(drmSchemeUuid, mediaDrm, drmCallback, keyRequestProperties)
After this Exoplayer handles most of the stuff, the following breakpoints are hit.
response = callback.executeKeyRequest(uuid, (KeyRequest) request);
in class DefaultDrmSession
return executePost(dataSourceFactory, url, request.getData(), requestProperties) in HttpMediaDrmCallback
I can observe that everything is fine till this point, the URL is correct, the headers are set fine.
in the following piece of code, I can observe that the dataSpec is fine, trying to POST a request to the license server with the correct data, but when making the connection the response code returns 405.
in class : DefaultHttpDataSource
in method : public long open(DataSpec dataSpec)
this.dataSpec = dataSpec;
this.bytesRead = 0;
this.bytesSkipped = 0;
transferInitializing(dataSpec);
try {
connection = makeConnection(dataSpec);
} catch (IOException e) {
throw new HttpDataSourceException("Unable to connect to " + dataSpec.uri.toString(), e,
dataSpec, HttpDataSourceException.TYPE_OPEN);
}
try {
responseCode = connection.getResponseCode();
responseMessage = connection.getResponseMessage();
} catch (IOException e) {
closeConnectionQuietly();
throw new HttpDataSourceException("Unable to connect to " + dataSpec.uri.toString(), e,
dataSpec, HttpDataSourceException.TYPE_OPEN);
}
When using postman to make a request to the URL, a GET request returns the following body with a response code of 405.
{
"Message": "The requested resource does not support http method 'GET'." }
a POST request also returns response code 405 but returns an empty body.
In both cases the following header is also returned, which I suppose the request must be accepting GET and POST requests.
Access-Control-Allow-Methods →GET, POST
I have no access to the configuration of the DRM server, and my contacts which are responsible of the DRM server tells me that POST requests must be working fine since there are clients which have managed to get the content to play from the same DRM server.
I am quite confused at the moment and think maybe I am missing some sort of configuration in exoplayer since I am quite new to the concept of DRMs.
Any help would be greatly appreciated.
We figured out the solution. The ticket supplied for the DRM license server was wrong. This works as it is supposed to now and the content is getting played. Just in case anyone somehow gets the same problem or is in need of a basic Widevine content playing code, this works fine at the moment.
Best regards.
I feel like this should be easy. I have an app where all I am trying to do is have a form page (index.jsp) that calls a servlet (CheckInfo.java) which sets a new header (myHeader) and redirects the user to another page (redirect.jsp). All of these files are on the same server. The index.jsp is sending the request just fine and CheckInfo is processing and redirecting, but myHeader is not showing up on redirect.jsp. I've read several posts talking about response.sendRedirect sends a 302 which doesn't pass headers and that I should use RequestDispatcher, but nothing seems to work. Is there no way to send headers from a servlet to a jsp?
Here is the servlet code:
response.setHeader("myHeader", "hey there");
response.sendRedirect("redirect.jsp");
I have also tried this:
response.setHeader("myHeader", "hey there");
RequestDispatcher view = request.getRequestDispatcher("redirect.jsp");
view.forward(request, response);
And I have this in redirect.jsp:
System.out.println(request.getHeader("myHeader"));
This does not print anything.
If the answer to my question is no... then I would settle for a way to set the header once I got back to the jsp. My reverse proxy is looking for a specific header to determine whether or not to perform an action. Obviously I tried response.addHeader() on redirect.jsp, but the page has already loaded at that point so that just made me feel dumb.
response.setHeader("myHeader", "hey there");
response.sendRedirect("redirect.jsp");
You are adding it as response header and it is 302 response. Browser on seeing a 302 response will just look for Location header and fire a new request to this location. Custom headers in the response are untouched whereas you are expecting these custom response headers to be included in the request (to new redirect location) which is not being sent.
Solution:-
1. you can use request dispatcher and forward the request instead of external redirect. And you need to use request attributes here.
2. you can call submit form using an ajax request may be jquery like and handle the response manually(for 302 response) but would not suggest you to use this approach as it is not a cleaner and intuitive approach. Just mentioning so that you know there are other ways to achieve this.
The problem is that the redirect() method of the response initiates a new request altogether, thereby loosing the attributes that were set before redirecting. Luckily there is a fluent way of solving the problem still. See below
response.setHeader("myHeader", "hey there");
request.getRequestDispatcher("redirect.jsp").forward(request, response);
Then in your destination you can do response.getHeaders("myHeader")
I have tested the code.
I hope it's clear that in case of asking the client to redirect to another URL - the browser shall not honor the cookies.
However, the 2nd method - where server forwards the request is feasible. The main mistake appears to be in mutating the response while we are supposed to change the request.
Then again, one cannot directly mutate a HttpServletRequest object. Here is one way to do so:
HttpServletRequestWrapper requestWrapper = new HttpServletRequestWrapper(request){
public String getHeader(String name) {
String value = super.getHeader(name);
if(Strings.isNullOrEmpty(value)) {
...
value = myNewHeader;
}
return value;
}
public Enumeration<String> getHeaders(String name) {
List<String> values = Collections.list(super.getHeaders(name));
if(values.size()==0) {
...
values.add(myNewHeader);
}
return Collections.enumeration(values);
}
public Enumeration<String> getHeaderNames() {
List<String> names = Collections.list(super.getHeaderNames());
names.add(myNewHeaderName);
...
return Collections.enumeration(names);
}
}
Followed by:
RequestDispatcher view = request.getRequestDispatcher("redirect.jsp");
// OR (If you can get servletContext)
RequestDispatcher view = servletContext.getRequestDispatcher("redirect.jsp");
view.forward(requestWrapper, response);
Reference:
https://docs.oracle.com/javaee/7/api/javax/servlet/http/HttpServletRequestWrapper.html
For the headers case - getHeader(), getHeaders() and getHeaderNames() fn in the reqWrapper obj need Overriding.
Similarly you can override cookies and params.
See also: Modify request parameter with servlet filter
NOTE: It might not be possible to forward a req to an endpoint which expects a different MIME type.
A client side redirect creates a new HTTP request/response pair.
This link may help you more on debugging perspective -
Sending Custom headers
i have images url , i need to check url is responding or not .
For Example :Below i i have written three image url, first two url is not valid only third url is valid .but second and fourth url is responding as valid image
and but there is no image.
http://media.expedia.com/hotels/1000000/90000/84900/84853/84853_744_b.jpg
http://www.iceportal.com/brochures/media/show.aspx?brochureid=ICE19044&did=3073&mtype=3073&type=pic&lang=en&publicid=4175749&resizing=X
http://images.trvl-media.com/hotels/1000000/30000/20400/20313/20313_166_b.jpg
http://www.iceportal.com/brochures/ice/ErrorPages/404.htm?aspxerrorpath=/brochures/media/show_A.aspx
here is my code:
public static bool CheckUrlExists(string url)
{
try
{
Uri u = new Uri(url);
WebRequest w = WebRequest.Create(u);
w.Method = WebRequestMethods.Http.Head;
using (StreamReader s = new StreamReader(w.GetResponse().GetResponseStream()))
{
return (s.ReadToEnd().Length >= 0);
}
}
catch
{
return false;
}
}
with this code i am validating only those url which is showing 404 error,but not those url which showing 'Sorry, requested brochure is temporarily un-published 'or any other type of message.
You will need a more complex logic to validate if the URL points to an image. If a resource is missing from the server or it is otherwise unavailable, you may get a HTTP error like the infamous 404, which will trigger a WebException. However, that is only part of the story.
Your second URL returns HTTP 200, confirming that the resource is there when in fact the resource is missing. What you really get there is a HTML document explaining the resource is not available. This is bad practice, but not without example.
At very least, you should examine the MIME type (Content-Type header, see WebResponse.ContentType) of the resource you test. A content type of image/* suggests an image-type resource. Failing to detect a known MIME type (e.g. if you receive application/octet-stream) you can actually HTTP GET get resource and run image type detection on the downloaded content.
I would suggest using HttpWebRequest and HttpWebResponse to do this, they are sub classes of WebRequest and WebResponse and as such are more granular for what you're trying to achive. The following code works with the example URIs provided
public static bool CheckUrlExists(string url)
{
try
{
Uri u = new Uri(url);
HttpWebRequest w = (HttpWebRequest)WebRequest.Create(u);
w.AllowAutoRedirect = false;
w.Method = WebRequestMethods.Http.Head;
HttpWebResponse response = (HttpWebResponse)w.GetResponse();
return response.StatusCode == HttpStatusCode.OK; //Check http status code
}
catch(WebException ex)
{
return false;
}
}
What's important here is that I'm checking the HttpStatus code. You're catch will already catch the 404s but the problem URIs ultimately lead to a 200 (OK). By setting AllowAutoRedirect to false the HttpWebRequest instance returns a 302 (redirect) status code, instead of following the redirect through to the "Sorry, requested brochure is temporarily un-published." page which is returning 200 (OK). This should serve your purpose.
Also: Catching a WebException will allow you to examine the status code (400+,500+, etc).
Be aware however, that you may be redirected to a new location for the image you're requesting. Taking that you might want to use PeterK's mime type check.
Is it possible to get if an URL is 404 or 301 in crawler4j ?
#Override
public void visit(Page page) {
String url = page.getWebURL().getURL();
System.out.println("URL: " + url);
if (page.getParseData() instanceof HtmlParseData) {
HtmlParseData htmlParseData = (HtmlParseData) page.getParseData();
String text = htmlParseData.getText();
String html = htmlParseData.getHtml();
List<WebURL> links = htmlParseData.getOutgoingUrls();
System.out.println("Text length: " + text.length());
System.out.println("Html length: " + html.length());
System.out.println("Number of outgoing links: " + links.size());
}
}
I use this in the crawler code .Can anyone tell me how ?
As Crawler4j Version 3.3 (Feb 2012 released) - Crawler4j supportting for handling http status codes for fetched pages.
to visit StatusHandlerCrawlerExample click.
Also you can parse pages by using Jsoup (Java HTML Parser, with best of DOM, CSS, and jquery). And there is an example here - shows how to download a page from given URL and getting page status code. I think you should use Crawler4j for crawling and Jsoup for page fetching.
I'm in the process of creating a REST API using HttpListener in .NET (C#). This all works out great, except for one slight issue.
I'm trying to return responses with Status Codes other than OK (200), for instance ResourceNotFound (404).
When I set the StatusCode of the HttpListenerResponse to something other than 200, and create a response body (using HttpListenerResponse.OutputStream), it seems to be resetting the status code to 200. I'm not able to send a response with StatusCode 404 and a message body. However, this should be possible according to the HTTP specs. I'm checking the requests and responses with Fiddler, but I'm not able to get what I'm looking for.
I've had the same problem and found the source of the problem :
If you write the body in the OutputStream before set the StatusCode (or any other property), the response will be sent before the modification is applied !
So, you have to proceed in this order :
public void Send(HttpListenerContext context, byte[] body)
{
// First, set a random status code and other stuffs
context.Response.StatusCode = (int)HttpStatusCode.Unauthorized;
context.Response.ContentType = "text/plain";
// Write to the stream IN LAST (will send request)
context.Response.OutputStream.Write(body, 0, body.Length);
}