So I've got a site in .NET 2.0 (VB.NET)
I need to send a request to a third party service with a WebRequest. Unfortunately, the request needs to send URL encoded data in the query string and it appears that the HTTP request object automatically decodes URLs.
I ran across this post
Creating an Uri in .NET automatically urldecodes all parameters from passed string
and this post Help with C# HttpWebRequest URI losing its encoding which point me to this documentation https://msdn.microsoft.com/en-us/library/system.uri(v=VS.100).aspx which basically tells me I'm screwed and I"m going to have to hack like crazy to get this to work.
There seems like there has to be an easier way to stop the WebRequest from automatically decoding the URLs. Does anyone have any ideas?
Hello everybody I'm trying to do a simple HTTP call to a Tomcat Server running on my server from my Android App. The server will then execute a certain command to my website. I created a button that when I click it runs the HTTP call from the App.
If I use the approach below, it opens the browser on my phone to run this HTTP. Is it possible to do something similar but not have my app open the browser???
Intent browserIntent = new Intent(Intent.ACTION_VIEW, Uri.parse("http://" + IP + ":8080/server/run.jsp"));
startActivity(browserIntent);
thank you so much in advance :D
Of course it starts your browser. Your code is explicitly asking Android to launch an app that can "view" the URL.
If you want your app to access the URL directly, use HttpURLConnection instead:
1.Obtain a new HttpURLConnection by calling URL.openConnection() and casting the result to HttpURLConnection.
2.Prepare the request. The primary property of a request is its URI. Request headers may also include metadata such as credentials, preferred content types, and session cookies.
3.Optionally upload a request body. Instances must be configured with setDoOutput(true) if they include a request body. Transmit data by writing to the stream returned by getOutputStream().
4.Read the response. Response headers typically include metadata such as the response body's content type and length, modified dates and session cookies. The response body may be read from the stream returned by getInputStream(). If the response has no body, that method returns an empty stream.
5.Disconnect. Once the response body has been read, the HttpURLConnection should be closed by calling disconnect(). Disconnecting releases the resources held by a connection so they may be closed or reused.
What is the conceptual difference between forward() and sendRedirect()?
In the web development world, the term "redirect" is the act of sending the client an empty HTTP response with just a Location header containing the new URL to which the client has to send a brand new GET request. So basically:
Client sends a HTTP request to some.jsp.
Server sends a HTTP response back with Location: other.jsp header
Client sends a HTTP request to other.jsp (this get reflected in browser address bar!)
Server sends a HTTP response back with content of other.jsp.
You can track it with the web browser's builtin/addon developer toolset. Press F12 in Chrome/IE9/Firebug and check the "Network" section to see it.
Exactly the above is achieved by sendRedirect("other.jsp"). The RequestDispatcher#forward() doesn't send a redirect. Instead, it uses the content of the target page as HTTP response.
Client sends a HTTP request to some.jsp.
Server sends a HTTP response back with content of other.jsp.
However, as the original HTTP request was to some.jsp, the URL in browser address bar remains unchanged. Also, any request attributes set in the controller behind some.jsp will be available in other.jsp. This does not happen during a redirect because you're basically forcing the client to create a new HTTP request on other.jsp, hereby throwing away the original request on some.jsp including all of its attribtues.
The RequestDispatcher is extremely useful in the MVC paradigm and/or when you want to hide JSP's from direct access. You can put JSP's in the /WEB-INF folder and use a Servlet which controls, preprocesses and postprocesses the requests. The JSPs in the /WEB-INF folder are not directly accessible by URL, but the Servlet can access them using RequestDispatcher#forward().
You can for example have a JSP file in /WEB-INF/login.jsp and a LoginServlet which is mapped on an url-pattern of /login. When you invoke http://example.com/context/login, then the servlet's doGet() will be invoked. You can do any preprocessing stuff in there and finally forward the request like:
request.getRequestDispatcher("/WEB-INF/login.jsp").forward(request, response);
When you submit a form, you normally want to use POST:
<form action="login" method="post">
This way the servlet's doPost() will be invoked and you can do any postprocessing stuff in there (e.g. validation, business logic, login the user, etc).
If there are any errors, then you normally want to forward the request back to the same page and display the errors there next to the input fields and so on. You can use the RequestDispatcher for this.
If a POST is successful, you normally want to redirect the request, so that the request won't be resubmitted when the user refreshes the request (e.g. pressing F5 or navigating back in history).
User user = userDAO.find(username, password);
if (user != null) {
request.getSession().setAttribute("user", user); // Login user.
response.sendRedirect("home"); // Redirects to http://example.com/context/home after succesful login.
} else {
request.setAttribute("error", "Unknown login, please try again."); // Set error.
request.getRequestDispatcher("/WEB-INF/login.jsp").forward(request, response); // Forward to same page so that you can display error.
}
A redirect thus instructs the client to fire a new GET request on the given URL. Refreshing the request would then only refresh the redirected request and not the initial request. This will avoid "double submits" and confusion and bad user experiences. This is also called the POST-Redirect-GET pattern.
See also:
How do servlets work? Instantiation, sessions, shared variables and multithreading
doGet and doPost in Servlets
How perform validation and display error message in same form in JSP?
HttpServletResponse sendRedirect permanent
requestDispatcher - forward() method
When we use the forward method, the request is transferred to another resource within the same server for further processing.
In the case of forward, the web container handles all processing internally and the client or browser is not involved.
When forward is called on the requestDispatcherobject, we pass the request and response objects, so our old request object is present on the new resource which is going to process our request.
Visually, we are not able to see the forwarded address, it is transparent.
Using the forward() method is faster than sendRedirect.
When we redirect using forward, and we want to use the same data in a new resource, we can use request.setAttribute() as we have a request object available.
SendRedirect
In case of sendRedirect, the request is transferred to another resource, to a different domain, or to a
different server for further processing.
When you use sendRedirect, the container transfers the request to the client or browser, so the URL given inside the sendRedirect method is visible as a new request to the client.
In case of sendRedirect call, the old request and response objects are lost because it’s treated as new request by the browser.
In the address bar, we are able to see the new redirected address. It’s not transparent.
sendRedirect is slower because one extra round trip is required, because a completely new request is created and the old request object is lost. Two browser request are required.
But in sendRedirect, if we want to use the same data for a new resource we have to store the data in session or pass along with the URL.
Which one is good?
Its depends upon the scenario for which method is more useful.
If you want control is transfer to new server or context, and it is treated as completely new task, then we go for sendRedirect.
Generally, a forward should be used if the operation can be safely repeated upon a browser reload of the web page and will not affect the result.
Source
The RequestDispatcher interface allows you to do a server side forward/include whereas sendRedirect() does a client side redirect. In a client side redirect, the server will send back an HTTP status code of 302 (temporary redirect) which causes the web browser to issue a brand new HTTP GET request for the content at the redirected location. In contrast, when using the RequestDispatcher interface, the include/forward to the new resource is handled entirely on the server side.
The main important difference between the forward() and sendRedirect() method is that in case of forward(), redirect happens
at server end and not visible to client, but in case of
sendRedirect(), redirection happens at client end and it's visible to
client.
Either of these methods may be "better", i.e. more suitable, depending on what you want to do.
A server-side redirect is faster insofar as you get the data from a different page without making a round trip to the browser. But the URL seen in the browser is still the original address, so you're creating a little inconsistency there.
A client-side redirect is more versatile insofar as it can send you to a completely different server, or change the protocol (e.g. from HTTP to HTTPS), or both. And the browser is aware of the new URL. But it takes an extra back-and-forth between server and client.
SendRedirect() will search the content between the servers. it is slow because it has to intimate the browser by sending the URL of the content. then browser will create a new request for the content within the same server or in another one.
RquestDispatcher is for searching the content within the server i think. its the server side process and it is faster compare to the SendRedirect() method. but the thing is that it will not intimate the browser in which server it is searching the required date or content, neither it will not ask the browser to change the URL in URL tab. so it causes little inconvenience to the user.
Technically redirect should be used either if we need to transfer control to different domain or to achieve separation of task.
For example in the payment application
we do the PaymentProcess first and then redirect to displayPaymentInfo. If the client refreshes the browser only the displayPaymentInfo will be done again and PaymentProcess will not be repeated. But if we use forward in this scenario, both PaymentProcess and displayPaymentInfo will be re-executed sequentially, which may result in incosistent data.
For other scenarios, forward is efficient to use since as it is faster than sendRedirect
Request Dispatcher is an Interface which is used to dispatch the request or response from web resource to the another web resource. It contains mainly two methods.
request.forward(req,res): This method is used forward the request from one web resource to another resource. i.e from one servlet to another servlet or from one web application to another web appliacation.
response.include(req,res): This method is used include the response of one servlet to another servlet
NOTE: BY using Request Dispatcher we can forward or include the request or responses with in the same server.
request.sendRedirect(): BY using this we can forward or include the request or responses across the different servers. In this the client gets a intimation while redirecting the page but in the above process the client will not get intimation
Simply difference between Forward(ServletRequest request, ServletResponse response) and sendRedirect(String url) is
forward():
The forward() method is executed in the server side.
The request is transfer to other resource within same server.
It does not depend on the client’s request protocol since the forward () method is provided by the servlet container.
The request is shared by the target resource.
Only one call is consumed in this method.
It can be used within server.
We cannot see forwarded message, it is transparent.
The forward() method is faster than sendRedirect() method.
It is declared in RequestDispatcher interface.
sendRedirect():
The sendRedirect() method is executed in the client side.
The request is transfer to other resource to different server.
The sendRedirect() method is provided under HTTP so it can be used only with HTTP clients.
New request is created for the destination resource.
Two request and response calls are consumed.
It can be used within and outside the server.
We can see redirected address, it is not transparent.
The sendRedirect() method is slower because when new request is created old request object is lost.
It is declared in HttpServletResponse.
I'm trying to write a simple script that fetches text from a webpage and processes that string. But, that website requires me to be logged in. I was successful in logging in to that website. This is how I logged in:
var payload = {"name1":"val1","name2":val2"};
var opt ={"payload":payload,"method":"post"};
var respose = UrlFetchApp.fetch("http://website.com/login",opt);
After logging in, the website places me in http://website.com/home. I checked response.getContentText() and I can confirm that I have been logged in successfully as it contains the text from http://website.com/home.
Now I need to get the contents of http://website.com/page and process it.
I first assumed the script can handle cookies by itself and proceeded with
var pagedata = UrlFetchApp.fetch("http://website.com/page);//Did not work
That obviously didnt work and pagedata.getContentText() says me to login first, which indicates cookies were not successfully passed..
I then tried to extract cookies which the server responded during login and to send it along with this request.
var cookie = response.getAllHeaders()['Set-Cookie'];
// variable cookie now contains a legitimate cookie.
// It contains 'JSESSIONID=blabla;Path=/' and
// it is the ONLY cookie that server responds.
I tried to send that cookie in my page request.
var header = {'Cookie':cookie};
var opt2 = {"header":header};
var pagedata = UrlFetchApp.fetch("http://website.com/page",opt2);
I think even now cookies were not properly sent, as the content again says me to login.
Am I passing cookies correctly? I need help regarding the correct method of sending cookies in a request.
Here you can find cookies specification:
http://www.w3.org/Protocols/rfc2109/rfc2109
You have a potential issue in your code:
response.getAllHeaders()['Set-Cookie'] can return either a string or a table of string if multiple 'set-cookie' attributes are sent back from the server.
Eric is right, you cannot return the cookie without digesting it.
Second error in your code:
var opt2 = {"header":header};
should be
var opt2 = {"headers":header};
Be aware also that GAS uses Google IPs. It can happen that two consecutive fetch use different IPs.
The server your are connecting to may be session-IP dependant.
Are you sure the server only send you back one cookie after an authentification ?
It looks like you are setting the headers correctly in UrlFetchApp.fetch().
I believe that the data in the Set-Cookie header is in a different format than the data that is expected in Cookie header. For example, Set-Cookie contains information about expiration, etc.
I have:
Request request = new Request(Method.GET, "https://www.awebsite.com/login");
Client client = new Client(Protocol.HTTPS);
Response response = client.handle(request);
...
response.getEntity().write(System.out);
But I don't know how to set the login parameters...
I want code that
does the escaping etc
can switch between get/post easily
Being a REST-based platform, I'm thinking I might need to use some parameter "representation" but that seems a bit strange. I'd think it would be common enough to build in this representational exception.
If by "login parameters" you mean sending credentials using Basic HTTP Authentication, it's done using Request.setChallengeResponse() like so:
Request request = new Request(Method.GET, "https://www.awebsite.com/login");
request.setChallengeResponse(new ChallengeResponse(ChallengeScheme.HTTP_BASIC, username, password));
This will work for any Request, using any HTTP method.
If, however, the server to which you're trying to authenticate expects credentials using some protocol other than Basic HTTP Auth, then you'll need to explain that protocol -- i.e. does it use cookies, headers, tokens, etc.
BTW, you might get faster/better responses by posting to the Restlet-Discuss mailing list; I've been on there for a year and a half and it's a great community.