XMLHttpRequest with Pentaho Data Integration to get the response header - http

Question

i'm working with Pentaho Data Integration and i need a step which gives me a response header of a http request. What i would really need is an implementatione of the HEAD Method of a http call because i have to see the last-modified field of a resorce on internet. I tried using the REST Client step but it doesn't work. So i would like to write a script that makes everything i need.
I actually created a Modified Java Script Value and i put this code inside:
var req = new XMLHttpRequest();
req.open('GET', 'URL', false);
req.send(null);
var headers = req.getAllResponseHeaders().toLowerCase();
When i run it it doesn't gives me any error but there's anything in "headers".
Any suggestions? I would appreciate a different solution to

Answer 1

If the URL you're requesting is on a different domain then I can tell you that the server is not sending the correct headers to allow cross-domain resource sharing.
Access-Control-Allow-Origin: *
Without the server sending that header the only way to access any of the response (including headers) is if your script is running on the same domain (and protocol) or if you setup a proxy on your own server to get the data from the remote server.
If I open up the JS console on this page and run:
var req = new XMLHttpRequest();
req.open('GET', 'http://dati.toscana.it/it/storage/f/2012-07-26T160139/intoscana-arte-e-cultura.csv', false);
req.send(null);
console.log(req.getAllResponseHeaders());
I get the following output:
XMLHttpRequest cannot load http://dati.toscana.it/it/storage/f/2012-07-26T160139/intoscana-arte-e-cultura.csv. Origin http://stackoverflow.com is not allowed by Access-Control-Allow-Origin.
NetworkError: A network error occurred.

Answer 2

But if i put an alert in the end:
var req = new XMLHttpRequest();
req.open('GET', 'http://dati.toscana.it/it/storage/f/2012-07-26T160139/intoscana-arte-e-cultura.csv', false);
req.send(null);
var headers = req.getAllResponseHeaders().toLowerCase();
alert(headers);
it gives me the response.
It gives me this too:
Error: Failure
req.send(null);
I'm using Firebug.

Answer 3

I had a similar problem. In my case was to detect a redirection, something that could happens in services that uses third part storage (like Amazon S3). First time I tried with XMLHttpRequest but apparently that is not implemented in the JavaScript available in Kettle, just in browsers. To workaround this, I follow the instructions from http://type-exit.org/adventures-with-open-source-bi/2010/06/using-java-in-pentaho-kettle/ to reference another Java Class, mentioned in http://www.mkyong.com/java/how-to-get-http-response-header-in-java/, inside a Modified Javascript Value Step. The limitation is that you need to do a primary request to get the header, it is not a "just one request" solution.
//Get the location of a link that has a 301 permanent redirection
var pathurl=documentourl; //URL to lookup. It must be a valid address, otherwise will give error
var location; //final location in case of redirect
//fake import of the Java Classes
var HTTPRequest=javax.servlet.http.HttpServletRequest;
var url=java.net.URL;
//open a url connection
var conn= new url(pathurl).openConnection();
//if you yant to watch all the headers uncomment the following lines
//var headerobj= conn.getHeaderFields();
var status=conn.getResponseCode(); //get status code
if (status == 301){ //ie: permanent redirection
location=conn.getHeaderField("Location");
}
else{
//return the same url
location=pathurl;
}
I hope it helps to anyone with this problem!