How to capture data from third party web site? - web-scraping

Question

For example, I would just like to capture the data for the 30 latest events for the scrolling info shown on this URL:
http://hazmat.globalincidentmap.com/home.php#
Any idea how to capture it?

Answer 1

What language are you using? In Java, you can get the page HTML content using something like this:
URL url;
InputStream is = null;
BufferedReader br;
String line;
try {
url = new URL("http://hazmat.globalincidentmap.com/home.php");
is = url.openStream(); // throws an IOException
br = new BufferedReader(new InputStreamReader(is));
while ((line = br.readLine()) != null) {
// Here you need to parse the HTML lines until
//you find something you want, like for example
// "eventdetail.php?ID", and then read the content of
// the <td> tag or whatever you want to do.
}
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
} finally {
try {
if (is != null) is.close();
} catch (IOException ioe) {
}
}
Example in PHP:
$c = curl_init('http://hazmat.globalincidentmap.com/home.php');
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
$html = curl_exec($c);
if (curl_error($c))
die(curl_error($c));
$status = curl_getinfo($c, CURLINFO_HTTP_CODE);
curl_close($c);
And then parse the contents of $html variable.