Jsoup to parse multiple websites for links published today - web-scraping

Question

I am currently using jsoup (below) to output a .csv of links which include a string date format in the url from just one website.
import java.io.IOException;
import java.util.HashSet;
import java.util.Set;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.select.Elements;
import java.io.FileOutputStream;
import java.io.PrintStream;
import org.jsoup.nodes.Element;
public class readAllLinks {
public static Set<String> uniqueURL = new HashSet<String>();
public static String my_site;
public static String published = ("20180731");
public static void main(String[] args) {
readAllLinks obj = new readAllLinks();
my_site = ("news24.com/SouthAfrica/News");
obj.get_links("https://www.news24.com/SouthAfrica/News/");
}
private void get_links(String url) {
try {
Document doc = Jsoup.connect(url).get();
Elements links = doc.select("a");
FileOutputStream fout=new FileOutputStream("links.csv");
PrintStream csv=new PrintStream(fout);
links.stream().map((link) -> link.attr("abs:href")).forEachOrdered((this_url) -> {
boolean add = uniqueURL.add(this_url);
if (add && this_url.contains(my_site) && this_url.contains(published)) {
System.out.println(this_url);
get_links(this_url);
}
if (this_url.contains(published))
csv.println(this_url);
} );
} catch (IOException ex) {
}
}
}
Instead I would like to make a csv of links published today (i.e using today's date) from multiple websites.
How do you specify the .select for the newly published links to get the date contained in a span?
And how do you parse multiple websites from a list?
Many thanks for your help.

Answer 1

This will select all links that contains value of variable 'published'.
Elements links = doc.select("a[href*="+published+"]");