Get the text of a span in a span using BeautifulSoup - web-scraping

Question

I'm trying to get the City, Country and Region back from the site using Beautiful Soup on this site:
https://www.geodatatool.com/en/?ip=82.47.160.231
(Don't worry that's not my IP; dummy ip)
This is what I'm trying:
url = "https://www.geodatatool.com/en/?ip="+ip
# Getting site's data in plain text..
sourceCode = requests.get(url)
plainText = sourceCode.text
soup = BeautifulSoup(plainText)
tags = soup('span')
# Parsing data.
data_item = soup.body.findAll('div','data-item')
#bold_item = data_item.findAll('span')
for tag in tags:
print(tag.contents)
I just get an array back of all span content. Trying to narrow it down to specifically my needs but that's not happening anytime soon.
Can someone help me out with this?

Answer 1

This should work. Basically we find all divs with class: 'data-item', and then in here we are looking for the 2 spans, where the first span is the city:, country:, etc. and the second span contains the data.
data_items = soup.findAll('div', {'class': 'data-item'})
# Country
country = data_items[2].findAll('span')[1].text.strip()
# City
city = data_items[5].findAll('span')[1].text.strip()
# Region
country = data_items[4].findAll('span')[1].text.strip()
In general this works, but if the website shows different data or orders the data differently per search, we might want to make the code a bit more robust. We can do this by using regex to find the country, city and region fields. The solution to that would look as follows:
# Country
country = soup.find(text=re.compile('country', re.IGNORECASE)).parent.parent.findAll('span')[1].text.strip()
# City
city = soup.find(text=re.compile('city', re.IGNORECASE)).parent.parent.findAll('span')[1].text.strip()
# Region
region = soup.find(text=re.compile('region', re.IGNORECASE)).parent.parent.findAll('span')[1].text.strip()
We try to find the pattern 'country', 'city' or 'region' inside the HTML code. Then grabing their parent 2 times to get the same results as the data_items as in the codeblock before and perform the same operations to get to the answer.

Answer 2

It's easier to do it with css selectors:
data_items = soup.select('div.sidebar-data div.data-item')
targets = ['Country:','City:','Region:']
for item in data_items:
if item.select('span.bold')[0].text in targets:
print(item.select('span.bold')[0].text, item.select('span')[1].text.strip())
Output:
Country: United Kingdom
Region: England
City: Plymouth