Data extraction through webscraping - web-scraping

Question

Can someone please guide me as to how I can go about extracting data from this particular table? I have tried it multiple times but have not succeeded in extracting the required data.
`import requests
from bs4 import BeautifulSoup as bs
import pandas as pd
r = requests.get('https://etfdb.com/etf/ICLN/#fact-sheet', proxies = proxy_support).text
soup = bs(r,'html.parser')
da = soup.find_all('ul', {'class':'list-unstyled'})[0]
n_rows = 0
n_columns = 0
column_names = []
for row in da.find_all('li'):
td_tags = row.find('span')
if len(td_tags) > 0:
n_rows+=1
if n_columns == 0:
n_columns = len(td_tags)
th_tags = row.find_all('a href')
if len(th_tags) > 0 and len(column_names) == 0:
for th in th_tags:
column_names.append(th.get_text())
if len(column_names) > 0 and len(column_names) != n_columns:
raise Exception("Column titles do not match the number of columns")
columns = column_names if len(column_names) > 0 else range(0,n_columns)
df = pd.DataFrame(columns = columns, index= range(0,n_rows))
row_marker = 0
for row in da.find_all('li'):
column_marker = 0
columns = row.find_all('span')
for column in columns:
df.iat[row_marker,column_marker] = columns.get_text()
column_marker += 1
if len(columns) > 0:
row_marker += 1
For the code above I get the following error :
AttributeError: ResultSet object has no attribute 'get_text'. You're
probably treating a list of items like a single item. Did you call
find_all() when you meant to call find()?
Can any one tell me what I am doing wrong?

Answer 1

With bs4 4.7.1. to get first table
import requests
from bs4 import BeautifulSoup as bs
r = requests.get('https://etfdb.com/etf/ICLN/#fact-sheet')
soup = bs(r.content, 'lxml')
items = soup.select('h3:contains(Vitals) + ul li')
for item in items:
print([i.text for i in item.select('span')])
Earlier bs versions
items = soup.select_one('h3 + ul').select('li')
for item in items:
print([i.text for i in item.select('span')])