Web Scraping Problems - web-scraping

Question

I am having a problem with my Web Scraping Application. I am wanting to return a list of the counties in a state, but I am having a problem only printing the text out. Here it prints all of the elements (being counties) in the selection, but I only want the list of counties (No html stuff, just the contents).
import urllib.request
from bs4 import BeautifulSoup
url = 'http://www.stats.indiana.edu/dms4/propertytaxes.asp'
page = urllib.request.urlopen(url)
soup = BeautifulSoup(page.read(), "html.parser")
counties = soup.find_all(id='Select1')#Works
print(counties)
This returns the text of everything on the web page without the html stuff, which is what I want, but it prints everything on the page:
import urllib.request
from bs4 import BeautifulSoup
url = 'http://www.stats.indiana.edu/dms4/propertytaxes.asp'
page = urllib.request.urlopen(url)
soup = BeautifulSoup(page.read(), "html.parser")
counties = soup.get_text()#works
print(counties)
I was wondering if there was a way to combine the two, but every time I do I am getting error messages. I thought this might work:
counties = soup.find_all(id=’Select1’).get_text()
I keep getting a “has no attribute ‘get_text’”

Answer 1

So what you actually want to do here is find the children (the options) in the select field.
select = soup.find_all(id='Select1')
options = select.findChildren()
for option in options :
print(option.get_text())
BeautifulSoup reference is pretty good. You can look around to find other methods you can use on the tag objects, as well as find options to pass to findChildren.