Beautiful Soup Issue- Can't Find the correct Issue - web-scraping

Question

I am having trouble with Beautiful Soup. I am trying to scrape Kayak, but when I print the length of the find_all it is returning 0. I am using selenium in conjunction with Beautiful Soup as well.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
from bs4 import BeautifulSoup
from webdriver_manager.chrome import ChromeDriverManager
## Kayak URL
origin = "PIT"
destination = "ARN"
startdate = "2022-12-18"
url = "https://www.kayak.com/flights/" + origin + "-" + destination + "/" +\
startdate + "?sort=price_a "
## Setting Up Webdriver
driver = webdriver.Chrome(ChromeDriverManager().install(), options=chrome_options)
driver.implicitly_wait(40)
driver.get(url)
soup = BeautifulSoup(driver.page_source, "lxml")
print(len(soup.find_all("span", attrs={'class': 'depart-time base-time'})))
deptimes = soup.find_all("span", attrs={'class': 'depart-time base-time'})
arrtimes = soup.find_all('span', attrs={'class': 'arrival-time base-time'})
meridies = soup.find_all('span', attrs={'class': 'time-meridiem meridiem'})
This is what I am trying to take from the kayak website.
<span class="depart-time base-time">12:45 </span>

Answer 1

With playwright python you can do something like this, it will be very similar with selenium.
You need to select each box of flight and after iterate each element extracting data that you want.
Here you have an example:
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch(headless=False)
## Kayak URL
origin = "PIT"
destination = "ARN"
startdate = "2022-12-18"
url = "https://www.kayak.com/flights/" + origin + "-" + destination + "/" + \
startdate + "?sort=price_a "
page = browser.new_page()
page.goto(url)
#select all boxes of each flight
allFlight = page.query_selector_all(".resultInner")
#extract data from each flight
for index, flight in enumerate(allFlight, start=1):
print("Flight nº: "+str(index))
print("From " + flight.query_selector(".depart-time").inner_text()
+ flight.query_selector(".depart-time+span").inner_text()
+ " To " + flight.query_selector(".arrival-time").inner_text()
+ flight.query_selector(".arrival-time+span").inner_text())
print("Flight companies: " + flight.query_selector(".section.times .bottom,.multi-airline-names").inner_text().strip())
browser.close()
OUTPUT:
Here you have playwright documentation.
I hope i was able to help you.