Webscraping bs4 - sorting resaults from different URLs into table - web-scraping

Question

I have written the script below to scrape a website.
I have left out the URL, if you need this write to me and i will supply it to you.
The current output is kinda messy but it does the job.
Im very novice at scraping so if you have any suggestions on how to improve the scraping it selfe please tell me.
Im looking for help to structure the results into a table that looks like this:
| source | columns... |
| -------- | -------------- |
| url1 | values |
| url2 | values |
Columns: Antal aktier, Börsvärde MSEK, Direktavkastning %, P/E-tal, P/S-tal, etc...
values from data1: 59840000, 5084,00, 0,00,11,11, 0,59, etc...
values from data2: 14532434, 2284,50, 2,70, 9,73, 0,52, etc...
Ides on how to solve this is very welcome.
Script:
import bs4
import requests
import re
from bs4 import BeautifulSoup as bs
URL1 = "XXX"
URL2 ="YYY"
r1 = requests.get(URL1)
r2 = requests.get(URL2)
soup1 = bs(r1.content)
soup2 = bs(r2.content)
data1 = soup1.find_all('dl', attrs= {"class": "border XSText rightAlignText noMarginTop highlightOnHover thickBorderBottom noTopBorder"})
data2 = soup2.find_all('dl', attrs= {"class": "border XSText rightAlignText noMarginTop highlightOnHover thickBorderBottom noTopBorder"})
print(data1[1])
print(data2[1])
Webscraping output:
<dt><span>Antal aktier</span></dt>
<dd><span>59 840 000</span></dd>
<dt><span>Börsvärde MSEK</span></dt>
<dd><span>5 084,00</span></dd>
<dt><span>Direktavkastning %</span></dt>
<dd><span>0,00</span></dd>
<dt><span>P/E-tal</span></dt>
<dd><span>11,11</span></dd>
<dt><span>P/S-tal</span></dt>
<dd><span>0,59</span></dd>
<dt><span>Kurs/eget kapital </span></dt>
<dd><span>2,60</span></dd>
<dt><span>Omsättning/aktie SEK</span></dt>
<dd><span>132,00</span></dd>
<dt><span>Vinst/aktie SEK</span></dt>
<dd><span>6,98</span></dd>
<dt><span>Eget kapital/aktie SEK</span></dt>
<dd><span>29,55</span></dd>
<dt><span>Försäljning/aktie SEK</span></dt>
<dd><span>-</span></dd>
<dt><span>Effektivavkastning %</span></dt>
<dd><span>0,00</span></dd>
<dt><span>Antal ägare hos Avanza</span></dt>
<dd><span>16 041</span></dd>
</dl>
<dl class="border XSText rightAlignText noMarginTop highlightOnHover thickBorderBottom noTopBorder">
<dt><span>Antal aktier</span></dt>
<dd><span>14 532 434</span></dd>
<dt><span>Börsvärde MSEK</span></dt>
<dd><span>2 284,50</span></dd>
<dt><span>Direktavkastning %</span></dt>
<dd><span>2,70</span></dd>
<dt><span>P/E-tal</span></dt>
<dd><span>9,73</span></dd>
<dt><span>P/S-tal</span></dt>
<dd><span>0,52</span></dd>
<dt><span>Kurs/eget kapital </span></dt>
<dd><span>2,73</span></dd>
<dt><span>Omsättning/aktie SEK</span></dt>
<dd><span>303,47</span></dd>
<dt><span>Vinst/aktie SEK</span></dt>
<dd><span>16,16</span></dd>
<dt><span>Eget kapital/aktie SEK</span></dt>
<dd><span>58,34</span></dd>
<dt><span>Försäljning/aktie SEK</span></dt>
<dd><span>-</span></dd>
<dt><span>Effektivavkastning %</span></dt>
<dd><span>2,70</span></dd>
<dt><span>Antal ägare hos Avanza</span></dt>
<dd><span>3 994</span></dd>
</dl>```

Answer 1

There are many ways you can solve this!
Looking at your output tells us that <dt> elements hold the column names and <dd> elements hold the values. So we can iterate through them and append the data to lists.
column_list = []
value_list = []
columns = soup1.find_all('dt')
for col in columns:
column_list.append(col.text.strip()) # strip() removes extra space from the text
values = soup1.find_all('dd')
for val in values:
value_list.append(val.text.strip())
for i in range(len(column_list)):
print(column_list[i] + ': ' + value_list[i])
Now you can use the data in your lists as you wish. It currently gives an output likes this:
Kortnamn: AAPL
ISIN: US0378331005
Marknad: NASDAQ
Bransch: Teknik
Handlas i: USD
Beta: 1,1927
Volatilitet %: 24,99
Belåningsvärde %: 60
Säkerhetskrav %: 150
Superränta: Ja
Blankningsbar: Nej
Antal aktier: 17 001 802 000
Börsvärde MUSD: 2 226 555,99
Direktavkastning %: 0,62
P/E-tal: 38,24
P/S-tal: 8,16
Kurs/eget kapital: 31,05
Omsättning/aktie USD: 16,05
Vinst/aktie USD: 3,42
Eget kapital/aktie USD: 4,25
Försäljning/aktie USD: -
Effektivavkastning %: 0,62
Antal ägare hos Avanza: 34 331