How to View Data (for Scraping) of Interactive Graph with Hover - web-scraping

Question

I would like to scrape an interactive plot that displays different information based on where the pointer is hovering. This website is what I am interested in:https://embed.chartblocks.com/1.0/?c=60dcd8c53ba0f68e2d162a90&t=44027b4de63d924
I would like to scrape:
Hover green bar for 2011 and get "Credits 6.6B".
Hover blue bar for 2011 and get "Debits 9.48B".
Any suggestion on what I should try? Thank you

Answer 1

Here's a method with python.
import requests
from bs4 import BeautifulSoup
import json
import re
import pandas as pd
url = 'https://embed.chartblocks.com/1.0/?c=60dcd8c53ba0f68e2d162a90&t=44027b4de63d924'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
script =str(soup.find('script'))
jsonStr = re.search(r"(var chartResponse = )({.*);", script).group(2)
jsonData = json.loads(jsonStr)['data']['series']
debits_data = jsonData['ds-0']['values']
credits_data = jsonData['ds-1']['values']
debits_data = [x for idx, x in enumerate(debits_data) if idx%2 == 0]
debits_df = pd.DataFrame(debits_data)
debits_df['type'] = 'Debit'
credits_df = pd.DataFrame(credits_data)
credits_df['type'] = 'Credit'
results_df = debits_df.append(credits_df, sort=False).reset_index(drop=True)
Output:
print(results_df)
y x type
0 9476793016 2011 Debit
1 9789776200 2012 Debit
2 10197864252 2013 Debit
3 10639447839 2014 Debit
4 11262181013 2015 Debit
5 11859026050 2016 Debit
6 12515031664 2017 Debit
7 13437815328 2018 Debit
8 14436466583 2019 Debit
9 15179342495 2020 Debit
10 6602575885 2011 Credit
11 6960716962 2012 Credit
12 7353902696 2013 Credit
13 7658670144 2014 Credit
14 8050922524 2015 Credit
15 8469721067 2016 Credit
16 8974604919 2017 Credit
17 9532204297 2018 Credit
18 10313095156 2019 Credit
19 11611223625 2020 Credit