Scrape supporters name from buymeacoffee website - web-scraping

Question

I am trying to scrape supporters names from this https://www.buymeacoffee.com/singtaousa website.
Currently, I am able to get the total number of supporters using axios and cheerio modules. The problem is I can't figure out how to get the supporters name.
I also tried to search with span, not a single supporters name comes out. Not sure whether my code is wrong or the names are impossible to be retrieved.
Here is my code:
import cheerio from 'cheerio'
import axios from 'axios'
export default async function handler(req, res) {
const { data } = await axios.get('https://www.buymeacoffee.com/singtaousa') // example
const $ = cheerio.load(data)
const count = $('.text-fs-16.av-medium.clr-grey.xs-text-fs-14.mg-t-8').text()
const supporters = []
// to be change
$('span').each((i, element) => {
const name = $(element).text()
supporters.push(name)
})
res.status(200).json({ count, supporters })
}

Answer 1

The names are added by JavaScript, so you need something like puppeteer or any other headless browser runner to get full-fledged script-based page content. Here is an example for your case using puppeteer:
import puppeteer from 'puppeteer';
const browser = await puppeteer.launch();
try {
const [page] = await browser.pages();
await page.goto('https://www.buymeacoffee.com/singtaousa');
const namesMinimum = 20;
const nameSelector = 'div.supp-wrapper span.av-heavy';
const moreSelector = 'button#load-more-recent';
await page.waitForSelector(moreSelector);
while (await page.$$eval(nameSelector, names => names.length) < namesMinimum) {
await Promise.all([
page.click(moreSelector),
page.waitForResponse(
response => response.url().includes('www.buymeacoffee.com')
),
]);
}
const data = await page.evaluate(() => {
const names = Array.from(
document.querySelectorAll('div.supp-wrapper span.av-heavy'),
span => span.innerText,
);
return names;
});
console.log(data);
} catch (err) { console.error(err); } finally { await browser.close(); }

Answer 2

You will need to load all supporters with this method from console or manually because you don't have all of them loaded once:
await document.getElementById("load-more-recent").click();
The request for loading supporters is traceable via network tab of developer tools. After loading all, you can copy a list of names from output of code below. You can change concatenation for your output, or ignore null values, but basically that's working:
var supporters = $("div.supp-wrapper");
var list = [];
for(var i = 0; i < supporters.length; i++){
list.push(supporters[i].querySelectorAll("span.av-heavy")[0].textContent.trim(" "));
}
console.log(list);
this script will result:
(10) ['Amy', 'Wong', 'Someone', 'Someone', 'Someone', 'Emily', 'KWONG Wai Oi Anna', 'Simon wong', 'Elaine Liu', 'Someone']
To get all of the supporters name you need to load all with the click script above. Otherwise you can checkout network tab to use API request.