extract email from craiglists post - web-scraping

Question

Is there a way to find the email from listing on craigslist without the use of selenium
import requests,re
from bs4 import BeautifulSoup as bs
url='https://newyork.craigslist.org/wch/prk/d/hawthorne-10x15-drive-up-storage-unit/7122801839.html' #example url
headers={'User-Agent':'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36'}
res=requests.get(url,headers=headers)
the email changes with each request made (I assume), I tried x=re.findall('(\w{32})',res.text) but it doesn't work

Answer 1

Craigslist fetches the email address by sending a POST request to this special URL:
https://newyork.craigslist.org/contactinfo/nyc/prk/U_ID
The value of this U_ID is 7122801839 in this case (from the URL you provided).
You can replicate this request like this:
from bs4 import BeautifulSoup
import requests
import json
U_ID = "7122801839"
URL = f"https://newyork.craigslist.org/contactinfo/nyc/prk/{U_ID}"
COOKIE_VALUE = "cookie" # Replace this with a valid cookie
HEADERS = {
'Content-Type':'application/x-www-form-urlencoded; charset=UTF-8',
'Accept':'*/*',
'Accept-Language':'en-us',
'Accept-Encoding':'gzip, deflate, br',
'Host':'newyork.craigslist.org',
'Origin':'https',
'User-Agent':'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Safari/605.1.15',
'Connection':'keep-alive',
'Referer':'https',
'Content-Length':'44816',
'Cookie':COOKIE_VALUE,
'X-Requested-With':'XMLHttpRequest',
}
PAYLOAD = {
'MIME Type':'application/x-www-form-urlencoded; charset=UTF-8',
}
response = requests.request(
method='POST',
url=URL,
headers=HEADERS,
data=PAYLOAD
)
html = json.loads(response.text)['replyContent']
soup = BeautifulSoup(html,'html.parser')
email = soup.find(class_='mailapp').get('href')
email = email.split('?subject')[0].replace('mailto:','')
print(email)
Please note that this code won't work without a cookie, so you will need to copy the cookie from your browser.