[FIXED] Web Scrapping with beautiful soup

Issue

To start, this is my first time using stack overflow! I started my journey yesterday on python and I'm trying to extract the value of some pages automatically.

This is my code

    import requests
    from bs4 import BeautifulSoup
    url = 'https://www.jpg.store/collection/chilledkongs'
    r = requests.get(url)

    soup = BeautifulSoup(r.content, 'html.parser')
    div = soup.find('div', class_ = 'stat-title')
    print(div)

I'm getting nothing and my objective is to get the floor price. Atm is 888

Solution

The floor price is loaded via JavaScript from external source. To get it via requests use next example:

import json
import requests
from bs4 import BeautifulSoup


url = "https://www.jpg.store/collection/chilledkongs"
api_url = "https://server.jpgstoreapis.com/collection/{}/floor"

soup = BeautifulSoup(requests.get(url).content, "html.parser")
data = soup.select_one("#__NEXT_DATA__").contents[0]
data = json.loads(data)
policy_id = data["props"]["pageProps"]["collection"]["policy_id"]
data = requests.get(api_url.format(policy_id)).json()

print(data["floor"] / 1_000_000)

Prints:

888.0

Answered By - Andrej Kesely