[FIXED] Beginner in web scraping with python: When using BeautifulSoup and finding via class I either get the wrong class or the result "None"

Issue

I want to grab the UCI points from the following website: https://www.procyclingstats.com/rider/tadej-pogacar

To start, I just want to grab the uci point from the latest race.

My code so far is:

import requests
from bs4 import BeautifulSoup
import pandas as pd

page = requests.get("https://www.procyclingstats.com/rider/tadej-pogacar")
soup = BeautifulSoup(page.text, "lxml")
result_date = soup.find("td", class_ = False, id = False, style = False).text
print(result_date)
uci = soup.find("td", class_ = "cu600 ")
print(uci)

When I run the code, I either get the class "gc cu600" or "None" depending if i search for "cu600" og "cu600 ".

Does anybody know what I'm doing wrong?

Solution

Here's how you can get the UCI Points

import requests
import pandas as pd

url = "https://www.procyclingstats.com/rider/tadej-pogacar"

df = pd.concat(pd.read_html(requests.get(url).text, flavor="lxml"))
print(df["Points UCI"].dropna().to_string(index=False))

To get the points from the latest race, just change the last line to:

print(df["Points UCI"].iloc[0])

Output (all points):

   800
    85
   150
   125
    60
     3
   600
  1040
   210
   150
   150
   110
    20
    15
    30
   210
    40
   110
   110
   100
    50
   400
   500
   800
   260
   440
   500
    60
60 +10
   +10
60 +10
  2.86
   200
 20 +5
    +5
 20 +5
 20 +5
   125

Answered By - baduker