[FIXED] Unable to scrape the table data from a webpage

Issue

I tried to scrape the table data from the page below, but it gave me errors of "Message: no such element: Unable to locate element".

The code below worked very well yesterday but generate the errors now. Thankful if any help here.enter image description here

import time

import pandas as pd

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()

options.headless = True
driver = webdriver.Chrome(options=options)

driver.get("https://www.binance.com/en/futures/funding-history/0")
time.sleep(5)


table = driver.find_element_by_xpath('//*[@class="rc-table-tbody"]')
data = []
for tr in table.find_elements_by_xpath('tr'):
    columns = tr.find_elements_by_xpath('td')
    data.append({
        'Contract': columns[0].text,
        'Funding Rate': columns[2].text
    })
# Convert lits of dictionaries into a dataframe
df = pd.DataFrame(data)

Solution

Your tbody class name is wrong. It should be //*[@class='bn-table-tbody']

Also, to get to tr and td, your xpath should be .//td and .//td

Code :

driver.get("https://www.binance.com/en/futures/funding-history/0")
time.sleep(5)
table = driver.find_element_by_xpath("//*[@class='bn-table-tbody']")
data = []
for tr in table.find_elements_by_xpath('.//tr'):
    columns = tr.find_elements_by_xpath('.//td')
    print(columns[0].text)
    print(columns[2].text)
    data.append({
        'Contract': columns[0].text,
        'Funding Rate': columns[2].text
    })
# Convert lits of dictionaries into a dataframe
df = pd.DataFrame(data)

Imports :

RAYUSDT Perpetual
-0.0770%
SUSHIUSDT Perpetual
0.0100%
CVCUSDT Perpetual
0.0100%
BTSUSDT Perpetual
0.0100%
HOTUSDT Perpetual
0.0100%
ZRXUSDT Perpetual
0.0100%
QTUMUSDT Perpetual
0.0100%
IOTAUSDT Perpetual
-0.0061%
BTCBUSD Perpetual
0.0100%
WAVESUSDT Perpetual
0.0100%
ADAUSDT Perpetual
-0.0146%
LITUSDT Perpetual
0.0100%
NUUSDT Perpetual
0.0100%
XTZUSDT Perpetual
0.0100%
BNBUSDT Perpetual
0.0000%
AKROUSDT Perpetual
0.0100%
HNTUSDT Perpetual
0.0100%
ETCUSDT Perpetual
-0.0045%
XMRUSDT Perpetual
0.0100%
YFIUSDT Perpetual
0.0100%
FTTBUSD Perpetual
0.0100%
ETHUSDT Perpetual
0.0242%
ALICEUSDT Perpetual
0.0100%
ALPHAUSDT Perpetual
0.0100%
SFPUSDT Perpetual
0.0100%
REEFUSDT Perpetual
-0.0348%
BATUSDT Perpetual
0.0100%
DOGEUSDT Perpetual
0.0100%
RLCUSDT Perpetual
0.0100%
TRXUSDT Perpetual
0.0100%
STORJUSDT Perpetual
0.0100%
SNXUSDT Perpetual
0.0100%
1000XECUSDT Perpetual
0.0100%
AUDIOUSDT Perpetual
0.0100%
XLMUSDT Perpetual
0.0100%
IOTXUSDT Perpetual
0.0100%
NEOUSDT Perpetual
0.0100%
UNFIUSDT Perpetual
0.0100%
SANDUSDT Perpetual
0.0100%
DASHUSDT Perpetual
0.0100%
KAVAUSDT Perpetual
0.0100%
RUNEUSDT Perpetual
0.0100%
CTKUSDT Perpetual
0.0100%
LINKUSDT Perpetual
0.0100%
CELRUSDT Perpetual
0.0100%
RSRUSDT Perpetual
0.0100%
ADABUSD Perpetual
0.0067%
DGBUSDT Perpetual
0.0100%
SKLUSDT Perpetual
0.0100%
RENUSDT Perpetual
0.0100%
TOMOUSDT Perpetual
0.0100%
MTLUSDT Perpetual
0.0100%
LTCUSDT Perpetual
0.0100%
DODOUSDT Perpetual
0.0100%
KSMUSDT Perpetual
0.0100%
EGLDUSDT Perpetual
0.0100%
BNBBUSD Perpetual
0.0000%
VETUSDT Perpetual
0.0100%
ONTUSDT Perpetual
0.0100%
TRBUSDT Perpetual
0.0100%
MANAUSDT Perpetual
0.0100%
COTIUSDT Perpetual
0.0100%
CHRUSDT Perpetual
-0.0117%
BAKEUSDT Perpetual
0.0100%
GRTUSDT Perpetual
0.0100%
FLMUSDT Perpetual
0.0100%
MASKUSDT Perpetual
0.0100%
EOSUSDT Perpetual
0.0100%
OGNUSDT Perpetual
0.0100%
SCUSDT Perpetual
0.0100%
BALUSDT Perpetual
0.0100%
STMXUSDT Perpetual
0.0100%
BTTUSDT Perpetual
0.0100%
LUNAUSDT Perpetual
0.0100%
DENTUSDT Perpetual
0.0100%
KNCUSDT Perpetual
0.0100%
SRMUSDT Perpetual
0.0100%
ENJUSDT Perpetual
0.0100%
C98USDT Perpetual
0.0100%
ZENUSDT Perpetual
0.0100%
ATOMUSDT Perpetual
0.0100%
NEARUSDT Perpetual
0.0100%
SOLBUSD Perpetual
0.0100%
BCHUSDT Perpetual
0.0100%
ATAUSDT Perpetual
0.0100%
IOSTUSDT Perpetual
0.0100%
HBARUSDT Perpetual
0.0100%
ZECUSDT Perpetual
0.0100%
1000SHIBUSDT Perpetual
0.0039%
TLMUSDT Perpetual
0.0100%
BZRXUSDT Perpetual
0.0100%
ETHBUSD Perpetual
0.0100%
GALAUSDT Perpetual
0.0100%
AAVEUSDT Perpetual
-0.0648%
GTCUSDT Perpetual
0.0100%
ALGOUSDT Perpetual
0.0100%
ICPUSDT Perpetual
0.0100%
LRCUSDT Perpetual
0.0100%
AVAXUSDT Perpetual
0.0100%
ARPAUSDT Perpetual
0.0100%
CELOUSDT Perpetual
0.0100%
MATICUSDT Perpetual
0.0100%
1INCHUSDT Perpetual
0.0100%
MKRUSDT Perpetual
0.0100%
THETAUSDT Perpetual
0.0100%
UNIUSDT Perpetual
0.0100%
LINAUSDT Perpetual
0.0100%
ARUSDT Perpetual
0.0100%
RVNUSDT Perpetual
0.0100%
FILUSDT Perpetual
0.0100%
NKNUSDT Perpetual
0.0100%
KLAYUSDT Perpetual
0.0100%
DEFIUSDT Perpetual
0.0100%
COMPUSDT Perpetual
0.0100%
BTCDOMUSDT Perpetual
0.0181%
SOLUSDT Perpetual
0.0100%
BTCUSDT Perpetual
0.0295%
OMGUSDT Perpetual
-0.0784%
ICXUSDT Perpetual
0.0100%
BLZUSDT Perpetual
0.0100%
FTMUSDT Perpetual
0.0100%
YFIIUSDT Perpetual
0.0100%
KEEPUSDT Perpetual
0.0100%
BANDUSDT Perpetual
0.0100%
XRPBUSD Perpetual
0.0100%
DOGEBUSD Perpetual
0.0100%
XRPUSDT Perpetual
0.0100%
SXPUSDT Perpetual
0.0100%
CRVUSDT Perpetual
0.0100%
BELUSDT Perpetual
0.0100%
DOTUSDT Perpetual
0.0100%
XEMUSDT Perpetual
0.0100%
ONEUSDT Perpetual
0.0100%
ZILUSDT Perpetual
0.0100%
AXSUSDT Perpetual
-0.1329%
DYDXUSDT Perpetual
0.0100%
OCEANUSDT Perpetual
0.0100%
CHZUSDT Perpetual
0.0100%
ANKRUSDT Perpetual
0.0072%
CTSIUSDT Perpetual
0.0100%

Answered By - cruisepandey