[FIXED] Pagination for BeautifulSoup with Selenium in python webscraping

Issue

I'm new to Webscraping and hence to bs4 and selenium too. I was able to retrieve data from the table in Olympics medalist's page. But i have no idea how to obtain the data form the rest of the pages, since the doesn't update it's url according to the page (based from initial tutorials I went thru).

I would like to know how I'd could cycle through the pages in this case.

Edit: Thanks for all the answers. Every answer added a concept for me to get data from a website in different ways. Thanks.

Solution

In order to go to the next page you have to click the next pagination button on the bottom right part of the page.
To do so you will have to scroll the page down and click that page.
For scrolling you will use action_chains class.
So per each page you are going to gather your data and then do this:

next_page_btn = driver.find_element_by_xpath('//li[@class="paginate_button page-item next"]//a')
actions.move_to_element(next_page_btn).perform()
time.sleep(0.5)
next_page_btn.click()

Before that you will have to import

from selenium.webdriver.common.action_chains import ActionChains

and initialize the actions object with

actions = ActionChains(driver)

Answered By - Prophet