[FIXED] Can't use Selenium to find elements. Try to avoid deprecated commands warning, got result only one element. (on Colab)

Issue

As my title said. To avoid deprecated warning of Selenium.I try to use the new format as I found. and somehow result return only 1 element. How can I fix it to return elements? FYI I scraping website. I'm working on Colab so I can share with my workmates easier.

Here my code

#!pip install selenium
#!pip install IPython
#!pip install pandas
#!apt-get update
#!apt install chromium-chromedriver

from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome('chromedriver',chrome_options=chrome_options)

import pandas as pd
from IPython.display import display
from selenium.webdriver.common.by import By
website = 'https://www.bitkub.com/fee/cryptocurrency'
driver.get(website)

#pull_data
coins = driver.find_elements(By.TAG_NAME,'tr')

checking what element it contains (which somehow got only 1)

print(coins)
[<selenium.webdriver.remote.webelement.WebElement (session="091bac0ea46529043f4d84786ebc705f", element="76396e66-6516-4dd0-9d85-bc66bfad4676")>]

Solution

Ideally coins should have been a list of elements.

However, to extract the name of the coins you need to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

• Using CSS_SELECTOR:

  driver.get('https://www.bitkub.com/fee/cryptocurrency')
  print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "tbody tr td:nth-child(2) span")))])
  

• Using XPATH:

  driver.get('https://www.bitkub.com/fee/cryptocurrency')
  print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//tbody//tr//following::td[2]//span")))])
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

Answered By - undetected Selenium