[FIXED] Identifying html structures with data-v-xxxxxxxx and pressing them using selenium

Issue

Trying to identify a javascript button on a website and press it to extend the page.

The website in question is the tencent appstore after performing a basic search. At the bottom of the page is a button titled "div.load-more-new" where upon pressing will extend the page with more apps.

the html is as follows

<div data-v-33600cb4="" class="load-more-btn-new" style="">
     <a data-v-33600cb4="" href="javascript:void(0);">加载更多
         <i data-v-33600cb4="" class="load-more-icon">
         </i>
     </a>
</div>

At first I thought I could identify the button using BeautifulSoup but all calls to find result as empty.

from selenium import webdriver
import BeautifulSoup
import time

url = 'https://webcdn.m.qq.com/webapp/homepage/index.html#/appSearch?kw=%25E7%2594%25B5%25E5%25BD%25B1'

WebDriver = webdriver.Chrome('/chromedriver')
WebDriver.get(url)
time.sleep(5)

# Find using BeuatifulSoup
soup = BeautifulSoup(WebDriver.page_source,'lxml')
button = soup.find('div',{'class':'load-more-btn-new'})

[0] []

After looking around here, it became apparent that even if I could it in BeuatifulSoup, it would not help in pressing the button. Next I tried to find the element in the driver and use .click()

driver.find_element_by_class_name('div.load-more-btn-new').click()

[1] NoSuchElementException

driver.find_element_by_css_selector('.load-more-btn-new').click()

[2] NoSuchElementException

driver.find_element_by_class_name('a.load-more-new.load-more-btn-new[data-v-33600cb4]').click()

[3] NoSuchElementException

but all return with the same error: 'NoSuchElementException'

Solution

Your selections wont work, cause they do not point on the <a>.

• This one selects by class name and you try to click the <div> that holds your <a>:

  driver.find_element_by_class_name('div.load-more-btn-new').click()
  

• This one is very close but is missing the a in selection:

  driver.find_element_by_css_selector('.load-more-btn-new').click()
  

• This one try to find_element_by_class_name but is a wild mix of tag, attribute and class:

  driver.find_element_by_class_name('a.load-more-new.load-more-btn-new[data-v-33600cb4]').click()
  

---

How to fix?

Select your element more specific and nearly like in your second apporach:

driver.find_element_by_css_selector('.load-more-btn-new a').click()

or

driver.find_element_by_css_selector('a[data-v-33600cb4]').click()

Note:

While working with newer selenium versions you will get DeprecationWarning: find_element_by_ commands are deprecated. Please use find_element()*

from selenium.webdriver.common.by import By
driver.find_element(By.CSS_SELECTOR, '.load-more-btn-new a').click()

Answered By - HedgeHog