[FIXED] Invalid expression in XPATH

Issue

I'm trying to scrape from Google Search the first links using Selenium.

from selenium import webdriver
from scrapy import Selector

PATH = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(PATH)

start_url = "https://www.google.com/search?q=Iwtglobal+linkedin"

def parse(link):
    driver.get(link)
    page_source = driver.page_source
    selector = Selector(text=page_source)
    link = selector.xpath('//div(contains[@class="MjjYud"])').extract()
    print(link)

File "src\lxml\etree.pyx", line 1599, in lxml.etree._Element.xpath
File "src\lxml\xpath.pxi", line 305, in lxml.etree.XPathElementEvaluator.__call__
File "src\lxml\xpath.pxi", line 225, in lxml.etree._XPathEvaluatorBase._handle_result
ValueError: XPath error: Invalid expression in //div(contains[@class="MjjYud"]

What am I doing wrong?

Solution

Try this..

link = selector.xpath(//div[contains(@class, 'MjjYud')]).extract()

here div is the selector not a function. Where contains is a function which take two arguments, one is element attribute another is attribute value. You assigned class attribute to a value like @class = foo. which is not correct.

Answered By - Bashar