[FIXED] Scrapy- Finding the correct CSS selectors

Issue

# -*- coding: utf-8 -*-
import scrapy
from ..items import LowesspiderItem


class LowesSpider(scrapy.Spider):
    name = 'lowes'
    #allowed_domains = ['lowes.com']
    start_urls = ['https://www.lowes.com/pd/ZLINE-KITCHEN-BATH-Ducted-Red-Matte-Wall-Mounted-Range-Hood-Common-42-Inch-Actual-42-in/1001440644']

    def parse(self, response):
        items = response.css('.grid-container')
        for product in items:
            item = LowesspiderItem()

        #get product price
            productPrice = product.css('.art-pd-price::text').getall()


            item["productPrice"] = productPrice


            yield item

Here is how I got the selectors:

I'm still a little confused on how to find the correct selectors when looking at the html of a website. I'm finding the top level item by scrolling to the top and finding all items that are highlighted. Then I use the CSS selector extension on google chrome to find the specific elements I want to scrape. The program should be giving me the price back, but I have no results yielded. Any help or guidance would be appreciated!

Solution

So you're missing the location of the store so you can just have to add a location cookie

Answered By - user8786463