Issue
I'm trying to get all links on a page 'https://www.jumia.com.eg' using scrapy.
The code is like this:
all_categories = response.xpath ('//a')
But I found a lot of missing links in the results.
The count of the results is 242 links.
When I tried Chrome developer tools, I got all the links, the count of the results was 608 with the same selector xpath (//a).
Why doesn't Scarpy get all the links using the mentioned selector while Chrome does?
Solution
It turned out that the problem is because data is loaded using Javascript as Justin commented.
Answered By - Ehab
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.