Issue
I'm trying to scrape data for a school project but for some reason I can't figure out how to gt the data from the table on this page. I'm testing out in scrapy shell and just not getting any data back. Here's what I'm trying
scrapy shell https://www.psacard.com/pop/baseball-cards/1911/turkey-red-cabinets-t3-t9/51453
table = response.xpath('//*[@id="DataTables_Table_0"]/table/tbody')
this is returning []
I've tried with the entire table class as well like such
table = response.xpath('//*[@class="clear interior-result-table pop-grid items psa table-striped dataTable no-footer dtr-inline"]')
and getting the same []
response
i'm stumped. any ideas?
Solution
All data that you need is in another request. You can find it in developer tools in Chrome, for example. Check https://www.psacard.com/Pop/GetItemTable?headingID=51453&categoryID=20003&isPSADNA=false&pf=0&_=1555080293549
. Even common GET request to this link will give you table rows.
Answered By - vezunchik
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.