Issue
Hi I'm having trouble getting my scrapy spider script to login to a aspx (asp.net) website
The script is supposed to crawl a website for product information (it's a suppliers website so we are allowed to do this) but for whatever reason the script is not able to login to the webpage using the script below, there is a username and password field along with a image button but when the script runs it simply doesn't work and we are redirected to the main page... I believe it has something to do with the page being asp.net and apparently i need to pass more information but i've honestly tried everything and im at a loss of what to do next!
What am I doing wrong?
import scrapy
class LeedaB2BSpider(scrapy.Spider):
name = 'leedab2b'
start_urls = [
'https://www.leedab2b.co.uk/customerlogin.aspx'
]
def parse(self, response):
return scrapy.FormRequest.from_response(response=response,
formdata={'ctl00$ContentPlaceHolder1$tbUsername': '[email protected]', 'ctl00$ContentPlaceHolder1$tbPassword': 'yourpassword'},
clickdata={'id': 'ctl00_ContentPlaceHolder1_lbcustomerloginbutton'},
callback=self.after_login)
def after_login(self, response):
self.logger.info("you are at %s" % response.url)
Solution
FormRequest.from_response
doesn't seem to send __EVENTTARGET, __EVENTARGUMENT in formdata, try to add them manually:
formdata={
'__EVENTTARGET': 'ctl00$ContentPlaceHolder1$lbcustomerloginbutton',
'__EVENTARGUMENT': '',
'ctl00$ContentPlaceHolder1$tbUsername': '[email protected]',
'ctl00$ContentPlaceHolder1$tbPassword': 'yourpassword'
}
Answered By - basckerwil
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.