Issue
According to this Regex code for getting the next line of a match, the commented line of my code should work after i added " +([^\r\n]+) " to it. But for some reason it isn't working ... I'm new to regex, so any tip is very welcome.
import scrapy
import json
class MlSpider(scrapy.Spider):
url1='https://produto.mercadolivre.com.br/MLB-1304118411-sandalia-feminina-anabela-confortavel-pingente-mac-cod-133-_JM?attributes=COLOR_SECONDARY_COLOR%3AUHJldGE%3D%2CSIZE%3AMzU%3D&quantity=1'
url2='https://www.mercadolivre.com.br/chinelo-kenner-rakka-pretolaranja-36-br-para-adulto-homem/p/MLB19132834?product_trigger_id=MLB19130858&attributes=COLOR%3APreto%2FAzul%2CSIZE%3A36+BR&pdp_filters=category%3AMLB273770%7Cshipping_cost%3Afree&applied_product_filters=MLB19132871&quantity=1'
name = 'detalhador'
start_urls=[url2]
def parse(self, response,**kwargs):
d = response.xpath("//script[contains(., 'window.__PRELOADED_STATE__')]/text()").re_first(r'(?s)window.__PRELOADED_STATE__ = (.+?\});') # This only gets url1, because the following text of the string is in the same line as the string
if not d : # so this was made to get url2 as well
d = response.xpath("//script[contains(., 'window.__PRELOADED_STATE__')]/text()").re_first(r'(?s)window.__PRELOADED_STATE__ = +([^\r\n]+)') #This should get the line bellow the matching string, but i dosent
Solution
The issue is with your regex expression. You are not escaping certain symbols that regex uses internally as parsing tools and directions. Also you are using the literal ' ' space character where there is actually a newline characte immidiately after the =
sign. Using \s
is usually better because it means any whitespace character.
Try using this instead. I already tested and got the results you desire.
d = response.xpath("//script[contains(., 'window.__PRELOADED_STATE__')]/text()"
).re_first(r'window\.__PRELOADED_STATE__\s?\=\s*?(\{.*?\});')
The .{}=
characters are all used as parsing instructions by regex, so they need to be escaped with a \
when you want to use the literal character in your expression.
I also removed the (?s)
from the beginning of your expression, I'm not entirely sure why that was there.
Answered By - Alexander
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.